Zenon Pylyshyn

��In four experiments we address the question whether several visual objects can be selected voluntarily (exogenously) and then tracked in a Multiple Object Tracking paradigm and, if so, whether the selection involves a different process. Experiment 1 showed that items can indeed be selected based on their labels. Experiment 2 showed that to select the complement set to a set that is automatically (exogenously) selected — e.g. to select all objects not flashed — observers require additional time and that given 1080 ms they were able to select and track them as well as those selected automatically. Experiment 3 showed that the additional time needed in the previous experiment cannot be attributed solely to time required to disengage attention from the initially automatic selections. Experiment 4 showed that the added time provides a monotonically greater benefit when there are more targets, suggesting a serial process. These results are discussed in relation to the Visual Index (FINST) theory which assumes that visual indexes are captured by a data-driven process. It is suggested that voluntarily allocated attention can be used to facilitate the automatic attention capture by objects of interest.

You might reasonably surmise from the title of this paper that I will be discussing a theory of vision. After all, what is a theory of vision but a theory of how the world is connected to our visual representations? Theories of visual perception universally attempt to give an account of how a proximal stimulus (presumably a pattern impinging on the retina) can lead to a rich representation of a three dimensional world and thence to either the recognition of known objects or to the coordination of actions with visual information. Such theories typically provide an effective (i.e., computable) mapping from a 2D pattern to a representation of a 3D scene, usually in the form of a symbol structure. But such a mapping, though undoubtedly the essential purpose of a theory of vision, leaves at least one serious problem that I intend to discuss here. It is this problem, rather than a theory of vision itself, that is the subject of this talk

We previously reported that in the Multiple Object Tracking (MOT) task, which requires tracking several identical targets moving unpredictably among identical nontargets, the nontargets appear to be inhibited, as measured by a probe-dot detection method. The inhibition appears to be local to nontargets and does not extend to the space between objects – dropping off very rapidly away from targets and nontargets. In the present three experiments we show that (1) nontargets that are identical to targets but remain in a fixed location are not inhibited and (2) moving objects that have a different shape from targets are inhibited as much as same-shape nontargets, and (3) nontargets that are on a different depth plane and so are easily filtered out are not inhibited. This is consistent with a taskdependent view of item inhibition wherein nontargets are inhibited if (and only if) they are likely to be mistaken for targets.

This systematic investigation of computation and mental phenomena by a noted psychologist and computer scientist argues that cognition is a form of computation, that the semantic contents of mental states are encoded in the same general way as computer representations are encoded. It is a rich and sustained investigation of the assumptions underlying the directions cognitive science research is taking. 1 The Explanatory Vocabulary of Cognition 2 The Explanatory Role of Representations 3 The Relevance of Computation 4 The Psychological Reality of Programs: Strong Equivalence 5 Constraining Functional Architecture 6 The Bridge from Physical to Symbolic: Transduction 7 Functional Architecture and Analogue Processes 8 Mental Imagery and Functional Architecture 9 Epilogue: What is Cognitive Science the Science of?

This article defends the claim that a significant part of visual perception (called “early vision”) is impervious to the influence of beliefs, expectations or knowledge. We examine a wide range of empirical evidence that has been cited in support of the continuity of vision and cognition and argue that the evidence either shows within- vision top-down effects, or else the extra-visual effects that are demonstrated occur before the operation of the autonomous early vision system (through the allocation of focal attention) or after the visual system has produced its 3D shape-description (through the intervention of post-visual decision processes).

This paper describes a programmatic theory of a process in early vision called indexing. The theory hypothesizes that a small number of primitive indexes are available for individuating, tracking and providing direct access to salient visual objects. We discuss some empirical and theoretical arguments in favor of the proposed index as a resource-limited link between an internal visual representation and objects in the visual world. We argue that this link is needed to explain a large range of properties of vision, including the ability to simultaneously track multiple moving objects, to select a subset of visual items to process, as well as such larger issues as how mental images get their apparent metrical properties and why the world appears stable despite constant retinal motion.

For many of the authors in this volume, this is the second attempt to explore what McCarthy and Hayes (1969) first called the “Frame Problem”. Since the first compendium (Pylyshyn, 1987), nicely summarized here by Ronald Loui, there have been several conferences and books on the topic. Their goals range from providing a clarification of the problem by breaking it down into subproblems (and sometimes declaring the hard subproblems to not be the_ real_ Frame Problem), to providing formal “solutions” to certain aspects of the problem. But more often the message has been that the problem is not solvable except in a piecemeal way in special circumstances by some sort of heuristic approximations. It has sometimes also been said that solving the Frame Problem is not only an unachievable goal, but it is also an unnecessary one since_ humans_ do not solve it either; we simply get along as best we can and deal with the problem of planning in ways that, to use Dennett’s phrase, is “good enough for government work”.

After thirty years of the current “imagery debate,” it appears far from resolved, even though there seems to be a growing acceptance that a cortical display cannot be identified directly with the experienced mental image, nor can it account for the experimental findings on imagery, at least not without additional ad hoc assumptions. The commentaries on the target article range from the annoyed to the supportive, with a surprising number of the latter. In this response I attempt to correct some misreadings of the target article and discuss some of the ideas and evidence introduced by the commentators – much of which I found helpful, even though they do not alter my basic thesis. I also further develop the idea that the spatial character of images may come from the way they are connected to our immediate or immediately-recalled environment (by attention or by visual indexes) and towards which we may orient while we are imaging, thus leaving the alleged spatial properties of images outside the head and freeing image-representations from having to be displayed on any surface.

In Multiple Object Tracking (MOT), an observer is able to track 4 – 5 objects in a group of otherwise indistinguishable objects that move independently and unpredictably about a display. According to the Visual Indexing Theory (Pylyshyn, 1989), successful tracking requires that target objects be indexed while they are distinct -- before tracking begins. In the typical MOT task, the target objects are briefly flashed resulting in the automatic assignment of indexes. The question arises whether indexes are only assigned automatically or whether they can be assigned voluntarily in a top-down manner. This study compares several ways of specifying which of 8 items are the targets to be tracked. In the Flash condition the target items were flashed, in the Nonflash condition the targets were the items not flashed, and in the Number condition the targets were specified by number (e.g., items numbered 1-4). The results showed no difference between the three conditions, suggesting that tracking was possible with either voluntary or involuntary indexing. The second experiment tested the hypothesis that voluntary indexing is possible only if the target items are visited serially. The conditions were the same as experiment 1 except that the time available for index assignment was too short to allow targets to be visited serially. In this experiment, targets flashed only once (or, in the Numbers condition, remained visible for about 400 ms). The results showed a decrease in tracking performance for the Number condition, but the Flash and the Nonflash conditions did not differ, suggesting that as long as the designation of targets was done rapidly, the observer did not have to visit each target serially in order to index it. These results suggest that indexing can occur both automatically and voluntarily, and without serially visiting them, so long as the items are successfully specified.