Zenon Pylyshyn

In _Seeing and Visualizing_, Zenon Pylyshyn argues that seeing is different from thinking and that to see is not, as it may seem intuitively, to create an inner replica of the world. Pylyshyn examines how we see and how we visualize and why the scientific account does not align with the way these processes seem to us "from the inside." In doing so, he addresses issues in vision science, cognitive psychology, philosophy of mind, and cognitive neuroscience. First, Pylyshyn argues that there is a core stage of vision independent from the influence of our prior beliefs and examines how vision can be intelligent and yet essentially knowledge-free. He then proposes that a mechanism within the vision module, called a visual index (or FINST), provides a direct preconceptual connection between parts of visual representations and things in the world, and he presents various experiments that illustrate the operation of this mechanism. He argues that such a deictic reference mechanism is needed to account for many properties of vision, including how mental images attain their apparent spatial character without themselves being laid out in space in our brains. The final section of the book examines the "picture theory" of mental imagery, including recent neuroscience evidence, and asks whether any current evidence speaks to the issue of the format of mental images. This analysis of mental imagery brings together many of the themes raised throughout the book and provides a framework for considering such issues as the distinction between the form and the content of representations, the role of vision in thought, and the relation between behavioral, neuroscientific, and phenomenological evidence regarding mental representations.

In _Things and Places_, Zenon Pylyshyn argues that the process of incrementally constructing perceptual representations, solving the binding problem (determining which properties go together), and, more generally, grounding perceptual representations in experience arise from the nonconceptual capacity to pick out and keep track of a small number of sensory individuals. He proposes a mechanism in early vision that allows us to select a limited number of sensory objects, to reidentify each of them under certain conditions as the same individual seen before, and to keep track of their enduring individuality despite radical changes in their properties -- all without the machinery of concepts, identity, and tenses. This mechanism, which he calls FINSTs (for "Fingers of Instantiation"), is responsible for our capacity to individuate and track several independently moving sensory objects -- an ability that we exercise every waking minute, and one that can be understood as fundamental to the way we see and understand the world and to our sense of space. Pylyshyn examines certain empirical phenomena of early vision in light of the FINST mechanism, including tracking and attentional selection. He argues provocatively that the initial selection of perceptual individuals is our primary nonconceptual contact with the perceptual world (a contact that does not depend on prior encoding of any properties of the thing selected) and then draws upon a wide range of empirical data to support a radical externalist theory of spatial representation that grows out of his indexing theory.

This study investigates a new experimental paradigm called the Modified Traveling Salesman Problem. This task requires subjects to visit once and only once n invisible targets in a 2D display, using a virtual vehicle controlled by the subject. Subjects can only see the directions of the targets from the current location of the vehicle, displayed by a set of oriented segments that can be viewed inside a circular window surrounding the vehicle. Two conditions were compared. In the “allocentric” condition, subjects see the vehicle move across the screen and change orientation under their command. The “egocentric” condition is similar except for how the information is provided: the position and orientation of the vehicle icon remains fixed at the center of the screen and only target directions, as indicated by the oriented segments, change as the subject “moves” the vehicle. The unexpected finding was that this task can be performed, in either condition, for up to 10 targets. We consider two possible strategies that might be used, a location-based strategy and a segment strategy. The location-based strategy relies on spatial memory and attempts to infer the locations of all the targets. The segment strategy is more local and focuses on the directional segments themselves, keeping track of the ones that represent already-visited targets. A number of observations suggest that the segment strategy was used, at least for larger numbers of targets. According to our hypothesis, keeping track of the segments requires one to use indexical reference for associating the segments with their status in the task - given by current status predicates Visited or Not-visited -, perhaps using visual indexes, deictic pointers, or object files.

This paper explores the difference between Connectionist proposals for cognitive a r c h i t e c t u r e a n d t h e s o r t s o f m o d e l s t hat have traditionally been assum e d i n c o g n i t i v e s c i e n c e . W e c l a i m t h a t t h e m a j o r d i s t i n c t i o n i s t h a t , w h i l e b o t h Connectionist and Classical architectures postulate representational mental states, the latter but not the former are committed to a symbol-level of representation, or to a ‘language of thought’: i.e., to representational states that have combinatorial syntactic and semantic structure. Several arguments for combinatorial structure in mental representations are then reviewed. These include arguments based on the ‘systematicity’ of mental representation: i.e., on the fact that cognitive capacities always exhibit certain symmetries, so that the ability to entertain a given thought implies the ability to entertain thoughts with semantically related contents. We claim that such arguments make a powerful case that mind/brain architecture is not Connectionist at the cognitive level. We then consider the possibility that Connectionism may provide an account of the neural (or ‘abstract neurological’) structures in which Classical cognitive architecture is implemented. We survey a n u m b e r o f t h e s t a n d a r d a r g u m e n t s t h a t h a v e b e e n o f f e r e d i n f a v o r o f Connectionism, and conclude that they are coherent only on this interpretation.

Using a novel enumeration task, we examined the encoding of spatial information during subitizing. Observers were shown masked presentations of randomly-placed discs on a screen and were required to mark the perceived locations of these discs on a subsequent blank screen. This provided a measure of recall for object locations and an indirect measure of display numerosity. Observers were tested on three stimulus durations and eight numerosities. Enumeration performance was high for displays containing up to six discs—a higher subitizing range than reported in previous studies. Error in the location data was measured as the distance between corresponding stimulus and response discs. Overall, location errors increased in magnitude with larger numerosities and shorter display durations. When errors were computed as disc distance from display centroid, results suggest a compressed representation by observers. Additionally, enumeration and localization accuracy increased with display regularity.

��In four experiments we address the question whether several visual objects can be selected voluntarily (exogenously) and then tracked in a Multiple Object Tracking paradigm and, if so, whether the selection involves a different process. Experiment 1 showed that items can indeed be selected based on their labels. Experiment 2 showed that to select the complement set to a set that is automatically (exogenously) selected — e.g. to select all objects not flashed — observers require additional time and that given 1080 ms they were able to select and track them as well as those selected automatically. Experiment 3 showed that the additional time needed in the previous experiment cannot be attributed solely to time required to disengage attention from the initially automatic selections. Experiment 4 showed that the added time provides a monotonically greater benefit when there are more targets, suggesting a serial process. These results are discussed in relation to the Visual Index (FINST) theory which assumes that visual indexes are captured by a data-driven process. It is suggested that voluntarily allocated attention can be used to facilitate the automatic attention capture by objects of interest.

In Multiple Object Tracking (MOT), an observer is able to track 4 – 5 objects in a group of otherwise indistinguishable objects that move independently and unpredictably about a display. According to the Visual Indexing Theory (Pylyshyn, 1989), successful tracking requires that target objects be indexed while they are distinct -- before tracking begins. In the typical MOT task, the target objects are briefly flashed resulting in the automatic assignment of indexes. The question arises whether indexes are only assigned automatically or whether they can be assigned voluntarily in a top-down manner. This study compares several ways of specifying which of 8 items are the targets to be tracked. In the Flash condition the target items were flashed, in the Nonflash condition the targets were the items not flashed, and in the Number condition the targets were specified by number (e.g., items numbered 1-4). The results showed no difference between the three conditions, suggesting that tracking was possible with either voluntary or involuntary indexing. The second experiment tested the hypothesis that voluntary indexing is possible only if the target items are visited serially. The conditions were the same as experiment 1 except that the time available for index assignment was too short to allow targets to be visited serially. In this experiment, targets flashed only once (or, in the Numbers condition, remained visible for about 400 ms). The results showed a decrease in tracking performance for the Number condition, but the Flash and the Nonflash conditions did not differ, suggesting that as long as the designation of targets was done rapidly, the observer did not have to visit each target serially in order to index it. These results suggest that indexing can occur both automatically and voluntarily, and without serially visiting them, so long as the items are successfully specified.

Each of the chapters in this volume devotes considerable attention to defining and elaborating the notion of the frame problem-one of the hard problems of artificial intelligence. Not only do the chapters clarify the problems at hand, they shed light on the different approaches taken by those in artificial intelligence and by certain philosophers who have been concerned with related problems in their field. The book should therefore not be read merely as a discussion of the frame problem narrowly conceived, but also as a general analysis of what could be a major challenge to the design of computer systems exhibiting general intelligence.

We present three studies examining whether multiple-object tracking (MOT) benefits from the active inhibition of nontargets, as proposed in (Pylyshyn, 2004). Using a probedot technique, the first study showed poorer probe detection on nontargets than on either the targets being tracked or in the empty space between objects. The second study used a matching nontracking task to control for possible masking of probes, independent of target tracking. The third study examined how localized the inhibition is to individual nontargets. The result of these three studies led to the conclusion that nontargets are subject to a highly localized object-based inhibition. Implications of this finding for the FINST visual index theory are discussed. We suggest that we need to distinguish between the differentiation (or individuation) of enduring token objects and the process of making the objects accessible through indexes, with only the latter being limited to 4 or 5 objects