Zenon Pylyshyn

In _Seeing and Visualizing_, Zenon Pylyshyn argues that seeing is different from thinking and that to see is not, as it may seem intuitively, to create an inner replica of the world. Pylyshyn examines how we see and how we visualize and why the scientific account does not align with the way these processes seem to us "from the inside." In doing so, he addresses issues in vision science, cognitive psychology, philosophy of mind, and cognitive neuroscience. First, Pylyshyn argues that there is a core stage of vision independent from the influence of our prior beliefs and examines how vision can be intelligent and yet essentially knowledge-free. He then proposes that a mechanism within the vision module, called a visual index (or FINST), provides a direct preconceptual connection between parts of visual representations and things in the world, and he presents various experiments that illustrate the operation of this mechanism. He argues that such a deictic reference mechanism is needed to account for many properties of vision, including how mental images attain their apparent spatial character without themselves being laid out in space in our brains. The final section of the book examines the "picture theory" of mental imagery, including recent neuroscience evidence, and asks whether any current evidence speaks to the issue of the format of mental images. This analysis of mental imagery brings together many of the themes raised throughout the book and provides a framework for considering such issues as the distinction between the form and the content of representations, the role of vision in thought, and the relation between behavioral, neuroscientific, and phenomenological evidence regarding mental representations.

In _Things and Places_, Zenon Pylyshyn argues that the process of incrementally constructing perceptual representations, solving the binding problem (determining which properties go together), and, more generally, grounding perceptual representations in experience arise from the nonconceptual capacity to pick out and keep track of a small number of sensory individuals. He proposes a mechanism in early vision that allows us to select a limited number of sensory objects, to reidentify each of them under certain conditions as the same individual seen before, and to keep track of their enduring individuality despite radical changes in their properties -- all without the machinery of concepts, identity, and tenses. This mechanism, which he calls FINSTs (for "Fingers of Instantiation"), is responsible for our capacity to individuate and track several independently moving sensory objects -- an ability that we exercise every waking minute, and one that can be understood as fundamental to the way we see and understand the world and to our sense of space. Pylyshyn examines certain empirical phenomena of early vision in light of the FINST mechanism, including tracking and attentional selection. He argues provocatively that the initial selection of perceptual individuals is our primary nonconceptual contact with the perceptual world (a contact that does not depend on prior encoding of any properties of the thing selected) and then draws upon a wide range of empirical data to support a radical externalist theory of spatial representation that grows out of his indexing theory.

This study investigates a new experimental paradigm called the Modified Traveling Salesman Problem. This task requires subjects to visit once and only once n invisible targets in a 2D display, using a virtual vehicle controlled by the subject. Subjects can only see the directions of the targets from the current location of the vehicle, displayed by a set of oriented segments that can be viewed inside a circular window surrounding the vehicle. Two conditions were compared. In the “allocentric” condition, subjects see the vehicle move across the screen and change orientation under their command. The “egocentric” condition is similar except for how the information is provided: the position and orientation of the vehicle icon remains fixed at the center of the screen and only target directions, as indicated by the oriented segments, change as the subject “moves” the vehicle. The unexpected finding was that this task can be performed, in either condition, for up to 10 targets. We consider two possible strategies that might be used, a location-based strategy and a segment strategy. The location-based strategy relies on spatial memory and attempts to infer the locations of all the targets. The segment strategy is more local and focuses on the directional segments themselves, keeping track of the ones that represent already-visited targets. A number of observations suggest that the segment strategy was used, at least for larger numbers of targets. According to our hypothesis, keeping track of the segments requires one to use indexical reference for associating the segments with their status in the task - given by current status predicates Visited or Not-visited -, perhaps using visual indexes, deictic pointers, or object files.

This paper explores the difference between Connectionist proposals for cognitive a r c h i t e c t u r e a n d t h e s o r t s o f m o d e l s t hat have traditionally been assum e d i n c o g n i t i v e s c i e n c e . W e c l a i m t h a t t h e m a j o r d i s t i n c t i o n i s t h a t , w h i l e b o t h Connectionist and Classical architectures postulate representational mental states, the latter but not the former are committed to a symbol-level of representation, or to a ‘language of thought’: i.e., to representational states that have combinatorial syntactic and semantic structure. Several arguments for combinatorial structure in mental representations are then reviewed. These include arguments based on the ‘systematicity’ of mental representation: i.e., on the fact that cognitive capacities always exhibit certain symmetries, so that the ability to entertain a given thought implies the ability to entertain thoughts with semantically related contents. We claim that such arguments make a powerful case that mind/brain architecture is not Connectionist at the cognitive level. We then consider the possibility that Connectionism may provide an account of the neural (or ‘abstract neurological’) structures in which Classical cognitive architecture is implemented. We survey a n u m b e r o f t h e s t a n d a r d a r g u m e n t s t h a t h a v e b e e n o f f e r e d i n f a v o r o f Connectionism, and conclude that they are coherent only on this interpretation.

Using a novel enumeration task, we examined the encoding of spatial information during subitizing. Observers were shown masked presentations of randomly-placed discs on a screen and were required to mark the perceived locations of these discs on a subsequent blank screen. This provided a measure of recall for object locations and an indirect measure of display numerosity. Observers were tested on three stimulus durations and eight numerosities. Enumeration performance was high for displays containing up to six discs—a higher subitizing range than reported in previous studies. Error in the location data was measured as the distance between corresponding stimulus and response discs. Overall, location errors increased in magnitude with larger numerosities and shorter display durations. When errors were computed as disc distance from display centroid, results suggest a compressed representation by observers. Additionally, enumeration and localization accuracy increased with display regularity.

��In four experiments we address the question whether several visual objects can be selected voluntarily (exogenously) and then tracked in a Multiple Object Tracking paradigm and, if so, whether the selection involves a different process. Experiment 1 showed that items can indeed be selected based on their labels. Experiment 2 showed that to select the complement set to a set that is automatically (exogenously) selected — e.g. to select all objects not flashed — observers require additional time and that given 1080 ms they were able to select and track them as well as those selected automatically. Experiment 3 showed that the additional time needed in the previous experiment cannot be attributed solely to time required to disengage attention from the initially automatic selections. Experiment 4 showed that the added time provides a monotonically greater benefit when there are more targets, suggesting a serial process. These results are discussed in relation to the Visual Index (FINST) theory which assumes that visual indexes are captured by a data-driven process. It is suggested that voluntarily allocated attention can be used to facilitate the automatic attention capture by objects of interest.

It is generally accepted that there is something special about reasoning by using mental images. The question of how it is special, however, has never been satisfactorily spelled out, despite more than thirty years of research in the post-behaviorist tradition. This article considers some of the general motivation for the assumption that entertaining mental images involves inspecting a picture-like object. It sets out a distinction between phenomena attributable to the nature of mind to what is called the cognitive architecture, and ones that are attributable to tacit knowledge used to simulate what would happen in a visual situation. With this distinction in mind, the paper then considers in detail the widely held assumption that in some important sense images are spatially displayed or are depictive, and that examining images uses the same mechanisms that are deployed in visual perception. I argue that the assumption of the spatial or depictive nature of images is only explanatory if taken literally, as a claim about how images are physically instantiated in the brain, and that the literal view fails for a number of empirical reasons – for example, because of the cognitive penetrability of the phenomena cited in its favor. Similarly, while it is arguably the case that imagery and vision involve some of the same mechanisms, this tells us very little about the nature of mental imagery and does not support claims about the pictorial nature of mental images. Finally, I consider whether recent neuroscience evidence clarifies the debate over the nature of mental images. I claim that when such questions as whether images are depictive or spatial are formulated more clearly, the evidence does not provide support for the picture-theory over a symbol-structure theory of mental imagery. Even if all the empirical claims were true, they do not warrant the conclusion that many people have drawn from them: that mental images are depictive or are displayed in some (possibly cortical) space. Such a conclusion is incompatible with what is known about how images function in thought. We are then left with the provisional counterintuitive conclusion that the available evidence does not support rejection of what I call the “null hypothesis”; namely, that reasoning with mental images involves the same form of representation and the same processes as that of reasoning in general, except that the content or subject matter of thoughts experienced as images includes information about how things would look

Assuming that the vehicle of imaginal thought is a spatial model may not be quite as egregious an error as assuming it is a two-dimensional picture, but it represents no less a reification error. Because the model is not a literal physical layout, one is still owed an explanation of why spatial properties hold in the model – whether because of architectural constraints or by stipulation. The difference is like the difference between explaining behavior from a principle and predicting it by looking it up in a list. In the latter case no purpose is being served by calling it a mental model.

A major cognitive framework for individuating, visualizing, and keeping track of different items of knowledge (such as who said what in a conference or what items of data go with what) is the use of real 3D spatial locations. We use space both literally (as in the desktop or office model of data organization) and also figuratively. Examples of the latter includes such techniques as mentally locating different facts and premises in certain imagined spatial loci -- a technique widely used in mnemonic aids, and the use of spatial location in reasoning where so-called "spatial paralogic" provides an important scheme for keeping track of different components of a problem. The use of distinct spatial loci in reasoning and visualizing can be enhanced and its effectiveness in communication increased if distinct spatial locations can be shared. This, of course, happens routinely when people use gestures, pointing, and carving shapes in the air when they converse. Sharing a common workspace and conceptual space is now becoming technologically feasible through the use of interactive multimedia workstations, in which sound and 3D visual locations can be communicated and spatial indicators such as pointing in space can form part of the human-computer interaction.

We previously reported that in the Multiple Object Tracking (MOT) task, which requires tracking several identical targets moving unpredictably among identical nontargets, the nontargets appear to be inhibited, as measured by a probe-dot detection method. The inhibition appears to be local to nontargets and does not extend to the space between objects – dropping off very rapidly away from targets and nontargets. In the present three experiments we show that (1) nontargets that are identical to targets but remain in a fixed location are not inhibited and (2) moving objects that have a different shape from targets are inhibited as much as same-shape nontargets, and (3) nontargets that are on a different depth plane and so are easily filtered out are not inhibited. This is consistent with a taskdependent view of item inhibition wherein nontargets are inhibited if (and only if) they are likely to be mistaken for targets.

The computational view of mind rests on certain intuitions regarding the fundamental similarity between computation and cognition. We examine some of these intuitions and suggest that they derive from the fact that computers and human organisms are both physical systems whose behavior is correctly described as being governed by rules acting on symbolic representations. Some of the implications of this view are discussed. It is suggested that a fundamental hypothesis of this approach is that there is a natural domain of human functioning that can be addressed exclusively in terms of a formal symbolic or algorithmic vocabulary or level of analysis. Much of the paper elaborates various conditions that need to be met if a literal view of mental activity as computation is to serve as the basis for explanatory theories. The coherence of such a view depends on there being a principled distinction between functions whose explanation requires that we posit internal representations and those that we can appropriately describe as merely instantiating causal physical or biological laws. In this paper the distinction is empirically grounded in a methodological criterion called the " cognitive impenetrability condition." Functions are said to be cognitively impenetrable if they cannot be influenced by such purely cognitive factors as goals, beliefs, inferences, tacit knowledge, and so on. Such a criterion makes it possible to empirically separate the fixed capacities of mind from the particular representations and algorithms used on specific occasions. In order for computational theories to avoid being ad hoc, they must deal effectively with the "degrees of freedom" problem by constraining the extent to which they can be arbitrarily adjusted post hoc to fit some particular set of observations. This in turn requires that the fixed architectural function and the algorithms be independently validated. It is argued that the architectural assumptions implicit in many contemporary models run afoul of the cognitive impenetrability condition, since the required fixed functions are demonstrably sensitive to tacit knowledge and goals. The paper concludes with some tactical suggestions for the development of computational cognitive theories

Marr (1982) may have been one of the rst vision researchers to insist that in modeling vision it is important to separate the location of visual features from their type. He argued that in early stages of visual processing there must be “place tokens” that enable subsequent stages of the visual system to treat locations independent of what specic feature type was at that location. Thus, in certain respects a collinear array of diverse features could still be perceived as a line, and under certain conditions could function as such in perceptual phenomena like the Poggendorf illusion