Zenon Pylyshyn

It is argued that the traditional distinction between artificial intelligence and cognitive simulation amounts to little more than a difference in style of research - a different ordering in goal priorities and different methodological allegiances. Both enterprises are constrained by empirical considerations and both are directed at understanding classes of tasks that are defined by essentially psychological criteria. Because of the different ordering of priorities, however, they occasionally take somewhat different stands on such issues as the power/generality trade-off and on the relevance of the sort of data collected in experimental psychology laboratories

This paper argues that a theory of situated vision, suited for the dual purposes of object recognition and the control of action, will have to provide something more than a system that constructs a conceptual representation from visual stimuli: it will also need to provide a special kind of direct (preconceptual, unmediated) connection between elements of a visual representation and certain elements in the world. Like natural language demonstratives (such as `this' or `that') this direct connection allows entities to be referred to without being categorized or conceptualized. Several reasons are given for why we need such a preconcep- tual mechanism which individuates and keeps track of several individual objects in the world. One is that early vision must pick out and compute the relation among several individual objects while ignoring their properties. Another is that incrementally computing and updating representations of a dynamic scene requires keeping track of token individuals despite changes in their properties or locations. It is then noted that a mechanism meeting these requirements has already been proposed in order to account for a number of disparate empiri- cal phenomena, including subitizing, search-subset selection and multiple object tracking (Pylyshyn et al., Canadian Journal of Experimental Psychology 48(2) (1994) 260). This mechanism, called a visual index or FINST, is brie

Jacques Mehler was notoriously charitable in embracing a diversity of approaches to science and to the use of many different methodologies. One place where his ecumenism brought the two of us into disagreement is when the evidence of brain imaging was cited in support of different psychological doctrines, such as the picture-theory of mental imagery. Jacques remained steadfast in his faith in the ability of neuroscience data (where the main source of evidence has been from clinical neurology and neuro-imaging) to choose among different psychological positions. I personally have seen little reason for this optimism so Jacques and I frequently found ourselves disagreeing on this issue, though I should add that we rarely disagreed on substantive issues on which we both had views. This particular bone of contention, however, kept us busy at parties and during the many commutes between New York and New Jersey, where Jacques was a frequent visitor at the Rutgers Center for Cognitive Science. Now that I am in a position where he is a captive audience it seems an opportune time to raise the question again

People have always wondered how thinking takes place and what thoughts are constructed from. We typically experience our thoughts as involving pictorial (or sensory) contents or as being in words. Although this idea has been enshrined in psychology as the “dual code” theory of reasoning and memory, serious questions have been raised concerning this view. It appears that whatever the form of our thoughts it is unlikely that it is anything like our experience of them. But if thought is not in pictures or words, what form does it take? If we do not sometimes think in words, then what actually goes on when we think by engaging in an “inner dialogue”? And if we do not sometimes think in pictures, what goes on when we reason by creating and examining “mental images”?

The task of tracking a small number (about four or five) visual targets within a larger set of identical items, each of which moves randomly and independently, has been used extensively to study object-based attention. Analysis of this multiple object tracking (MOT) task shows that it logically entails solving the correspondence problem for each target over time, and thus that the individuality of each of the targets must be tracked. This suggests that when successfully tracking objects, observers must also keep track of them as unique individuals. Yet in the present studies we show that observers are poor at recalling the identity of successfully tracked objects (as specified by a unique identifier associated with each target, such as a number or starting location). Studies also show that the identity of targets tends to be lost when they come close together and that this tendency is greater between pairs of targets than between targets and nontargets. The significance of this finding in relation to the multiple object tracking paradigm is discussed.

The target article proposes that visual experience arises when sensorimotor contingencies are exploited in perception. This novel analysis of visual experience fares no better than the other proposals that the article rightly dismisses, and for the same reasons. Extracting invariants may be needed for recognition, but it is neither necessary nor sufficient for having a visual experience. While the idea that vision involves the active extraction of sensorimotor invariants has merit, it does not replace the need for perceptual representations. Vision is not just for the immediate controlling of action; it is also for finding out about the world, from which inferences may be drawn and beliefs changed.

In three experiments, subjects attempted to track multiple items as they moved independently and unpredictably about a display. Performance was not impaired when the items were briefly (but completely) occluded at various times during their motion, suggesting that occlusion is taken into account when computing enduring perceptual objecthood. Unimpaired performance required the presence of accretion and deletion cues along fixed contours at the occluding boundaries. Performance was impaired when items were present on the visual field at the same times and to the same degrees as in the occlusion conditions, but disappeared and reappeared in ways which did not implicate the presence of occluding surfaces (e.g. by imploding and exploding into and out of existence, instead of accreting and deleting along a fixed contour). Unimpaired performance did not require visible occluders (i.e. Michotte’s tunnel effect) or globally consistent occluder positions. We discuss implications of these results for theories of objecthood in visual attention.

Few areas of study have led to such close and intense interactions among computer scientists, psychologists, and philosophers as the area now referred to as cognitive science. Within this discipline, few problems have inspired as much debate as the use of notions such as meaning, intentionality, or the semantic content of mental states in explaining human behavior. The set of problems surrounding these notions have been viewed by some observers as threatening the foundations of cognitive science as currently conceived, and by others as providing a new and scientifically sound formulation of certain classical problems in the philosophy of mind. The chapters in this volume help bridge the gap among contributing disciplines-computer science, philosophy, psychology, neuroscience-and discuss the problems posed from various perspectives.

The target article claimed that although visual apprehension involves all of general cognition, a significant component of vision (referred to as early vision) works independently of cognition and yet is able to provide a surprisingly high level interpretation of visual inputs, roughly up to identifying general shape-classes. The commentators were largely sympathetic, but frequently disagreed on how to draw the boundary, on exactly what early vision delivers, on the role that attention plays, and on how to interpret the neurophysiological data showing top-down effects. A significant number simply asserted that they were not willing to accept any distinction between vision and cognition, and a surprising number even felt that we could never tell for sure, so why bother? Among the topics covered was the relation of cognition and consciousness, the relation of early vision to other modules such as face recognition and language, and the role of natural constraints.

Assuming that the vehicle of imaginal thought is a spatial model may not be quite as egregious an error as assuming it is a two-dimensional picture, but it represents no less a reification error. Because the model is not a literal physical layout, one is still owed an explanation of why spatial properties hold in the model – whether because of architectural constraints or by stipulation. The difference is like the difference between explaining behavior from a principle and predicting it by looking it up in a list. In the latter case no purpose is being served by calling it a mental model.

This commentary argues that the “illusion” to which Wegner refers in The Illusion of Conscious Will is actually the illusion that our conscious experience of mentally causing certain behaviors explains the behavior in question: It is not the subjective experience itself that is illusory, but the implied causal explanation. The experience of “mental effort” is cited as another example of this sort of illusion. Another significant example is the experience that properties of the representation of our mental images are responsible for certain patterns of behavior observed in mental imagery experiments. Examples include the increase in reaction time found when details are reported from smaller images or when attention is switched between different places and features (imagined as further apart than they are) within a single image. These examples illustrate the nature of the “illusion” involved: It is the illusion that certain observed regularities occur because of the content of the experience, as opposed to the converse – that experience has the content it does because of what the person figures out would happen in the imagined situation.

The computational view of mind rests on certain intuitions regarding the fundamental similarity between computation and cognition. We examine some of these intuitions and suggest that they derive from the fact that computers and human organisms are both physical systems whose behavior is correctly described as being governed by rules acting on symbolic representations. Some of the implications of this view are discussed. It is suggested that a fundamental hypothesis of this approach is that there is a natural domain of human functioning that can be addressed exclusively in terms of a formal symbolic or algorithmic vocabulary or level of analysis. Much of the paper elaborates various conditions that need to be met if a literal view of mental activity as computation is to serve as the basis for explanatory theories. The coherence of such a view depends on there being a principled distinction between functions whose explanation requires that we posit internal representations and those that we can appropriately describe as merely instantiating causal physical or biological laws. In this paper the distinction is empirically grounded in a methodological criterion called the " cognitive impenetrability condition." Functions are said to be cognitively impenetrable if they cannot be influenced by such purely cognitive factors as goals, beliefs, inferences, tacit knowledge, and so on. Such a criterion makes it possible to empirically separate the fixed capacities of mind from the particular representations and algorithms used on specific occasions. In order for computational theories to avoid being ad hoc, they must deal effectively with the "degrees of freedom" problem by constraining the extent to which they can be arbitrarily adjusted post hoc to fit some particular set of observations. This in turn requires that the fixed architectural function and the algorithms be independently validated. It is argued that the architectural assumptions implicit in many contemporary models run afoul of the cognitive impenetrability condition, since the required fixed functions are demonstrably sensitive to tacit knowledge and goals. The paper concludes with some tactical suggestions for the development of computational cognitive theories

I recently discovered that work I was doing in the laboratory and in theoretical writings was implicitly taking a position on a set of questions that philosophers had been worrying about for much of the past 30 or more years. My clandestine involvement in philosophical issues began when a computer science colleague and I were trying to build a model of geometrical reasoning that would draw a diagram and notice things in the diagram as it drew it (Pylyshyn, Elcock, Marmor, & Sander, 1978). One problem we found we had to face was that if the system discovered a right angle it had no way to tell whether this was the intersection of certain lines it had drawn earlier while constructing a certain figure, and if so which particular lines they were. Moreover, the model had no way of telling whether this particular right angle was identical to some bit of drawing it had earlier encountered and represented as, say, the base of a particular triangle. There was, in other words, no way to determined the identity of an element (I use the term “element” when referring to a graphical unit such as used in experiments. Otherwise when speaking informally I use the term “thing” on the grounds that nobody would mistake that term for a technical theoretical construct. Eventually I end up calling them “Visual Objects” to conform to usage in psychology) at two different times if it was represented differently at those times. This led to some speculation about the need for what we called a “finger” that could be placed at a particular element of interest and that could be used to identify it as particular token thing (the way you might identify a particular feature on paper by labeling it). In general we needed something like a finger that would stay attached to a particular element and could be used to maintain a correspondence between the individual element that was just noticed now and one that had been represented in some fashion at an earlier time. The idea of such fingers (which....

Each of the chapters in this volume devotes considerable attention to defining and elaborating the notion of the frame problem-one of the hard problems of artificial intelligence. Not only do the chapters clarify the problems at hand, they shed light on the different approaches taken by those in artificial intelligence and by certain philosophers who have been concerned with related problems in their field. The book should therefore not be read merely as a discussion of the frame problem narrowly conceived, but also as a general analysis of what could be a major challenge to the design of computer systems exhibiting general intelligence.