Zenon Pylyshyn

Few areas of study have led to such close and intense interactions among computer scientists, psychologists, and philosophers as the area now referred to as cognitive science. Within this discipline, few problems have inspired as much debate as the use of notions such as meaning, intentionality, or the semantic content of mental states in explaining human behavior. The set of problems surrounding these notions have been viewed by some observers as threatening the foundations of cognitive science as currently conceived, and by others as providing a new and scientifically sound formulation of certain classical problems in the philosophy of mind. The chapters in this volume help bridge the gap among contributing disciplines-computer science, philosophy, psychology, neuroscience-and discuss the problems posed from various perspectives.

Using the Multiple Object Tracking (MOT) task, Pylyshyn & Leonard (VSS03) showed that a small brief probe dot was detected more poorly when it occurred on a nontarget than when it occurred either on a target or in the space between items, suggesting that moving nontarget items were inhibited. Here we generalize this finding by comparing probe detection performance against a baseline condition in which no tracking was required. We examined both a baseline condition in which objects did not move and one in which they moved exactly as in the probe detection task, but without tracking. Detection in the nonmoving control was essentially error-free, but probe detection performance in the moving (non-tracking) control task did depend on the probe location. Nonetheless, the findings reported earlier (worse detection on nontargets than anywhere else) remained after we took account of the baseline performance (using several different baseline comparison methods). Using this new baseline-control method we also showed that inhibition does not spread more than about 1.3 degrees of visual angle from the nontargets as the latter move during a tracking trial. In the present report we also describe some preliminary studies of conditions under which enhancement of targets, as well as inhibition of nontargets, may be obtained. Increasing the number and type of nontargets appeared to improve the detection of probes on targets, relative to baseline. However, we failed to find evidence that potentially distracting nontarget objects were inhibited more than clearly task-irrelevant objects, as would be expected if inhibition were a top-down process applied in order to improve performance on the main tracking task. For example, using the comparison with its matching baseline, we showed that task-irrelevant stationary items (which are never confused with targets in the tracking task) appeared to be inhibited even more than nontargets.

It is argued that the traditional distinction between artificial intelligence and cognitive simulation amounts to little more than a difference in style of research - a different ordering in goal priorities and different methodological allegiances. Both enterprises are constrained by empirical considerations and both are directed at understanding classes of tasks that are defined by essentially psychological criteria. Because of the different ordering of priorities, however, they occasionally take somewhat different stands on such issues as the power/generality trade-off and on the relevance of the sort of data collected in experimental psychology laboratories

I recently discovered that work I was doing in the laboratory and in theoretical writings was implicitly taking a position on a set of questions that philosophers had been worrying about for much of the past 30 or more years. My clandestine involvement in philosophical issues began when a computer science colleague and I were trying to build a model of geometrical reasoning that would draw a diagram and notice things in the diagram as it drew it (Pylyshyn, Elcock, Marmor, & Sander, 1978). One problem we found we had to face was that if the system discovered a right angle it had no way to tell whether this was the intersection of certain lines it had drawn earlier while constructing a certain figure, and if so which particular lines they were. Moreover, the model had no way of telling whether this particular right angle was identical to some bit of drawing it had earlier encountered and represented as, say, the base of a particular triangle. There was, in other words, no way to determined the identity of an element (I use the term “element” when referring to a graphical unit such as used in experiments. Otherwise when speaking informally I use the term “thing” on the grounds that nobody would mistake that term for a technical theoretical construct. Eventually I end up calling them “Visual Objects” to conform to usage in psychology) at two different times if it was represented differently at those times. This led to some speculation about the need for what we called a “finger” that could be placed at a particular element of interest and that could be used to identify it as particular token thing (the way you might identify a particular feature on paper by labeling it). In general we needed something like a finger that would stay attached to a particular element and could be used to maintain a correspondence between the individual element that was just noticed now and one that had been represented in some fashion at an earlier time. The idea of such fingers (which....

This paper describes a programmatic theory of a process in early vision called indexing. The theory hypothesizes that a small number of primitive indexes are available for individuating, tracking and providing direct access to salient visual objects. We discuss some empirical and theoretical arguments in favor of the proposed index as a resource-limited link between an internal visual representation and objects in the visual world. We argue that this link is needed to explain a large range of properties of vision, including the ability to simultaneously track multiple moving objects, to select a subset of visual items to process, as well as such larger issues as how mental images get their apparent metrical properties and why the world appears stable despite constant retinal motion.

Jacques Mehler was notoriously charitable in embracing a diversity of approaches to science and to the use of many different methodologies. One place where his ecumenism brought the two of us into disagreement is when the evidence of brain imaging was cited in support of different psychological doctrines, such as the picture-theory of mental imagery. Jacques remained steadfast in his faith in the ability of neuroscience data (where the main source of evidence has been from clinical neurology and neuro-imaging) to choose among different psychological positions. I personally have seen little reason for this optimism so Jacques and I frequently found ourselves disagreeing on this issue, though I should add that we rarely disagreed on substantive issues on which we both had views. This particular bone of contention, however, kept us busy at parties and during the many commutes between New York and New Jersey, where Jacques was a frequent visitor at the Rutgers Center for Cognitive Science. Now that I am in a position where he is a captive audience it seems an opportune time to raise the question again

For many of the authors in this volume, this is the second attempt to explore what McCarthy and Hayes (1969) first called the “Frame Problem”. Since the first compendium (Pylyshyn, 1987), nicely summarized here by Ronald Loui, there have been several conferences and books on the topic. Their goals range from providing a clarification of the problem by breaking it down into subproblems (and sometimes declaring the hard subproblems to not be the_ real_ Frame Problem), to providing formal “solutions” to certain aspects of the problem. But more often the message has been that the problem is not solvable except in a piecemeal way in special circumstances by some sort of heuristic approximations. It has sometimes also been said that solving the Frame Problem is not only an unachievable goal, but it is also an unnecessary one since_ humans_ do not solve it either; we simply get along as best we can and deal with the problem of planning in ways that, to use Dennett’s phrase, is “good enough for government work”.

After thirty years of the current “imagery debate,” it appears far from resolved, even though there seems to be a growing acceptance that a cortical display cannot be identified directly with the experienced mental image, nor can it account for the experimental findings on imagery, at least not without additional ad hoc assumptions. The commentaries on the target article range from the annoyed to the supportive, with a surprising number of the latter. In this response I attempt to correct some misreadings of the target article and discuss some of the ideas and evidence introduced by the commentators – much of which I found helpful, even though they do not alter my basic thesis. I also further develop the idea that the spatial character of images may come from the way they are connected to our immediate or immediately-recalled environment (by attention or by visual indexes) and towards which we may orient while we are imaging, thus leaving the alleged spatial properties of images outside the head and freeing image-representations from having to be displayed on any surface.

It is generally accepted that there is something special about reasoning by using mental images. The question of how it is special, however, has never been satisfactorily spelled out, despite more than thirty years of research in the post-behaviorist tradition. This article considers some of the general motivation for the assumption that entertaining mental images involves inspecting a picture-like object. It sets out a distinction between phenomena attributable to the nature of mind to what is called the cognitive architecture, and ones that are attributable to tacit knowledge used to simulate what would happen in a visual situation. With this distinction in mind, the paper then considers in detail the widely held assumption that in some important sense images are spatially displayed or are depictive, and that examining images uses the same mechanisms that are deployed in visual perception. I argue that the assumption of the spatial or depictive nature of images is only explanatory if taken literally, as a claim about how images are physically instantiated in the brain, and that the literal view fails for a number of empirical reasons – for example, because of the cognitive penetrability of the phenomena cited in its favor. Similarly, while it is arguably the case that imagery and vision involve some of the same mechanisms, this tells us very little about the nature of mental imagery and does not support claims about the pictorial nature of mental images. Finally, I consider whether recent neuroscience evidence clarifies the debate over the nature of mental images. I claim that when such questions as whether images are depictive or spatial are formulated more clearly, the evidence does not provide support for the picture-theory over a symbol-structure theory of mental imagery. Even if all the empirical claims were true, they do not warrant the conclusion that many people have drawn from them: that mental images are depictive or are displayed in some (possibly cortical) space. Such a conclusion is incompatible with what is known about how images function in thought. We are then left with the provisional counterintuitive conclusion that the available evidence does not support rejection of what I call the “null hypothesis”; namely, that reasoning with mental images involves the same form of representation and the same processes as that of reasoning in general, except that the content or subject matter of thoughts experienced as images includes information about how things would look

Assuming that the vehicle of imaginal thought is a spatial model may not be quite as egregious an error as assuming it is a two-dimensional picture, but it represents no less a reification error. Because the model is not a literal physical layout, one is still owed an explanation of why spatial properties hold in the model – whether because of architectural constraints or by stipulation. The difference is like the difference between explaining behavior from a principle and predicting it by looking it up in a list. In the latter case no purpose is being served by calling it a mental model.