Zenon Pylyshyn

I’m one of those who is awed and impressed by the potential of this field and have devoted some part of my energy to persuading people that it is a positive force. I have done so largely on the grounds of its economic benefits and it potential for making the fruits of computer technology more generally available to the public — for example, to help the overworked physician; to search for oil and minerals and help manage our valuable resources; to explore, mine, and experimentindangerousenvironments;toallowthenon-computingpublicaccesstovast libraries of important information and even advice, and in the process give real meaning to the..

Using the Multiple Object Tracking (MOT) task, Pylyshyn & Leonard (VSS03) showed that a small brief probe dot was detected more poorly when it occurred on a nontarget than when it occurred either on a target or in the space between items, suggesting that moving nontarget items were inhibited. Here we generalize this finding by comparing probe detection performance against a baseline condition in which no tracking was required. We examined both a baseline condition in which objects did not move and one in which they moved exactly as in the probe detection task, but without tracking. Detection in the nonmoving control was essentially error-free, but probe detection performance in the moving (non-tracking) control task did depend on the probe location. Nonetheless, the findings reported earlier (worse detection on nontargets than anywhere else) remained after we took account of the baseline performance (using several different baseline comparison methods). Using this new baseline-control method we also showed that inhibition does not spread more than about 1.3 degrees of visual angle from the nontargets as the latter move during a tracking trial. In the present report we also describe some preliminary studies of conditions under which enhancement of targets, as well as inhibition of nontargets, may be obtained. Increasing the number and type of nontargets appeared to improve the detection of probes on targets, relative to baseline. However, we failed to find evidence that potentially distracting nontarget objects were inhibited more than clearly task-irrelevant objects, as would be expected if inhibition were a top-down process applied in order to improve performance on the main tracking task. For example, using the comparison with its matching baseline, we showed that task-irrelevant stationary items (which are never confused with targets in the tracking task) appeared to be inhibited even more than nontargets.

In Multiple Object Tracking (MOT), an observer is able to track 4 – 5 objects in a group of otherwise indistinguishable objects that move independently and unpredictably about a display. According to the Visual Indexing Theory (Pylyshyn, 1989), successful tracking requires that target objects be indexed while they are distinct -- before tracking begins. In the typical MOT task, the target objects are briefly flashed resulting in the automatic assignment of indexes. The question arises whether indexes are only assigned automatically or whether they can be assigned voluntarily in a top-down manner. This study compares several ways of specifying which of 8 items are the targets to be tracked. In the Flash condition the target items were flashed, in the Nonflash condition the targets were the items not flashed, and in the Number condition the targets were specified by number (e.g., items numbered 1-4). The results showed no difference between the three conditions, suggesting that tracking was possible with either voluntary or involuntary indexing. The second experiment tested the hypothesis that voluntary indexing is possible only if the target items are visited serially. The conditions were the same as experiment 1 except that the time available for index assignment was too short to allow targets to be visited serially. In this experiment, targets flashed only once (or, in the Numbers condition, remained visible for about 400 ms). The results showed a decrease in tracking performance for the Number condition, but the Flash and the Nonflash conditions did not differ, suggesting that as long as the designation of targets was done rapidly, the observer did not have to visit each target serially in order to index it. These results suggest that indexing can occur both automatically and voluntarily, and without serially visiting them, so long as the items are successfully specified.

A major cognitive framework for individuating, visualizing, and keeping track of different items of knowledge (such as who said what in a conference or what items of data go with what) is the use of real 3D spatial locations. We use space both literally (as in the desktop or office model of data organization) and also figuratively. Examples of the latter includes such techniques as mentally locating different facts and premises in certain imagined spatial loci -- a technique widely used in mnemonic aids, and the use of spatial location in reasoning where so-called "spatial paralogic" provides an important scheme for keeping track of different components of a problem. The use of distinct spatial loci in reasoning and visualizing can be enhanced and its effectiveness in communication increased if distinct spatial locations can be shared. This, of course, happens routinely when people use gestures, pointing, and carving shapes in the air when they converse. Sharing a common workspace and conceptual space is now becoming technologically feasible through the use of interactive multimedia workstations, in which sound and 3D visual locations can be communicated and spatial indicators such as pointing in space can form part of the human-computer interaction.

This study investigates a new experimental paradigm called the Modified Traveling Salesman Problem. This task requires subjects to visit once and only once n invisible targets in a 2D display, using a virtual vehicle controlled by the subject. Subjects can only see the directions of the targets from the current location of the vehicle, displayed by a set of oriented segments that can be viewed inside a circular window surrounding the vehicle. Two conditions were compared. In the “allocentric” condition, subjects see the vehicle move across the screen and change orientation under their command. The “egocentric” condition is similar except for how the information is provided: the position and orientation of the vehicle icon remains fixed at the center of the screen and only target directions, as indicated by the oriented segments, change as the subject “moves” the vehicle. The unexpected finding was that this task can be performed, in either condition, for up to 10 targets. We consider two possible strategies that might be used, a location-based strategy and a segment strategy. The location-based strategy relies on spatial memory and attempts to infer the locations of all the targets. The segment strategy is more local and focuses on the directional segments themselves, keeping track of the ones that represent already-visited targets. A number of observations suggest that the segment strategy was used, at least for larger numbers of targets. According to our hypothesis, keeping track of the segments requires one to use indexical reference for associating the segments with their status in the task - given by current status predicates Visited or Not-visited -, perhaps using visual indexes, deictic pointers, or object files.