David Kirsh

The influence of imagery on perception depends on the content of the mental image. Sixty-three students responded to the location of the 2 hands of a clock while visualizing the correct or an incorrect clock. Reaction time was shorter with valid cueing. Could this have resulted from visual acquisition strategies such as planning visual saccades or shifting covert attention? No. in this study, a crucial control condition made participants look at rather than visualize the cue. Acquisition strategies should have affected equally both types of cueing, but we observed that the effect of the visual cue was smaller and limited to a particular subcase in which one expects visual acquisition strategies. Thus, what matters is the similarity of the content of the mental image with the visual scene. In addition, an interaction involving the hand used for responding supports the notion that composite imagery is lateralized.

A videographic study of origami is presented in which subjects were observed making four different origami objects under five modes of instruction: photos + captions, illustrations-only, illustrations with small captions, illustrations with large captions, and text-only as control. The objective of the study was to explore the gestures and other actions that subjects produce as they try to follow instructions rather than to determine the most effective style of instruction per se. We found that the task of situating instructions to the context at hand is error prone and that to facilitate it subjects gesture, point, re-orient illustrations, and generally do things that have no function other than to change the epistemic and interactive landscape of activity so they can more easily understand what is to be done. These studies bear on the new questions designers are asking about the placement, timing, and pace of instructions that digital aids now provide and on the fundamental question of how humans embed themselves in an activity by framing their task in a situation specific manner.

Cognitive organisms have three rather different techniques for intelligently regulating their intake of environmental information. In order of the time needed to uncover information they are: 1. control of attention: within an image produced by a given sensor certain elements can be selected for additional processing; 2. control of gaze: the orientation and resolution (center of foveation) of the sensor can be regulated to create a new image; 3. control of activity: certain non-perceptual actions can be performed to increase the probability of unearthing salient information that currently is unavailable, hard to detect, or hard to compute.

Using the perspective of situated cognition we studied how people interact with a physical map to help them navigate through an unfamiliar environment. The study used a mixture of cognitive ethnography and traditional experimental methods. We found that the difference between high and low performing navigators showed up in the speed they completed their task and also in the way they use maps. High performers plan routes using a survey method whereas low performers use a route strategy. We suggest that when people are given a task that does not match their cognitive style they try to transform the task to better suit their cognitive abilities and cognitive style.

Video data from three large captures of choreographic dance making was analyzed to determine if there is a difference between participant knowledge – the knowledge an agent acquires by being the cause of an action – and observer knowledge – the knowledge an observer acquires through close attention to someone else’s performance. The idea that there might be no difference has been challenged by recent findings about the action observation network and tacitly challenged by certain tenets in enactive perception. We explored why a choreographer ‘riff’s’ when appropriating and evaluating the movements of his dancers. By recruiting his body to help him cognize he is able to understand the possibilities of movement better than observation. He acquires participant knowledge.

Mechanisms for QoS provisioning in communication networks range from flow-based resource reservation schemes, providing QoS guarantees, through QoS differentiation based on reservation aggregation techniques to adaptation of applications, compensating for incomplete reservations. Scalable, aggregation-based reservations can also be combined with adaptations for a more flexible and robust overall QoS provisioning. Adaptation is particularly important in wireless networks, where reservations schemes are more difficult to realize. It is widely accepted that usability of Cellular or Mobile IP can be largely improved if adaptation is incorporated. Adaptation, in turn, depends on coding techniques involved and on user requirements. Those impose certain preferences and ordering relations on QoS hierarchies or degradation paths to be followed along when adaptation is performed. With the advent of MPEG4 coding techniques, more sophisticated adaptation schemes are more likely to be supported in the future.

This paper introduces the concept of Adaptive Rooms, which are virtual environments able to dynamically adapt to users’ needs, including ‘physical’ and cognitive workflow requirements, number of users, differing cognitive abilities and skills. Adaptive rooms are collections of virtual objects, many of them self-transforming objects, housed in an architecturally active room with information spaces and tools. An ontology of objects used in adap- tive rooms is presented. Virtual entities are classified as passive, reactive, ac- tive, and information entities, and their sub-categories. Only active objects can be self-transforming. Adaptive Rooms are meant to combine the insights of ubiquitous computing -- that computerization should be everywhere, transpar- ently incorporated -- with the insights of augmented reality -- that everyday ob- jects can be digitally enhanced to carry more information about their use. To display the special potential of adaptive rooms, concrete examples are given to show how the demands of cognitive workflow can be reduced.

The goal of three-dimensional visualization is to present information in such a way that the viewer suspends disbelief and uses the screen imagery the same way as he or she would use an identical, real 3D scene. To do this effectively, programmers employ a variety of 3D depth cues. Our own anecdotal experience says that shadows and stereopsis are two of the best for visualization. The nice thing is that both of these are possible to do in interactive programs. They sacrifice a certain amount of interactive speed, but they are possible.

A startling amount of intelligent activity can be controlled without reasoning or thought. By tuning the perceptual system to task relevant properties a creature can cope with relatively sophisticated environments without concepts. There is a limit, however, to how far a creature without concepts can go. Rod Brooks, like many ecologically oriented scientists, argues that the vast majority of intelligent behaviour is concept-free. To evaluate this position I consider what special benefits accrue to concept-using creatures. Concepts are either necessary for certain types of perception, learning, and control, or they make those processes computationally simpler. Once a creature has concepts its capacities are vastly multiplied.

This paper introduces the concept of Adaptive Rooms, which are virtual environments able to dynamically adapt to users’ needs, including ‘physical’ and cognitive workflow requirements, number of users, differing cognitive abilities and skills. Adaptive rooms are collections of virtual objects, many of them self-transforming objects, housed in an architecturally active room with information spaces and tools. An ontology of objects used in adap- tive rooms is presented. Virtual entities are classified as passive, reactive, ac- tive, and information entities, and their sub-categories. Only active objects can be self-transforming. Adaptive Rooms are meant to combine the insights of ubiquitous computing -- that computerization should be everywhere, transpar- ently incorporated -- with the insights of augmented reality -- that everyday ob- jects can be digitally enhanced to carry more information about their use. To display the special potential of adaptive rooms, concrete examples are given to show how the demands of cognitive workflow can be reduced.

Humans are closely coupled with their environments. They rely on being ‘embedded’ to help coordinate the use of their internal cognitive resources with external tools and resources. Consequently, everyday cognition, even cognition in the absence of others, may be viewed as partially distributed. As cognitive scientists our job is to discover and explain the principles governing this distribution: principles of coordination, externalization, and interaction. As designers our job is to use these principles, especially if they can be converted to metrics, in order to invent and evaluate candidate designs. After discussing a few principles of interaction and embedding I discuss the usefulness of a range of metrics derived from economics, computational complexity, and psychology.

In the course of daily life we solve problems often enough that there is a special term to characterize the activity and the right to expect a scientific theory to explain its dynamics. The classical view in psychology is that to solve a problem a subject must frame it by creating an internal representation of the problem‘s structure, usually called a problem space. This space is an internally generable representation that is mathematically identical to a graph structure with nodes and links. The nodes can be annotated with useful information, and the whole representation can be distributed over internal and external structures such as symbolic notations on paper or diagrams. If the representation is distributed across internal and external structures the subject must be able to keep track of activity in the distributed structure. Problem solving proceeds as the subject works from an initial state in this mentally supported space, actively construction possible solution paths, evaluating them and heuristically choosing the best. Control of this exploratory process is not well understood, as it is not always systematic, but various heuristic search algorithms have been proposed and some experimental support has been provided for them.

Metacognition is associated with planning, monitoring, evaluating and repairing performance Designers of elearning systems can improve the quality of their environments by explicitly structuring the visual and interactive display of learning contexts to facilitate metacognition. Typically page layout, navigational appearance, visual and interactivity design are not viewed as major factors in metacognition. This is because metacognition tends to be interpreted as a process in the head, rather than an interactive one. It is argued here, that cognition and metacognition are part of a continuum and that both are highly interactive. The tenets of this view are explained by reviewing some of the core assumptions of the situated and distribute approach to cognition and then further elaborated by exploring the notions of active vision, visual complexity, affordance landscape and cue structure. The way visual cues are structured and the way interaction is designed can make an important difference in the ease and effectiveness of cognition and metacognition. Documents that make effective use of markers such as headings, callouts, italics can improve students' ability to comprehend documents and 'plan' the way they review and process content. Interaction can be designed to improve 'the proximal zone of planning' - the look ahead and apprehension of what is nearby in activity space that facilitates decisions. This final concept is elaborated in a discussion of how e-newspapers combine effective visual and interactive design to enhance user control over their reading experience.

In the scientific study of mind a distinction is drawn between explicit knowledge–knowledge that can be elicited from a subject by suitable inquiry or prompting, can be brought to consciousness, and externally expressed in words–and implicit knowledge–knowledge that cannot be elicited, cannot be made directly conscious, and cannot be articulated. Michael Polanyi (1967) argued that we usually ‘know more than we can say’. The part we can articulate is explicitly known; the part we cannot is implicit.

The objective of research in the foundations of Al is to explore such basic questions as: What is a theory in Al? What are the most abstract assumptions underlying the competing visions of intelligence? What are the basic arguments for and against each assumption? In this essay I discuss five foundational issues: (1) Core Al is the study of conceptualization and should begin with knowledge level theories. (2) Cognition can be studied as a disembodied process without solving the symbol grounding problem. (3) Cognition is nicely described in propositional terms. (4) We can study cognition separately from learning. (5) There is a single architecture underlying virtually all cognition. I explain what each of these implies and present arguments from both outside and inside Al why each has been seen as right or wrong.

A startling amount of intelligent activity can be controlled without reasoning or thought. By tuning the perceptual system to task relevant properties a creature can cope with relatively sophisticated environments without concepts. There is a limit, however, to how far a creature without concepts can go. Rod Brooks, like many ecologically oriented scientists, argues that the vast majority of intelligent behaviour is concept-free. To evaluate this position I consider what special benefits accrue to concept-using creatures. Concepts are either necessary for certain types of perception, learning, and control, or they make those processes computationally simpler. Once a creature has concepts its capacities are vastly multiplied.

Individual creativity is standardly treated as an ‘internalist’ process occurring solely in the head. An alternative, more interactionist view is presented here, where working with objects, media and other external things is seen as a fundamental component of creative thought. The value of chance interaction and chance cueing — practices widely used in the creative arts — is explored briefly in an account of the creative method of choreographer Wayne McGregor and then more narrowly in an experimental study that compared performance on a Scrabble-like word discovery problem. Subjects were presented with seven letters and given two minutes to call out three-to-seven-letter English words. There were three conditions: The tiles were fixed in place, subjects were free to move the tiles manually or the tiles could be randomly shuffled. Results showed that random shuffling was best, with manual movement second. Three reasons are provided: Shuffling is faster and cheaper than mentally thinking of candidates; randomizing strings covers the search space better than a deterministic method based on past successes; and randomizing is equivalent to adding diversity to a team, which is known empirically to lead to more creative solutions.

A complementary strategy can be defined as any organizing activity which recruits external elements to reduce cognitive loads. Typical organizing activities include pointing, arranging the position and orientation of nearby objects, writing things down, manipulating counters, rulers or other artifacts that can encode the state of a process or simplify perception. To illustrate the idea of a complementary strategy, a simple experiment was performed in which subjects were asked to determine the dollar value of collections of coins. In the no-hands condition, subjects were not allowed to touch the coin images or to move their hands in any way. In the hands condition, they were allowed to use their hands and fingers however they liked. Significant improvements in time and number of errors were observed when S's used their hands over when they did not. To explain these facts, a brief account of some commonly observed complementary strategies is presented, and an account of their potential benefits to perception, memory and attention.

To explore the question of physical thinking – using the body as an instrument of cognition – we collected extensive video and interview data on the creative process of a noted choreographer and his company as they made a new dance. A striking case of physical thinking is found in the phenomenon of marking. Marking refers to dancing a phrase in a less than complete manner. Dancers mark to save energy. But they also mark to explore the tempo of a phrase, or its movement sequence, or the intention behind it. Because of its representational nature, marking can serve as a vehicle for thought. Importantly, this vehicle is less complex than the version of the same phrase danced ‘full-out’. After providing evidence for distinguishing different types of marking, three ways of understanding marking as a form of thought are considered: marking as a gestural language for encoding aspects of a target movement, marking as a method of priming neural systems involved in the target movement, and marking as a method for improving the precision of mentally projecting aspects of the target.