David Kirsh

This article addresses three main questions: What causes cognitive overload in the workplace? What analytical framework should be used to understand how agents interact with their work environments? How can environments be restructured to improve the cognitive workflow of agents? Four primary causes of overload are identified: too much tasking and interruption, and inadequate workplace infrastructure to help reduce the need for planning, monitoring, reminding, reclassifying information, etc… The first step in reducing the cognitive impact of these causes is to enrich classical frameworks for understanding work environments, such as Newell and Simon’s notion of a task environment, by recognizing that our actual workplace is a superposition of many specific environments – activity spaces – which we slip between. Each has its own cost structure arising from the tools and resources available, including the cognitive strategies and interpretational frameworks of individual agents. These cognitive factors are significant, affecting how easy or difficult it is to perform an action, such as finding a specific paper in a “mess” desk. A few simple examples show how work environments can be redesigned and how restructuring can alter the cost structure of activity spaces.

We present data and argument to show that in Tetris—a real-time interactive video game—certain cognitive and perceptual problems are more quickly, easily, and reliably solved by performing actions in the world rather than by performing computational actions in the head alone. We have found that some translations and rotations are best understood as being used to implement a plan, or to implement a reaction. To substantiate our position we have implemented a computational laboratory that lets us record keystrokes and game situations, as well as allows us to dynamically create situations. Using the data of over 30 subjects playing 6 games, tachistoscopic tests of some of these subjects, and results from our own successful efforts at building expert systems to play Tetris, we show why knowing how to use one’s environment to enhance speed and robustness are important components in skilled play.

A startling amount of intelligent activity can be controlled without reasoning or thought. By tuning the perceptual system to task relevant properties a creature can cope with relatively sophisticated environments without concepts. There is a limit, however, to how far a creature without concepts can go. Rod Brooks, like many ecologically oriented scientists, argues that the vast majority of intelligent behaviour is concept-free. To evaluate this position I consider what special benefits accrue to concept-using creatures. Concepts are either necessary for certain types of perception, learning, and control, or they make those processes computationally simpler. Once a creature has concepts its capacities are vastly multiplied.

It is sometimes argued that if PDP networks can be trained to make correct judgements of grammaticality we have an existence proof that there is enough information in the stimulus to permit learning grammar by inductive means alone. This seems inconsistent superficially with Gold's theorem and at a deeper level with the fact that networks are designed on the basis of assumptions about the domain of the function to be learned. To clarify the issue I consider what we should learn from Gold's theorem, then go on to inquire into what it means to say that knowledge is domain specific. I first try sharpening the intuitive notion of domain specific knowledge by reviewing the alleged difference between processing limitatons due to shartage of resources vs shortages of knowledge. After rejecting different formulations of this idea, I suggest that a model is language specific if it transparently refer to entities and facts about language as opposed to entities and facts of more general mathematical domains. This is a useful but not necessary condition. I then suggest that a theory is domain specific if it belongs to a model family which is attuned in a law-like way to domain regularities. This leads to a comparison of PDP and parameter setting models of language learning. I conclude with a novel version of the poverty of stimulus argument.

The influence of imagery on perception depends on the content of the mental image. Sixty-three students responded to the location of the 2 hands of a clock while visualizing the correct or an incorrect clock. Reaction time was shorter with valid cueing. Could this have resulted from visual acquisition strategies such as planning visual saccades or shifting covert attention? No. in this study, a crucial control condition made participants look at rather than visualize the cue. Acquisition strategies should have affected equally both types of cueing, but we observed that the effect of the visual cue was smaller and limited to a particular subcase in which one expects visual acquisition strategies. Thus, what matters is the similarity of the content of the mental image with the visual scene. In addition, an interaction involving the hand used for responding supports the notion that composite imagery is lateralized.

We designed an experiment to explore the learning effectiveness of three different ways of practicing dance movements. To our surprise we found that partial modeling, called marking in the dance world, is a better method than practicing the complete phrase, called practicing full-out; and both marking and full-out are better methods than practicing by repeated mental simulation. We suggest that marking is a form of practicing a dance phrase aspect-by-aspect. Our results also suggest that prior work on learning by observation and learning by mental practice may not scale up to complex movements.

A complementary strategy can be defined as any organizing activity which recruits external elements to reduce cognitive loads. Typical organizing activities include pointing, arranging the position and orientation of nearby objects, writing things down, manipulating counters, rulers or other artifacts that can encode the state of a process or simplify perception. To illustrate the idea of a complementary strategy, a simple experiment was performed in which subjects were asked to determine the dollar value of collections of coins. In the no-hands condition, subjects were not allowed to touch the coin images or to move their hands in any way. In the hands condition, they were allowed to use their hands and fingers however they liked. Significant improvements in time and number of errors were observed when S's used their hands over when they did not. To explain these facts, a brief account of some commonly observed complementary strategies is presented, and an account of their potential benefits to perception, memory and attention.

The goal of three-dimensional visualization is to present information in such a way that the viewer suspends disbelief and uses the screen imagery the same way as he or she would use an identical, real 3D scene. To do this effectively, programmers employ a variety of 3D depth cues. Our own anecdotal experience says that shadows and stereopsis are two of the best for visualization. The nice thing is that both of these are possible to do in interactive programs. They sacrifice a certain amount of interactive speed, but they are possible.

Metacognition is associated with planning, monitoring, evaluating and repairing performance Designers of elearning systems can improve the quality of their environments by explicitly structuring the visual and interactive display of learning contexts to facilitate metacognition. Typically page layout, navigational appearance, visual and interactivity design are not viewed as major factors in metacognition. This is because metacognition tends to be interpreted as a process in the head, rather than an interactive one. It is argued here, that cognition and metacognition are part of a continuum and that both are highly interactive. The tenets of this view are explained by reviewing some of the core assumptions of the situated and distribute approach to cognition and then further elaborated by exploring the notions of active vision, visual complexity, affordance landscape and cue structure. The way visual cues are structured and the way interaction is designed can make an important difference in the ease and effectiveness of cognition and metacognition. Documents that make effective use of markers such as headings, callouts, italics can improve students’ ability to comprehend documents and ‘plan’ the way they review and process content. Interaction can be designed to improve ‘the proximal zone of planning’ – the look ahead and apprehension of what is nearby in activity space that facilitates decisions. This final concept is elaborated in a discussion of how e-newspapers combine effective visual and interactive design to enhance user control over their reading experience.

We present data and argument to show that in Tetris - a real-time interactive video game - certain cognitive and perceptual problems are more quickly, easily, and reliably solved by performing actions in the world rather than by performing computational actions in the head alone. We have found that some translations and rotations are best understood as using the world to improve cognition. These actions are not used to implement a plan, or to implement a reaction; they are used to change the world in order to simplify the problem-solving task. Thus, we distinguish pragmatic actions ñ actions performed to bring one physically closer to a goal - from epistemic actions - actions performed to uncover information that is hidden or hard to compute mentally. To illustrate the need for epistemic actions, we first develop a standard information-processing model of Tetris-cognition, and show that it cannot explain performance data from human players of the game - even when we relax the assumption of fully sequential processing. Standard models disregard many actions taken by players because they appear unmotivated or superfluous. However, we describe many such actions that are actually taken by players that are far from superfluous, and that play valuable roles in improving human performance. We argue that traditional accounts are limited because they regard action as having a single function: to change the world. By recognizing a second function of action - an epistemic function - we can explain many of the actions that a traditional model cannot. Although, our argument is supported by numerous examples specifically from Tetris, we outline how the one category of epistemic action can be incorporated into theories of action more generally.

Architecture is about to enter its first magical phase: a time when buildings actively co-operate with their inhabitants; when objects know what they are, where they are, what is near them; when social and physical space lose their type coupling; when wall and partitions change with mood and task. As engineers and scientists explore how to digitse the world around us, the classical constraints of design, ruled so long by the physics of space, time, and materials, are starting to crumble. Documents can be laid down in one place, automatically cloned, and a copy picked up in another. Meetings scheduled for 9am to 10am can be joined by latecomers at noon, who then participate in a captured form of the event and are ‘edited into’ the past. People on the West coast of the USA can participate, in a telepresent way, with their colleagues on the East coast, and hold a meeting against a virtual backdrop, such as a production line in their Taiwanese factory. Walls seem to dematerialise, remote objects can be touched virtually, shaped, passed through one another. Technology is moving inexorably so that being in one place at a time no longer need dominate how we work and play. Material boundaries are losing their meaning, and interface and information space are catch words that architects must master. In this article I will discuss some of the theoretical ideas shaping our new conception of form, function, and interactivity. My view is that of a cognitive scientist interested in how cognition is distributed throughout our environment. Since the ground rules defining the structure of environments are changing, our very idea of how we are embedded in the world is changing. Architecture is at a new frontier.