Chaz Firestone

For outstanding experimental and theoretical contributions to our understanding of perception and its interface with cognition and motor performance. Chaz Firestone’s creative, elegant, and brilliant studies have time and again enriched our understanding of countless aspects of perceptual representation, including the perception of balance, structure, absences, complexity, possibility, perspectival shape, and abstract relations—revealing just how rich perceptual processing is. His masterful theoretical work, such as his landmark examination of putative cognitive influences on perception, raises the bar for experimental and conceptual rigor in the field. His boundless energy and commitment as a researcher, mentor, and teacher are an inspiration.

“Core knowledge” refers to a set of cognitive systems that underwrite early representations of the physical and social world, appear universally across cultures, and likely result from our genetic endowment. Although this framework is canonically considered as a hypothesis about early-emerging conception — how we think and reason about the world — here we present an alternative view: that many such representations are inherently perceptual in nature. This “core perception” view explains an intriguing (and otherwise mysterious) aspect of core-knowledge processes and representations: that they also operate in adults, where they display key empirical signatures of perceptual processing. We first illustrate this overlap using recent work on “core physics”, the domain of core knowledge concerned with physical objects, representing properties such as persistence through time, cohesion, solidity, and causal interactions. We review evidence that adult vision incorporates exactly these representations of core physics, while also displaying empirical signatures of genuinely perceptual mechanisms, such as rapid and automatic operation on the basis of specific sensory inputs, informational encapsulation, and interaction with other perceptual processes. We further argue that the same pattern holds for other areas of core knowledge, including geometrical, numerical, and social domains. In light of this evidence, we conclude that many infant results appealing to precocious reasoning abilities are better explained by sophisticated perceptual mechanisms shared by infants and adults. Our core-perception view elevates the status of perception in accounting for the origins of conceptual knowledge, and generates a range of ready-to-test hypotheses in developmental psychology, vision science, and more.

The relation between attention, perception, and awareness is among the most fundamental problems in the science of the mind. One of the most striking and well-known phenomena bearing on this question is inattentional blindness (IB). In IB, naive observers fail to report clearly visible stimuli when their attention is otherwise engaged—famously missing a gorilla parading before their eyes. IB carries tremendous significance, both as evidence that awareness requires attention and as a tool in seeking the neural correlates of consciousness. However, such implications rest on a notoriously biased measure: asking participants whether they noticed anything unusual (and interpreting negative answers as reflecting a complete lack of perception). Here, in the largest ever set of IB studies, we show that, as a group, inattentionally blind participants can successfully report the location, color, and shape of stimuli they deny noticing, demonstrating that perceptual information remains accessible in IB. By introducing absent trials, we further show that observers are collectively biased to report not noticing in IB—essentially ‘playing it safe’ in reporting their sensitivity. These data provide the strongest evidence to date of significant residual visual sensitivity in IB. They also challenge the use of inattentional blindness to argue that awareness requires attention.

Objects and events are fundamental units of perception: Objects structure our experience of space, and events structure our experience of time. A striking and counterintuitive finding about object representation is that it can warp perceived space, such that stimuli within an object appear farther apart than stimuli in empty space. Might events influence perceived time in the same way objects influence perceived space? Here, five experiments (N=500 adults) show that they do: Just as stimuli within an object are perceived as farther apart in space, stimuli within an event are perceived as further apart in time. Such “Event-based Warping” is elicited both by events characterized by sound (E1), as well as events characterized by silence (E2). Moreover, these effects cannot be explained by surprise, distraction or attentional-cueing (E3 and E4), and also arise cross-modally (from audition to vision; E5). We suggest that object-based warping and event-based warping are both instances of a more general phenomenon in which representations of structure — whether in space or in time — generate powerful and analogous relative perceptual distortions.

A fundamental question in the psychological sciences is the degree to which culture shapes core cognitive processes — perhaps none more foundational than how we perceive the world around us. A dramatic and oft-cited “case study” of culture’s power in this regard is the Müller-Lyer illusion, which depicts two lines of equal length but with arrowheads pointing either inward or outward, creating the illusion that one line is longer than the other. According to a line of research stretching back over a century, depending on the society you were raised in (and how much carpentry you were exposed to), you may not see the illusion at all — an ambitious and influential research program motivating claims that seemingly basic aspects of visual processing may actually be “culturally evolved byproducts”. This Cultural Byproduct Hypothesis bears on foundational issues in the science, philosophy, and sociology of psychology, and remains popular today. Yet, here we argue that it is almost certainly false. We synthesize evidence from diverse fields which demonstrate that: (1) the illusion is not limited to humans, appearing in non-human animals from diverse ecologies; (2) the statistics of natural scenes are sufficient to capture the illusion; (3) the illusion does not require straight lines typical of carpentry (nor even any lines at all); (4) the illusion arises in sense modalities other than vision; and (5) the illusion arises even in congenitally blind subjects. Moreover, by reexamining historical data and ethnographic descriptions from the original case studies, we show that the evidence for cultural variation and its correlation with key cultural variables is in fact highly inconsistent, beset by questionable research practices, and misreported by later discussions. Together, these considerations undermine the most popular and dramatic example of cultural influence on perception. We further extend our case beyond this phenomenon, showing that many of these considerations apply to other visual illusions as well, including similarly implicated visual phenomena such as the Ebbinghaus, Ponzo, Poggendorf, and Horizontal-Vertical illusions. We conclude by outlining future approaches to cross-cultural research on perception, and we also point to other potential sources of cultural variation in visual processing.

Whereas decades of research have cataloged striking errors in physical reasoning, a resurgence of interest in intuitive physics has revealed humans’ remarkable ability to successfully predict the unfolding of physical scenes. A leading interpretation intended to resolve these opposing results is that physical reasoning recruits a general-purpose mechanism that reliably models physical scenarios (explaining recent successes), but overly contrived tasks or impoverished and ecologically invalid stimuli can produce poor performance (accounting for earlier failures). But might there be tasks that persistently strain physical understanding, even in naturalistic contexts? Here, we explore this question by introducing a new intuitive physics task: evaluating the strength of knots and tangles. Knots are ubiquitous across cultures and time-periods, and evaluating them correctly often spells the difference between safety and peril. Despite this, 5 experiments show that observers fail to discern even very large differences in strength between knots. In a series of two-alternative forced-choice tasks, observers viewed a variety of simple “bends” (knots joining two pieces of thread) and decided which would require more force to undo. Though the strength of these knots is well-documented, observers’ judgments completely failed to reflect these distinctions, across naturalistic photographs (E1), idealized renderings (E2), dynamic videos (E3), and even when accompanied by schematic diagrams of the knots’ structures (E4). Moreover, these failures persisted despite accurate identification of the topological differences between the knots (E5); in other words, even when observers correctly perceived the underlying structure of the knot, they failed to correctly judge its strength. These results expose a blindspot in physical reasoning, placing new constraints on general-purpose theories of scene understanding.

When a piece of fruit is in a bowl, and the bowl is on a table, we appreciate not only the individual objects and their features, but also the relations containment and support, which abstract away from the particular objects involved. Independent representation of roles (e.g., containers vs. supporters) and “fillers” of those roles (e.g., bowls vs. cups, tables vs. chairs) is a core principle of language and higherlevel reasoning. But does such role-filler independence also arise in automatic visual processing? Here, we show that it does, by exploring a surprising error that such independence can produce. In four experiments, participants saw a stream of images containing different objects arranged in forcedynamic relations — e.g., a phone contained in a basket, a marker resting on a garbage can, or a knife sitting in a cup. Participants had to respond to a single target image (e.g., a phone in a basket) within a stream of distractors presented under time constraints. Surprisingly, even though participants completed this task quickly and accurately, they false-alarmed more often to images matching the target’s relational category than to those that did not — even when those images involved completely different objects. In other words, participants searching for a phone in a basket were more likely to mistakenly respond to a knife in a cup than to a marker on a garbage can. Follow-up experiments ruled out strategic responses and also controlled for various confounding image features. We suggest that visual processing represents relations abstractly, in ways that separate roles from fillers.

Science begins with mystery. What causes lightning? How did this mold stop bacterial growth? Why do we age? Arguably, the two greatest mysteries are the cosmos and consciousness—the vast world out there and the vibrant world within. Scientists captivated by one can be called to study the other, seduced by the thought that these mysteries are connected. Science writer George Musser’s book Putting Ourselves Back in the Equation reviews their progress: Can physics unlock the mystery of consciousness? Does consciousness underlie fundamental physics?

Many actions have instrumental aims, in which we move our bodies to achieve a physical outcome in the environment. However, we also perform actions with epistemic aims, in which we move our bodies to acquire information and learn about the world. A large literature on action recognition investigates how observers represent and understand the former class of actions; but what about the latter class? Can one person tell, just by observing another person’s movements, what they are trying to learn? Here, 5 experiments explore epistemic action understanding. We filmed volunteers playing a ‘physics game’ consisting of two rounds: Players shook an opaque box and attempted to determine (i) the number of objects hidden inside, or (ii) the shape of the objects inside. Then, independent subjects watched these videos and were asked to determine which videos came from which round: Who was shaking for number and who was shaking for shape? Across several variations, observers successfully determined what an actor was trying to learn, based only on their actions (i.e., how they shook the box)—even when the box’s contents were identical across rounds. These results demonstrate that humans can infer epistemic intent from physical behaviors, adding a new dimension to research on action understanding.

Embodied approaches to cognition have touched all corners of the mind, including higher‐level judgmental processes such as social evaluation, moral reasoning and theory of mind. After further characterizing and reviewing the evidence for moderately embodied visual perception, the chapter argues that such evidence does not at all support the moderate approach to embodied cognition, even when the relevant studies and accompanying theories are taken at face value. Even if body‐related factors do influence visual perception ‐ and indeed even if spatial perception is sometimes “body‐scaled” ‐ the author shows that the prevailing theories of such body‐based influences understand these effects in ways that exclude them as instances of moderately embodied cognition. The most favorable solution, then, is to accept that body‐based scaling in visual perception of which the “embodied perception” findings are intended as an example ‐ does not meet the moderate approach's criteria for embodiment.

Auditory perception is traditionally conceived as the perception of sounds — a friend’s voice, a clap of thunder, a minor chord. However, daily life also seems to present us with experiences characterized by the absence of sound — a moment of silence, a gap between thunderclaps, the hush after a musical performance. In these cases, do we positively hear silence? Or do we just fail to hear, and merely judge or infer that it is silent? This longstanding question remains controversial in both the philosophy and science of perception, with prominent theories holding that sounds are the only objects of auditory experience and thus that our encounter with silence is cognitive, not perceptual. However, this debate has largely remained theoretical, without a key empirical test. Here, we introduce a novel empirical approach, presenting experimental evidence that silence can be genuinely perceived (not just cognitively inferred). We ask whether silences can ‘substitute’ for sounds in event-based auditory illusions — empirical signatures of auditory event representation in which auditory events distort perceived duration. Seven experiments introduce three new ‘silence illusions’ — the one-silence-is-more illusion, silence-based warping, and the oddball-silence illusion — each adapted from a prominent perceptual illusion previously thought to arise only from sounds. Subjects were immersed in ambient noise interrupted by silences structurally identical to the sounds in the original illusions. In all cases, silences elicited temporal distortions perfectly analogous to the illusions produced by sounds. Our results suggest that silence is truly heard, not merely inferred, introducing a general approach for studying absence perception.

When a circular coin is rotated in depth, is there any sense in which it comes to resemble an ellipse? While this question is at the center of a rich and divided philosophical tradition (with some scholars answering affirmatively and some negatively), Morales et al. (2020, 2021) took an empirical approach, reporting 10 experiments whose results favor such perspectival similarity. Recently, Burge and Burge (2022) offered a vigorous critique of this work, objecting to its approach and conclusions on both philosophical and empirical grounds. Here, we answer these objections on both fronts. We show that Burge and Burge’s critique rests on misunderstandings of Morales et al.’s claims; of the relation between the data and conclusions; and of the philosophical context in which the work appears. Specifically, Burge and Burge attribute to us a much stronger (and stranger) view than we hold, involving the introduction of “a new entity” located “in some intermediate position(s) between the distal shape and the retinal image.” We do not hold this view. Indeed, once properly understood, most of Burge and Burge’s objections favor Morales et al.’s claims rather than oppose them. Finally, we discuss several questions that remain unanswered, and reflect on a productive path forward on these issues of foundational scientific and philosophical interest.

Perception is our primary means of accessing the external world. What is the nature of this core mental process? Although this question is at the center of scientific research on perception, it has also long been explored by philosophers, who ask fundamental questions about our capacity to perceive: Do our different senses represent the world in commensurable ways? How much of our environment can we be aware of at one time? Which aspects of perception are ‘objective’, and which ‘subjective’? What properties count as perceptual in the first place? Although these parallel research programs typically proceed independently in contemporary scholarship, previous eras recognized more active collaboration and interaction across philosophical and scientific approaches to perception. Here, we review an emerging research focus that has begun to reunite these approaches, by putting several long-standing philosophical questions to empirical test. Unlike more general sources of philosophical inspiration, the work described here draws a direct line from a prominent philosophical conjecture or thought experiment about perception to a key test of that question in the laboratory—such that the relevant experimental work would not (and even could not) have proceeded as it did without the preceding philosophical discussion. Finally, we explore themes arising from these successful interactions, and point to further philosophical questions that might be amenable to empirical approaches.

“What is the structure of thought?” is as central a question as any in cognitive science. A classic answer to this question has appealed to a Language of Thought (LoT). We point to emerging research from disparate branches of the field that supports the LoT hypothesis, but also uncovers diversity in LoTs across cognitive systems, stages of development, and species. Our letter formulates open research questions for cognitive science concerning the varieties of rules and representations that underwrite various LoT-based systems and how these variations can help researchers taxonomize cognitive systems

The recent emergence of machine-manipulated media raises an important societal question: How can we know whether a video that we watch is real or fake? In two online studies with 15,016 participants, we present authentic videos and deepfakes and ask participants to identify which is which. We compare the performance of ordinary human observers with the leading computer vision deepfake detection model and find them similarly accurate, while making different kinds of mistakes. Together, participants with access to the model’s prediction are more accurate than either alone, but inaccurate model predictions often decrease participants’ accuracy. To probe the relative strengths and weaknesses of humans and machines as detectors of deepfakes, we examine human and machine performance across video-level features, and we evaluate the impact of preregistered randomized interventions on deepfake detection. We find that manipulations designed to disrupt visual processing of faces hinder human participants’ performance while mostly not affecting the model’s performance, suggesting a role for specialized cognitive capacities in explaining human deepfake detection performance.

What is the relationship between complexity in the world and complexity in the mind? Intuitively, increasingly complex objects and events should give rise to increasingly complex mental representations (or perhaps a plateau in complexity after a certain point). However, a counterintuitive possibility with roots in information theory is an inverted U-shaped relationship between the “objective” complexity of some stimulus and the complexity of its mental representation, because excessively complex patterns might be characterized by surprisingly short computational descriptions (e.g., if they are represented as having been generated “randomly”). Here, we demonstrate that this is the case, using a novel approach that takes the notion of “description” literally. Subjects saw static and dynamic visual stimuli whose objective complexity could be carefully manipulated, and they described these stimuli in their own words by giving freeform spoken descriptions of them. Across three experiments totaling over 10,000 speech clips, spoken descriptions of shapes (Experiment 1), dot-arrays (Experiment 2), and dynamic motion-paths (Experiment 3) revealed a striking quadratic relationship between the raw complexity of these stimuli and the length of their spoken descriptions. In other words, the simplest and most complex stimuli received the shortest descriptions, while those stimuli with a “medium” degree of complexity received the longest descriptions. Follow-up analyses explored the particular words used by subjects, allowing us to further explore how such stimuli were represented. We suggest that the mind engages in a kind of lossy compression for overly complex stimuli, and we discuss the utility of such freeform responses for exploring foundational questions about mental representation.

Some things look more complex than others. For example, a crenulate and richly organized leaf may seem more complex than a plain stone. What is the nature of this experience—and why do we have it in the first place? Here, we explore how object complexity serves as an efficiently extracted visual signal that the object merits further exploration. We algorithmically generated a library of geometric shapes and determined their complexity by computing the cumulative surprisal of their internal skeletons—essentially quantifying the “amount of information” within each shape—and then used this approach to ask new questions about the perception of complexity. Experiments 1–3 asked what kind of mental process extracts visual complexity: a slow, deliberate, reflective process (as when we decide that an object is expensive or popular) or a fast, effortless, and automatic process (as when we see that an object is big or blue)? We placed simple and complex objects in visual search arrays and discovered that complex objects were easier to find among simple distractors than simple objects are among complex distractors—a classic search asymmetry indicating that complexity is prioritized in visual processing. Next, we explored the function of complexity: Why do we represent object complexity in the first place? Experiments 4–5 asked subjects to study serially presented objects in a self‐paced manner (for a later memory test); subjects dwelled longer on complex objects than simple objects—even when object shape was completely task‐irrelevant—suggesting a connection between visual complexity and exploratory engagement. Finally, Experiment 6 connected these implicit measures of complexity to explicit judgments. Collectively, these findings suggest that visual complexity is extracted efficiently and automatically, and even arouses a kind of “perceptual curiosity” about objects that encourages subsequent attentional engagement.

Does the human mind resemble the machines that can behave like it? Biologically inspired machine-learning systems approach “human-level” accuracy in an astounding variety of domains, and even predict human brain activity—raising the exciting possibility that such systems represent the world like we do. However, even seemingly intelligent machines fail in strange and “unhumanlike” ways, threatening their status as models of our minds. How can we know when human–machine behavioral differences reflect deep disparities in their underlying capacities, vs. when such failures are only superficial or peripheral? This article draws on a foundational insight from cognitive science—the distinction between performance and competence—to encourage “species-fair” comparisons between humans and machines. The performance/competence distinction urges us to consider whether the failure of a system to behave as ideally hypothesized, or the failure of one creature to behave like another, arises not because the system lacks the relevant knowledge or internal capacities (“competence”), but instead because of superficial constraints on demonstrating that knowledge (“performance”). I argue that this distinction has been neglected by research comparing human and machine behavior, and that it should be essential to any such comparison. Focusing on the domain of image classification, I identify three factors contributing to the species-fairness of human–machine comparisons, extracted from recent work that equates such constraints. Species-fair comparisons level the playing field between natural and artificial intelligence, so that we can separate more superficial differences from those that may be deep and enduring.

Arguably the most foundational principle in perception research is that our experience of the world goes beyond the retinal image; we perceive the distal environment itself, not the proximal stimulation it causes. Shape may be the paradigm case of such “unconscious inference”: When a coin is rotated in depth, we infer the circular object it truly is, discarding the perspectival ellipse projected on our eyes. But is this really the fate of such perspectival shapes? Or does a tilted coin retain an elliptical appearance even when we know it’s circular? This question has generated heated debate from Locke and Hume to the present; but whereas extant arguments rely primarily on introspection, this problem is also open to empirical test. If tilted coins bear a representational similarity to elliptical objects, then a circular coin should, when rotated, impair search for a distal ellipse. Here, nine experiments demonstrate that this is so, suggesting that perspectival shapes persist in the mind far longer than traditionally assumed. Subjects saw search arrays of three-dimensional “coins,” and simply had to locate a distally elliptical coin. Surprisingly, rotated circular coins slowed search for elliptical targets, even when subjects clearly knew the rotated coins were circular. This pattern arose with static and dynamic cues, couldn’t be explained by strategic responding or unfamiliarity, generalized across shape classes, and occurred even with sustained viewing. Finally, these effects extended beyond artificial displays to real-world objects viewed in naturalistic, full-cue conditions. We conclude that objects have a remarkably persistent dual character: their objective shape “out there,” and their perspectival shape “from here.”