Jonathan Birch

We face two urgent challenges concerning consciousness and AI. Challenge One is that millions of users will soon misattribute human-like consciousness to AI friends, partners, and assistants on the basis of mimicry and role-play, and we don’t know how to prevent this. Challenge Two is that profoundly alien forms of consciousness might genuinely be achieved in AI, but our theoretical understanding of consciousness is too immature to provide confident answers one way or the other. Centrism about AI consciousness is the position that we must take both challenges seriously. The two challenges interact in ways that make this difficult. Steps to address Challenge One might undermine attempts to address Challenge Two by portraying the idea of conscious AI as impossible or inherently unlikely. Conversely, attempts to address Challenge Two might lead to higher levels of misattribution from ordinary users. This “manifesto” attempts to construct mutually consistent strategies for addressing both challenges.

Motivational trade-off behaviours, where an organism behaves as if flexibly weighing up an opportunity for reward against a risk of injury, are often regarded as evidence that the organism has valenced experiences like pain. This type of evidence has been influential in shifting opinion regarding crabs and insects. Critics note that (i) the precise links between trade-offs and consciousness are not fully known; (ii) simple trade-offs are evinced by the nematode worm Caenorhabditis elegans, mediated by a mechanism plausibly too simple to support conscious experience; (iii) pain can sometimes interfere with rather than support making trade-offs rationally. However, rather than undermining trade-off evidence in general, such cases show that the nature of the trade-off, and its underlying neural substrate, matter. We investigate precisely how.

There is no agreement on whether any invertebrates are conscious and no agreement on a methodology that could settle the issue. How can the debate move forward? I distinguish three broad types of approach: theory-heavy, theory-neutral and theory-light. Theory-heavy and theory-neutral approaches face serious problems, motivating a middle path: the theory-light approach. At the core of the theory-light approach is a minimal commitment about the relation between phenomenal consciousness and cognition that is compatible with many specific theories of consciousness: the hypothesis that phenomenally conscious perception of a stimulus facilitates, relative to unconscious perception, a cluster of cognitive abilities in relation to that stimulus. This “facilitation hypothesis” can productively guide inquiry into invertebrate consciousness. What is needed? At this stage, not more theory, and not more undirected data gathering. What is needed is a systematic search for consciousness-linked cognitive abilities, their relationships to each other, and their sensitivity to masking.

In this report, we argue that there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future. That means that the prospect of AI welfare and moral patienthood — of AI systems with their own interests and moral significance — is no longer an issue only for sci-fi or the distant future. It is an issue for the near future, and AI companies and other actors have a responsibility to start taking it seriously. We also recommend three early steps that AI companies and other actors can take: They can (1) acknowledge that AI welfare is an important and difficult issue (and ensure that language model outputs do the same), (2) start assessing AI systems for evidence of consciousness and robust agency, and (3) prepare policies and procedures for treating AI systems with an appropriate level of moral concern. To be clear, our argument in this report is not that AI systems definitely are — or will be — conscious, robustly agentic, or otherwise morally significant. Instead, our argument is that there is substantial uncertainty about these possibilities, and so we need to improve our understanding of AI welfare and our ability to make wise decisions about this issue. Otherwise there is a significant risk that we will mishandle decisions about AI welfare, mistakenly harming AI systems that matter morally and/or mistakenly caring for AI systems that do not.

Rapid progress in artificial intelligence (AI) capabilities has drawn fresh attention to the prospect of consciousness in AI. There is an urgent need for rigorous methods to assess AI systems for consciousness, but significant uncertainty about relevant issues in consciousness science. We present a method for assessing AI systems for consciousness that involves exploring what follows from existing or future neuroscientific theories of consciousness. Indicators derived from such theories can be used to inform credences about whether particular AI systems are conscious. This method allows us to make meaningful progress because some influential theories of consciousness, notably including computational functionalist theories, have implications for AI that can be investigated empirically.

Consciousness has an important role in ethics: when a being consciously experiences the frustration or satisfaction of its interests, those interests deserve higher moral priority than those of a behaviourally similar but non-conscious being. I consider the relationship between this ethical role and an a posteriori (or ‘type-B’) materialist solution to the mind-body problem. It is hard to avoid the conclusion that, if type-B materialism is correct, then the reference of the concept Phenomenal Consciousness is radically indeterminate between a neuronal-level property that is distinctive to mammals and a high-level functional property that is much more widely shared. This would leave many non-mammalian animals (such as birds, fish, insects and octopuses) with indeterminate moral status. There are ways to manage this radical moral indeterminacy, but all of these ways lead to profoundly troubling consequences.

What makes fast, cumulative cultural evolution work? Where did it come from? Why is it the sole preserve of humans? We set out a self-assembly hypothesis: cultural evolution evolved culturally. We present an evolutionary account that shows this hypothesis to be coherent, plausible, and worthy of further investigation. It has the following steps: (0) in common with other animals, early hominins had significant capacity for social learning; (1) knowledge and skills learned by offspring from their parents began to spread because bearers had more offspring, a process we call CS1 (or Cultural Selection 1); (2) CS1 shaped attentional learning biases; (3) these attentional biases were augmented by explicit learning biases (judgements about what should be copied from whom). Explicit learning biases enabled (4) the high-fidelity, exclusive copying required for fast cultural accumulation of knowledge and skills by a process we call CS2 (or Cultural Selection 2), and (5) the emergence of cognitive processes such as imitation, mindreading and metacognition – ‘cognitive gadgets’ specialised for cultural learning. This self-assembly hypothesis is consistent with archaeological evidence that the stone tools used by early hominins were not dependent on fast, cumulative cultural evolution, and suggests new priorities for research on ‘animal culture’.

We are all guided by thousands of norms, but how did our capacity for normative cognition evolve? I propose there is a deep but neglected link between normative cognition and practical skill. In modern humans, complex motor skills and craft skills, such as toolmaking, are guided by internally represented norms of correct performance. Moreover, it is plausible that core components of human normative cognition evolved as a solution to the distinctive problems of transmitting complex motor skills and craft skills, especially skills related to toolmaking, through social learning. If this is correct, the expansion of the normative domain beyond technique to encompass more abstract norms of fairness, reciprocity, ritual and kinship involved the elaboration of a basic platform for the guidance of skilled action by technical norms. This article motivates and defends this “skill hypothesis” for the origin of normative cognition and sets out various ways in which it could be empirically tested.

Over the past two decades, Ginsburg and Jablonka have developed a novel approach to studying the evolutionary origins of consciousness: the Unlimited Associative Learning framework. The central idea is that there is a distinctive type of learning that can serve as a transition marker for the evolutionary transition from non-conscious to conscious life. The goal of this paper is to stimulate discussion of the framework by providing a primer on its key claims and a clear statement of its main empirical predictions.