Raphaël Millière

Associationism is one of the oldest, and, in some form or another, most widely held theories of thought. Associationism has been the engine behind empiricism for centuries, from the British Empiricists through the Behaviorists and modern day Connectionists. Nevertheless, “associationism” does not refer to one particular theory of cognition per se, but rather a constellation of related though separable theses. What ties these theses together is a commitment to a certain arationality of thought: a creature’s mental states are associated because of some facts about its causal history, and having these mental states associated entails that bringing one of a pair of associates to mind will, ceteris paribus, ensure that the other also becomes activated.

As AI systems progress, we rely more on them to make decisions with us and for us. To ensure that such decisions are aligned with human values, it is imperative for us to understand not only what decisions they make but also how they come to those decisions. Reasoning language models, which provide both final responses and (partially transparent) intermediate thinking traces, present a timely opportunity to study AI procedural reasoning. Unlike math and code problems which often have objectively correct answers, moral dilemmas are an excellent testbed for process-focused evaluation because they allow for multiple defensible conclusions. To do so, we present MoReBench: 1,000 moral scenarios, each paired with a set of rubric criteria that experts consider essential to include (or avoid) when reasoning about the scenarios. MoReBench contains over 23 thousand criteria including identifying moral considerations, weighing trade-offs, and giving actionable recommendations to cover cases on AI advising humans moral decisions as well as making moral decisions autonomously. Separately, we curate MoReBench-Theory: 150 examples to test whether AI can reason under five major frameworks in normative ethics. Our results show that scaling laws and existing benchmarks on math, code, and scientific reasoning tasks fail to predict models’ abilities to perform moral reasoning. Models also show partiality towards specific moral frameworks (e.g., Benthamite Act Utilitarianism and Kantian Deontology), which might be side effects of popular training paradigms. Together, these benchmarks advance process-focused reasoning evaluation towards safer and more transparent AI.

Large language models (LLMs) exhibit remarkable performance improvement through In-Context Learning (ICL) by leveraging task-specific examples in the input. However, the mechanisms behind this improvement remain elusive. In this work, we investigate how LLM embeddings and attention representations change following in-context-learning, and how these changes mediate improvement in behavior. We employ neuroscience-inspired techniques such as representational similarity analysis (RSA) and propose novel methods for parameterized probing and measuring ratio of attention to relevant vs. irrelevant information in Llama-2 70B and Vicuna 13B. We designed two tasks with a priori relationships among their conditions: linear regression and reading comprehension. We formed hypotheses about expected similarities in task representations and measured hypothesis alignment of LLM representations before and after ICL as well as changes in attention. Our analyses revealed a meaningful correlation between improvements in behavior after ICL and changes in both embeddings and attention weights across LLM layers. This empirical framework empowers a nuanced understanding of how latent representations shape LLM behavior, offering valuable tools and insights for future research and practical applications.

Variable binding—the ability to associate variables with values—is fundamental to symbolic computation and cognition. Although classical architectures typically implement variable binding via addressable memory, it is not well understood how modern neural networks lacking built-in binding operations may acquire this capacity. We investigate this by training a Transformer to dereference queried variables in symbolic programs where variables are assigned either numerical constants or other variables. Each program requires following chains of variable assignments up to four steps deep to find the queried value, and also contains irrelevant chains of assignments acting as distractors. Our analysis reveals a developmental trajectory with three distinct phases during training: (1) random prediction of numerical constants, (2) a shallow heuristic prioritizing early variable assignments, and (3) the emergence of a systematic mechanism for dereferencing assignment chains. Using causal interventions, we find that the model learns to exploit the residual stream as an addressable memory space, with specialized attention heads routing information across token positions. This mechanism allows the model to dynamically track variable bindings across layers, resulting in accurate dereferencing. Our results show how Transformer models can learn to implement systematic variable binding without explicit architectural support, bridging connectionist and symbolic approaches.

Analogical Reasoning—the capacity to identify and map structural relationships between different domains—is fundamental to human cognition and learning. Recent studies have shown that Large Language Models (LLMs) can sometimes match humans in analogical Reasoning tasks, opening the possibility that analogical Reasoning might emerge from domain-general processes. However, it is still debated whether these emergent capacities are largely superficial and limited to simple relations seen during training or whether they encompass the flexible representational and mapping capabilities which are the focus of leading cognitive models of analogy. In this study, we introduce novel analogical Reasoning tasks that require participants to map between semantically contentful words and sequences of letters and other abstract characters. This task necessitates the ability to flexibly re-represent rich semantic information—an ability which is known to be central to human analogy but which is thus far not well-captured by existing cognitive theories and models. We assess the performance of both human participants and LLMs on tasks focusing on Reasoning from semantic structure and semantic content, introducing variations that test the robustness of their analogical inferences. Advanced LLMs match human performance across several conditions, though humans and LLMs respond differently to certain task variations and semantic distractors. Our results thus provide new evidence that LLMs might offer a how-possibly explanation of human analogical Reasoning in contexts that are not yet well modeled by existing theories, but that even today’s best models are unlikely to yield how-actually explanations.

Deep learning has enabled major advances across most areas of artificial intelligence research. This remarkable progress extends beyond mere engineering achievements and holds significant relevance for the philosophy of cognitive science. Deep neural networks have made significant strides in overcoming the limitations of older connectionist models that once occupied the center stage of philosophical debates about cognition. This development is directly relevant to long-standing theoretical debates in the philosophy of cognitive science. Furthermore, ongoing methodological challenges related to the comparative evaluation of deep neural networks stand to benefit greatly from interdisciplinary collaboration with philosophy and cognitive science. The time is ripe for philosophers to explore foundational issues related to deep learning and cognition; this perspective paper surveys key areas where their contributions can be especially fruitful.

Evaluating the cognitive capacities of large language models (LLMs) requires overcoming not only anthropomorphic but also anthropocentric biases. This article identifies two types of anthropocentric bias that have been neglected: overlooking how auxiliary factors can impede LLM performance despite competence (auxiliary oversight), and dismissing LLM mechanistic strategies that differ from those of humans as not genuinely competent (mechanistic chauvinism). Mitigating these biases requires an empirical, iterative approach to mapping cognitive tasks to LLM-specific capacities and mechanisms, achieved by supplementing behavioral experiments with mechanistic studies.

This chapter critically examines the potential contributions of modern language models to theoretical linguistics. Despite their focus on engineering goals, these models’ ability to acquire sophisticated linguistic knowledge from mere exposure to data warrants a careful reassessment of their relevance to linguistic theory. I review a growing body of empirical evidence suggesting that language models can learn hierarchical syntactic structure and exhibit sensitivity to various linguistic phenomena, even when trained on developmentally plausible amounts of data. While the competence/performance distinction has been invoked to dismiss the relevance of such models to linguistic theory, I argue that this assessment may be premature. By carefully controlling learning conditions and making use of causal intervention methods, experiments with language models can potentially constrain hypotheses about language acquisition and competence. I conclude that closer collaboration between theoretical linguists and computational researchers could yield valuable insights, particularly in advancing debates about linguistic nativism.

The success of large language models (LLMs) across many domains of AI research has generated intense debate. Some attribute their impressive performance on complex tasks to human-like linguistic and cognitive capacities, whereas others ascribe it to shallow pattern matching. These disputes stem from deep-seated philosophical disagreements about the nature of language and cognition. We provide an opinionated survey of these disagreements across core topics in the philosophy of mind and language, including syntactic competence, compositionality, linguistic meaning, representation, attitudes, reasoning, agency, and consciousness. We contend that progress on these issues requires not only clarity about background philosophical commitments but also, in many cases, close engagement with emerging empirical evidence.

Large language models (LLMs) produce seemingly meaningful outputs, yet they are trained on text alone without direct interaction with the world. This leads to a modern variant of the classical symbol grounding problem in AI: can LLMs' internal states and outputs be about extra-linguistic reality, independently of the meaning human interpreters project onto them? We argue that they can. We first distinguish referential grounding—the connection between a representation and its worldly referent—from other forms of grounding and argue it is the only kind essential to solving the problem. We contend that referential grounding is achieved when a system's internal states satisfy two conditions derived from teleosemantic theories of representation: (1) they stand in appropriate causal-informational relations to the world, and (2) they have a history of selection that has endowed them with the function of carrying this information. We argue that LLMs can meet both conditions, even without multimodality or embodiment.

In this paper, the second of two companion pieces, we explore novel philosophical questions raised by recent progress in large language models (LLMs) that go beyond the classical debates covered in the first part. We focus particularly on issues related to interpretability, examining evidence from causal intervention methods about the nature of LLMs' internal representations and computations. We also discuss the implications of multimodal and modular extensions of LLMs, recent debates about whether such systems may meet minimal criteria for consciousness, and concerns about secrecy and reproducibility in LLM research. Finally, we discuss whether LLM-like systems may be relevant to modeling aspects of human cognition, if their architectural characteristics and learning scenario are adequately constrained.

Large language models like GPT-4 have achieved remarkable proficiency in a broad spectrum of language-based tasks, some of which are traditionally associated with hallmarks of human intelligence. This has prompted ongoing disagreements about the extent to which we can meaningfully ascribe any kind of linguistic or cognitive competence to language models. Such questions have deep philosophical roots, echoing longstanding debates about the status of artificial neural networks as cognitive models. This article -- the first part of two companion papers -- serves both as a primer on language models for philosophers, and as an opinionated survey of their significance in relation to classic debates in the philosophy cognitive science, artificial intelligence, and linguistics. We cover topics such as compositionality, language acquisition, semantic competence, grounding, world models, and the transmission of cultural knowledge. We argue that the success of language models challenges several long-held assumptions about artificial neural networks. However, we also highlight the need for further empirical investigation to better understand their internal mechanisms. This sets the stage for the companion paper (Part II), which turns to novel empirical methods for probing the inner workings of language models, and new philosophical questions prompted by their latest developments.

Many authors claim that being conscious constitutively involves being self-conscious, or conscious of oneself. This claim appears to be threatened by reports of ‘selfless’ episodes, or conscious episodes lacking self-consciousness, recently described in a number of pathological and nonpathological conditions. However, the credibility of these reports has in turn been challenged on the following grounds: remembering and reporting a past conscious episode as an episode that one went through is only possible if one was conscious of oneself while undergoing it. Call this the Memory Challenge. This paper argues that the Memory Challenge fails to undermine the credibility to reports of selfless episodes, because it rests on problematic assumptions about episodic memory. The paper further argues that we should distinguish between several kinds of self-representation that may be involved in the process of episodic remembering, and that once we do so, it is no longer mysterious how one could accurately remember and report a selfless episode as an episode that one went through. Thus, we should take reports of this kind seriously, and view them as credible counter-examples to the claim that consciousness constitutively involves self-consciousness.

The progress of AI systems such as large language models (LLMs) raises increasingly pressing concerns about their safe deployment. This paper examines the value alignment problem for LLMs, arguing that current alignment strategies are fundamentally inadequate to prevent misuse. Despite ongoing efforts to instill norms such as helpfulness, honesty, and harmlessness in LLMs through fine-tuning based on human preferences, they remain vulnerable to adversarial attacks that exploit conflicts between these norms. I argue that this vulnerability reflects a fundamental limitation of existing alignment methods: they reinforce shallow behavioral dispositions rather than endowing LLMs with a genuine capacity for normative deliberation. Drawing from on research in moral psychology, I show how humans’ ability to engage in deliberative reasoning enhances their resilience against similar adversarial tactics. LLMs, by contrast, lack a robust capacity to detect and rationally resolve normative conflicts, leaving them susceptible to manipulation; even recent advances in reasoning-focused LLMs have not addressed this vulnerability. This “shallow alignment” problem carries significant implications for AI safety and regulation, suggesting that current approaches are insufficient for mitigating potential harms posed by increasingly capable AI systems.

The remarkable performance of large language models (LLMs) on complex linguistic tasks has sparked debate about their capabilities. Unlike humans, these models learn language solely from textual data without directly interacting with the world. Yet they generate seemingly meaningful text on diverse topics. This achievement has renewed interest in the classical `Symbol Grounding Problem' -- the question of whether the internal representations and outputs of symbolic AI systems can possess intrinsic meaning that is not parasitic on external interpretation. Although modern LLMs compute over vectors rather than symbols, an analogous problem arises for these systems, which we call the Vector Grounding Problem. This paper has two main goals. First, we distinguish five main notions of grounding that are often conflated in the literature, and argue that only one of them, which we call referential grounding, is relevant to the Vector Grounding Problem. Second, drawing on philosophical theories of representational content, we provide two arguments for the claim that LLMs and related systems can achieve referential grounding: (1) through preference fine-tuning methods that explicitly establish world-involving functions, and (2) through pre-training alone, which in limited domains may select for internal states with world-involving content, as mechanistic interpretability research suggests. Through these pathways, LLMs can establish connections to the world sufficient for intrinsic meaning. One potentially surprising implication of our discussion is that that multimodality and embodiment are neither necessary nor sufficient to overcome the Grounding Problem. *Note: this is an updated and improved version of the preprint originally made available on ArXiv in 2023.*

The claim that consciousness constitutively involves self-consciousness has a long philosophical history, and has received renewed support in recent years. My aim in this paper is to argue that this surprisingly enduring idea is misleading at best, and insufficiently supported at worst. I start by offering an elucidatory account of consciousness, and outlining a number of foundational claims that plausibly follow from it. I subsequently distinguish two notions of self-consciousness: consciousness of oneself and consciousness of one’s experience. While “self-consciousness” is often taken to refer to the former notion, the most common variant of the constitutive claim, on which I focus here, targets the latter. This claim can be further interpreted in two ways: on a deflationary reading, it falls within the scope of foundational claims about consciousness, while on an inflationary reading, it points to determinate aspects of phenomenology that are not acknowledged by the foundational claims as being aspects of all conscious mental states. I argue that the deflationary reading of the constitutive claim is plausible, but should be formulated without using a term as polysemous and suggestive as “self-consciousness”; by contrast, the inflationary reading is not adequately supported, and ultimately rests on contentious intuitions about phenomenology. I conclude that we should abandon the idea that self-consciousness is constitutive of consciousness.

Recent breakthroughs in artificial intelligence have primarily resulted from training deep neural networks (DNNs) with vast numbers of adjustable parameters on enormous datasets. Due to their complex internal structure, DNNs are frequently characterized as inscrutable ``black boxes,'' making it challenging to interpret the mechanisms underlying their impressive performance. This opacity creates difficulties for explanation, safety assurance, trustworthiness, and comparisons to human cognition, leading to divergent perspectives on these systems. This chapter examines recent developments in interpretability methods for DNNs, with a focus on interventionist approaches inspired by causal explanation in philosophy of science. We argue that these methods offer a promising avenue for understanding how DNNs process information compared to merely behavioral benchmarking and correlational probing. We review key interventionist methods and illustrate their application through practical case studies. These methods allow researchers to identify and manipulate specific computational components within DNNs, providing insights into their causal structure and internal representations. We situate these approaches within the broader framework of causal abstraction, which aims to align low-level neural computations with high-level interpretable models. While acknowledging current limitations, we contend that interventionist methods offer a path towards more rigorous and theoretically grounded interpretability research, potentially informing both AI development and computational cognitive neuroscience.

Deep learning algorithms are rapidly changing the way in which audiovisual media can be produced. Synthetic audiovisual media generated with deep learning—often subsumed colloquially under the label “deepfakes”—have a number of impressive characteristics; they are increasingly trivial to produce, and can be indistinguishable from real sounds and images recorded with a sensor. Much attention has been dedicated to ethical concerns raised by this technological development. Here, I focus instead on a set of issues related to the notion of synthetic audiovisual media, its place within a broader taxonomy of audiovisual media, and how deep learning techniques differ from more traditional approaches to media synthesis. After reviewing important etiological features of deep learning pipelines for media manipulation and generation, I argue that “deepfakes” and related synthetic media produced with such pipelines do not merely offer incremental improvements over previous methods, but challenge traditional taxonomical distinctions, and pave the way for genuinely novel kinds of audiovisual media.

Many authors claim that being conscious constitutively involves being self-conscious, or conscious of oneself. This claim appears to be threatened by reports of `selfless' episodes, or conscious episodes lacking self-consciousness, recently described in a number of pathological and nonpathological conditions. However, the credibility of these reports has in turn been challenged on the following grounds: remembering and reporting a past conscious episode as an episode that one went through is only possible if one was conscious of oneself while undergoing it. Call this the Memory Challenge. This paper argues that the Memory Challenge fails to undermine the credibility to reports of selfless episodes, because it rests on problematic assumptions about episodic memory. The paper further argues that we should distinguish between several kinds of self-representation that may be involved in the process of episodic remembering, and that once we do so, it is no longer mysterious how one could accurately remember and report a selfless episode as an episode that one went through. Thus, we should take reports of this kind seriously, and view them as credible counter-examples to the claim that consciousness constitutively involves self-consciousness.

In recent years, a debate has emerged on whether bodily sensations are typically accompanied by a sense of body ownership, namely a distinctive experience of one's body or body part as one's own. Realists about the sense of body ownership heavily rely on evidence from experimentally-induced bodily illusions (e.g., the rubber hand illusion) and pathological disownership syndromes (e.g. somatoparaphrenia). In this chapter, I will introduce novel evidence regarding body disownership syndromes induced by psychoactive drugs rather than pathological conditions, and discuss its relevance to the debate on the sense of body ownership.

Text-guided image generation models can be prompted to generate images using nonce words adversarially designed to robustly evoke specific visual concepts. Two approaches for such generation are introduced: macaronic prompting, which involves designing cryptic hybrid words by concatenating subword units from different languages; and evocative prompting, which involves designing nonce words whose broad morphological features are similar enough to that of existing words to trigger robust visual associations. The two methods can also be combined to generate images associated with more specific visual concepts. The implications of these techniques for the circumvention of existing approaches to content moderation, and particularly the generation of offensive or harmful images, are discussed.

This thesis investigates the relationship between consciousness and self-consciousness. I consider two broad claims about this relationship: a constitutive claim, according to which all conscious experiences constitutively involve self-consciousness; and a typicalist claim, according to which ordinary conscious experiences contingently involve self-consciousness. Both of these claims call for elucidation of the relevant notions of consciousness and self-consciousness. In the first part of the thesis ('The Myth of Constitutive Self-Consciousness'), I critically examine the constitutive claim. I start by offering an elucidatory account of consciousness, and outlining a number of foundational claims that plausibly follow from it. I subsequently distinguish between two concepts of self-consciousness: consciousness of one's experience, and consciousness of oneself (as oneself). Each of these concepts yields a distinct variant of the constitutive claim. In turn, each resulting variant of the constitutive claim can be interpreted in two ways: on a 'minimal' or deflationary reading, they fall within the scope of foundational claims about consciousness, while on a 'strong' or inflationary reading, they point to determinate aspects of phenomenology that are not acknowledged by the foundational claims as being aspects of all conscious mental states. I argue that the deflationary readings of either variant of the constitutive claim are plausible and illuminating, but would ideally be formulated without using a term as polysemous as 'self-consciousness'; by contrast, the inflationary readings of either variant are not adequately supported. In the second part of the thesis ('Self-Consciousness in the Real World'), I focus on the second concept of self-consciousness, or consciousness of oneself as oneself. Drawing upon empirical evidence, I defend a pluralist account of self-consciousness so construed, according to which there are several ways in which one can be conscious of oneself as oneself – through conscious thoughts, bodily experiences and perceptual experiences – that make distinct determinate contributions to one's phenomenology. This pluralist account provides us with the resources to vindicate the typicalist claim according to which consciousness of oneself as oneself – a sense of self – is pervasive in ordinary conscious experiences, as a matter of contingent empirical fact. It also provides us with the resources to assess the possibility that a subject might be conscious without being conscious of herself as herself in any way.

It is widely assumed that ordinary conscious experience involves some form of sense of self or consciousness of oneself. Moreover, this claim is often restricted to a ‘thin’ or ‘minimal’ notion of self-consciousness, or even ‘the simplest form of self-consciousness’, as opposed to more sophisticated forms of self-consciousness which are not deemed ubiquitous in ordinary experience. These formulations suggest that self-consciousness comes in degrees, and that individual subjects may differ with respect to the degree of self-consciousness they exhibit at a given time. In this article, I critically examine this assumption. I consider what the claim that self-consciousness comes in degrees may mean, raise some challenges against the different versions of the claim, and conclude that none of them is both coherent and particularly plausible.

Many authors argue that conscious experience involves a sense of self or self-consciousness. According to the strongest version of this claim, there can be no selfless states of consciousness, namely states of consciousness that lack self-consciousness altogether. Disagreements about this claim are likely to remain merely verbal as long as the target notion of self-consciousness is not adequately specified. After distinguishing six notions of self-consciousness commonly discussed in the literature, I argue that none of the corresponding features is necessary for consciousness, because there are states of consciousness in which each of them is plausibly missing. Such states can be said to be at least partially selfless, since they lack at least one of the ways in which one could be self-conscious. Furthermore, I argue that there is also preliminary empirical evidence that some states of consciousness lack all of these six putative forms of self-consciousness. Such states might be totally selfless, insofar as they lack all the ways in which one could be self-conscious. I conclude by addressing four objections to the possibility and reportability of totally selfless states of consciousness.

Bayne and Carter argue that the mode of consciousness induced by psychedelic drugs does not fit squarely within the traditional account of modes as levels of consciousness, and favor instead a multi-dimensional account according to which modes of consciousness differ along several dimensions—none of which warrants a linear ordering of modes. We discuss the assumption that psychedelic drugs induce a single or paradigmatic mode of consciousness, as well as conceptual issues related to Bayne and Carter’s main argument against the traditional account. Finally, we raise a set of questions about the individuation of dimensions selected to differentiate modes of consciousness that could be addressed in future discussions of the multi-dimensional account.

Philosophical and empirical research on bodily awareness has mostly focused so far on bodily disorders – such as anorexia nervosa, somatoparaphrenia, or xenomelia (body integrity dysphoria) – and bodily illusions induced in an experimental setting – such as the rubber hand illusion, or the thermal grid illusion. Studying these conditions can be illuminating to investigate a broad range of issues about the nature, function, and etiology of bodily experience. However, a number of psychoactive compounds can also induce a remarkably wide variety of bodily effects that have received little attention despite their relevance to these issues. Some of these effects are similar to those associated with bodily disorders or experimental bodily illusions, while others appear to be unique. The purpose of this chapter is to provide an overview of these lesser known bodily effects of psychoactive compounds, and outline some ways in which they can bear on recent debates regarding bodily awareness and bodily ownership.