Preston Lennon

Some philosophers argue that near-term AI systems might soon be welfare subjects. We should take AI welfare seriously, they urge, because AI systems might soon have properties indicated by our best scientific theories of consciousness, and consciousness suffices for welfare subjectivity. I argue that this view faces constraints from the epistemology of consciousness. Specifically, I argue that there is an enormous gap in justification between our first-personal beliefs of consciousness in ourselves and our third-personal, theoretical belief of what consciousness is according to contemporary consciousness science. The science of consciousness is so in its infancy that it is doubtful that the set of theories currently on offer is exhaustive. In contrast, beliefs about the fact that we are conscious are, famously, among the most epistemically secure we can have. Our taking AI welfare seriously, in terms of action or policy, will be constrained by these considerations from the epistemology of consciousness.

Large language models (LLMs) exhibit impressive performance across a range of apparently cognitive tasks. Mentalists holds that this performance is best explained by the fact that LLMs have mental states, while anti-mentalists hold that this performance should be explained some other way. In this note we address representationalist folk mentalism, which holds (a) possessing a folk mental state like belief or desire is a matter of having an internal representation with appropriate content and (b) that LLMs have folk psychological states of this sort (or at least robust precursors to such states). Although representationalist folk mentalism appears attractive, we argue that neither probing nor intervention studies have uncovered representations of the relevant sort in state-of-the-art LLMs. However, while it would be premature to accept representationalist folk mentalism, our argument provides a roadmap for mechanistic interpretability research going forward.