Mamoru Nagae

Long Abstract It is not obvious why we are conscious. Why can’t all of our mental activities take place unconsciously? What is consciousness for? We aim to make progress on this question, focusing on conscious vision. We review evidence on the timescale of visual consciousness, showing that it is surprisingly slow: postdictive effects reveal windows of unconscious integration lasting up to 400 milliseconds. We argue that if consciousness is slow, it cannot be for online action-guidance. Instead, we propose that conscious vision evolved to support offline cognition, in tandem with the larger visual sensory horizons afforded by the water-to-land transition. Smaller visual horizons typical in aquatic environments require fast, reflexive actions of the sort that are guided unconsciously in humans. Conversely, larger terrestrial visual horizons allow benefits to accrue from “model-based” planning of the sort that is associated with consciousness in humans. We further propose that the acquisition of these capacities for internal simulation and planning provided pressures for the evolution of reality monitoring – the capacity to distinguish between internally and externally triggered signals, and to solve “Hamlet’s problem” in perception – the problem of when to stop integrating evidence, and fix a particular model of reality. In line with higher-order theories of consciousness, we associate the emergence of consciousness with the emergence of this reality monitoring function. We discuss novel empirical predictions that arise from this account, and explore its implications for the distribution of conscious (vs. unconscious) vision in aquatic and terrestrial animals.

We explore and argue for the view that intelligence beyond a certain threshold is negatively correlated with consciousness, so that the more intelligent a system is, the less likely it is to be conscious. “Intelligence is the enemy of consciousness,” as we put it. This view entails that superintelligent AI models are likely to be zombies. After presenting an initial defense of the view that draws on the research program of resource-rational analysis, we argue that recent developments in AI research provide additional evidence supporting it. We conclude by exploring its moral implications. First, we suggest that if our view is right, a certain type of moral risk can be averted since researchers are unlikely to stumble into accidentally creating AI models that are conscious. Second, we discuss a new form of AI risk that arises from our view.