Michael Cannon

Contemporary mind sciences are made up of two main paradigms, each with their own working metaphor for what mind and cognition are. Broadly speaking, for the mainstream paradigm - 'cognitivism' - the mind is a computer and cognition is computation, and for 4E/Enactive cognition, a thoroughly established paradigm but a bit more niche, the mind is a living system and cognition is sensemaking. Given that both paradigms are supported by decades of scientific evidence, the article engages the metatheoretical question of how to hold the two together. It does so by treating both paradigms as stories about what a human is and exploring the power of these stories in the context of the cultural and planetary challenges we are navigating. It argues that the paradigm that stories a human as functionally no different to a machine incurs a kind of epistemic injustice upon humans, effecting how we make sense of ourselves and the world of which we are a part and that for this reason we should be wary of an Arendtian banality in being led by a model and theory that diminishes us.

Inspired by earlier literature on the question of artificial superintelligence and the possibility of AI systems meaningfully surpassing human capacities, the question of this thesis is whether AI can become more ethical than humans. The thesis begins by recognising that we have not one but two major paradigms of mind research, each with their own model of mind - the mind-as-computer model of the computational 'cognitivist' paradigm, and the mind-as-living system of 4E/Enactive 'post-cognivitist' paradigm. Following a presentation of the fundamental tenets of each paradigm, the thesis argues that the answer to the question depends on the paradigm with which we think. In the computational/cognitivist paradigm, AI systems could become more ethical than humans, though it might not be in a meaningful sense, whilst in the 4E/post-cognitivist paradigm, AI systems cannot become more ethical than humans.

The “Value Alignment Problem” is the challenge of how to align the values of artificial intelligence with human values, whatever they may be, such that AI does not pose a risk to the existence of humans. Existing approaches appear to conceive of the problem as "how do we ensure that AI solves the problem in the right way", in order to avoid the possibility of AI turning humans into paperclips in order to “make more paperclips” or eradicating the human race to “solve climate change”. This paper proposes an approach to Alignment rooted in the Enactive theory of mind that reconceptualises it as "how do we make relevant to AI what it relevant to humans". This conceptualisation is supported with a discussion of 4E cognition and goes on to suggest that the Alignment problem and the Frame problem are the same problem. The paper concludes with a discussion of the tradeoffs of the different approaches to value alignment.

The “Value Alignment Problem” is the challenge of how to align the values of artificial intelligence with human values, whatever they may be, such that AI does not pose a risk to the existence of humans. A fundamental feature of how the problem is currently understood is that AI systems do not take the same things to be relevant as humans, whether turning humans into paperclips in order to “make more paperclips” or eradicating the human race to “solve climate change”. Specifically, existing approaches approaches to alignment appear to be concerned with how AI might *solve* problems in the relevant way. This paper presents and explores an approach to alignment rooted in the Enactive Theory of mind. It offers an alternative conception of the alignment as “how do we make relevant to AI what is relevant to humans?” In this conception, alignment is concerned with building AI so that it *discerns* and defines the problem in the relevant way. In this way, the Alignment Problem is shown to be the same problem as the Frame Problem. The paper concludes with a consideration of tradeoffs between these conceptions of the alignment problem.

The standard argument to the conclusion that artificial intelligence (AI) constitutes an existential risk for the human species uses two premises: (1) AI may reach superintelligent levels, at which point we humans lose control (the ‘singularity claim’); (2) Any level of intelligence can go along with any goal (the ‘orthogonality thesis’). We find that the singularity claim requires a notion of ‘general intelligence’, while the orthogonality thesis requires a notion of ‘instrumental intelligence’. If this interpretation is correct, they cannot be joined as premises and the argument for the existential risk of AI turns out invalid. If the interpretation is incorrect and both premises use the same notion of intelligence, then at least one of the premises is false and the orthogonality thesis remains itself orthogonal to the argument to existential risk from AI. In either case, the standard argument for existential risk from AI is not sound.—Having said that, there remains a risk of instrumental AI to cause very significant damage if designed or used badly, though this is not due to superintelligence or a singularity.