Tuğba Yoldaş

Most work on ethics and artificial intelligence (AI) rightly focuses on how the design and use of AI systems affect individuals other than the systems themselves. However, as AI systems become more sophisticated and capable of emulating intelligent behavior, there is growing interest in whether and under what circumstances AIs would become moral patients, i.e., entities that are themselves capable of receiving morally significant harms and benefits, and hence are owed moral considerations. It may seem far-fetched to think that present-day AI systems, which are widely considered complex tools, could ever become the kinds of entities to whom we owe moral obligations. Yet, I believe that it is timely to begin thinking about this prospect. It can help us better understand the nature of minds, the value of life and consciousness, the harm of death, and the immense responsibilities that would come with creating artificial moral patients. This dissertation addresses two main questions about artificial moral patiency: What would it take for an AI system to be a moral patient? And should we create artificial moral patients? First, I ask the question of what it would take for an entity to be capable of being harmed and benefited in morally significant ways. I argue that whichever theory of well-being we accept, an entity counts as a moral patient only if it is capable of phenomenally conscious mental states, i.e., states ‘there is something it is like’, such as experiences, motivations, and beliefs. I also argue that the capacity for phenomenal conscious states requires being capable of mental states with unified, rich, multisensory experiences that are integrated and experienced from an egocentric, or self-referential, perspective. Second, I ask the question of what it would take for an entity to be capable of these mental states that are required for moral patiency. I argue that on the most plausible theories of consciousness, what it is for an entity to have the capacity for having not only subjectively experienced representational states like beliefs and perceptions but also affective states like pain, pleasure, and emotions is for it to have states with distinctive sort of intentionality, i.e., to be about or directed towards the world that is capable of genuine error and unfulfillment, or to have content. Third, I ask the question of what it would take for an entity to be capable of states with intentionality. Drawing on the philosopher Daniel Dennett’s intentional stance, I claim that attributions of intentional states like beliefs and desires to entities like us who are capable of states with original or true intentionality pick out explanatorily important regularity in how we are disposed to behave in a wide range of circumstances, which does not apply to attributions of such states to entities that are capable of these states merely in the metaphorical sense. After discussing the main philosophical theories of intentionality, I find that the theory of success semantics provides the most plausible naturalistic explanation of content. On this view, an entity’s representational and motivational states such as beliefs and desires count as beliefs and desires only if they are capable of systematically and flexibly interacting with the entity’s wide variety of other representational and motivational states to produce a wide variety of behaviors that would successfully fulfill the system’s goals if its representations were accurate. Drawing on this view, I discuss a hierarchy of intentional states at the bottom of which there are basic maximally egocentric representational and motivational states, the contents of which are accurate and fulfilled without reference to the contents of the entity’s more sophisticated representational and motivational states. Next, I apply this account to the case of present-day AI systems and argue that none of them are moral patients yet as none has egocentric motivations, though self-driving cars and care robots come closer to meeting the conditions for moral patiency. Finally, by examining the main views in population ethics, I argue that this is good news because even on the least restrictive views in population ethics, we have good moral reasons to be hesitant to bring artificial moral patients into existence, at least for now.

This paper explores the ethical challenges, particularly around Responsible AI, from the integration of large language models (LLMs) in generative AI (GenAI) applications across various domains. While LLMs enhance creativity, improve productivity, and enable human-like conversations, their opaque reasoning raises concerns about accountability and moral responsibility. The paper points out the limits of the existing framework of Meaningful Human Control (MHC), which emphasizes human oversight of AI systems. I argue that MHC alone is insufficient in addressing the challenges posed by LLMs. Instead, I suggest that MHC needs to be supplemented with virtue ethics and virtue epistemology, which focuses on exercising ‘acts of intellectual virtue’, such as open-mindedness, critical thinking, and epistemic humility. By integrating these frameworks, I propose a more holistic approach to Responsible GenAI to counteract the risks of automation complacency, to promote a more responsible, thoughtful engagement with GenAI, and ultimately to foster human flourishing and safeguard against the erosion of cognitive skills due to GenAI use.

In this paper, we defend three claims about what it will take for an AI system to be a basic moral patient to whom we can owe duties of non-maleficence not to harm her and duties of beneficence to benefit her: (1) Moral patients are mental patients; (2) Mental patients are true intentional systems; and (3) True intentional systems are systematically flexible. We suggest that we should be particularly alert to the possibility of such systematically flexible true intentional systems developing in the areas of exploratory robots and artificial personal assistants. Finally, we argue that in light of our failure to respect the well-being of existing biological moral patients and worries about our limited resources, there are compelling moral reasons to treat artificial moral patiency as something to be avoided at least for now.