Maria Danielsen

AI systems and tools are being implemented at an increasingly rapid rate in society for a variety of purposes such as decision-making, managing job applications, and socializing. These new technologies have a lot of promise but may also introduce new risks by threatening human moral and relational values, as well as values connected to flourishing. Mainstream approaches to risk assessment do not pay sufficient attention to these values. The study of emotions as they are connected to human values can therefore play an important role in risk management. We will contribute to this discussion by introducing the concept of human needs, or what we consider to be the sources of values that constitute emotions. This brings a new perspective to the debate around AI and risk. By combining insights from Martha Nussbaum and Soran Reader, we argue that while emotions are crucial for highlighting what values are activated in a particular situation, the sources of an important part of human values are human needs. This provides for what we call the ‘needs-values-emotions nexus’. We argue that this framework can add to the discussion about the ethical risks of AI in two fundamental ways. First, highlighting the crucial role of needs helps to explain why AI systems cannot develop, feel, nor reason according to human values. On the most basic level, AI systems lack a constitutive part of these values, i.e., they lack needs. The deployment of AI, for example to replace human decision-making, may therefore threaten human values. We discuss this by zooming in on a recent example, the so-called Dutch tax benefit scandal. Second, this paper argues that we need emotions to concretize and deliberate on what values are at risk when developing and using AI technology. Further building on the ‘needs-values-emotions nexus’ developed in this paper, we argue why art is a preeminent medium to elicit emotions and ethical reflection on the risks of AI. Discussing a concrete example, we illustrate how contemporary artists can contribute to ethical risk-assessments by focusing on the societal impact of AI.

The existential risk posed by ubiquitous artificial intelligence (AI) is a subject of frequent discussion with descriptions of the prospect of misuse, the fear of mass destruction, and the singularity. In this paper I address an under-explored category of existential risk posed by AI, namely emotional risk. Values are a main source of emotions. By challenging some of our most essential values, AI systems are therefore likely to expose us to emotional risks such as loss of care and loss of meaning. Part one presents a study of a leading bank in Germany where an AI system was implemented to replace humans in decision-making processes. Part two explains why humans actively make use of values to make decisions. Part three shows the connection between values and emotions. Part four relates parts two and three to the bank study by giving concrete examples of how the employees saw their roles as workers, and how the relationships to their customers changed emotionally after the AI system was implemented to make decisions.