Kevin Baum

Jeder von uns ist bloß einer unter vielen, und wir alle stehen in wechselseitigen Beziehungen zueinander. Daher sind die Folgen unseres Handelns oft zumindest teilweise durch das Handeln anderer bestimmt. Was eine Person erreichen (oder ‚vermasseln‘) kann, hängt damit gemeinhin auch davon ab, was andere tun. Diese scheinbar triviale Erkenntnis bedroht jedoch grundlegende Überzeugungen des Utilitarismus. Denn ungünstige gegenseitige Abhängigkeiten erlauben Situationen, in denen mehrere Akteure zusammen suboptimale, gar katastrophale Folgen hervorbringen können, während prima facie keiner der Beteiligten durch einseitiges Andershandeln einen Unterschied zum Besseren machen kann. Dann jedoch, so scheint es, handelt jeder richtig im Sinne des Utilitarismus. Da der Utilitarismus aber zumindest auf einen Gleichklang zwischen richtigem Handeln und dem Hervorbringen bestmöglicher Folgen zielt und ihm vielleicht gar theoretisch verpflichtet ist, entsteht das Problem kollektiven Handelns.

As AI systems increasingly permeate high-stakes decisionmaking, the terminology regarding human involvement - Human-in-the-Loop (HITL), Human-on-the-Loop (HOTL), and Human Oversighthas become vexingly ambiguous. This ambiguity complicates interdisciplinary collaboration between computer science, law, philosophy, psychology, and sociology and can lead to regulatory uncertainty. We propose a clarification grounded in causal structure, focused on human involvement during the runtime of AI systems. The distinction between HITL and HOTL, we argue, is not primarily spatial but causal: HITL is constitutive (a human contribution is necessary for the decision output), while HOTL is corrective (external to the primary causal chain, capable of preventing or modifying outputs). Within HOTL, we distinguish three temporal modes - synchronous, asynchronous, and anticipatorysituated within a nested model of provider and deployer runtime that clarifies their different capacities for intervention. A second, orthogonal dimension captures cognitive integration: whether human and machine operate as complementary or hybrid intelligence, yielding four structurally distinct configurations. Finally, we distinguish these descriptive categories from the normative requirements they serve: statutory "Human Oversight" is a specific normative mode of HOTL that demands not merely a corrective causal position, but genuine preparedness and capacity for effective intervention. Because the same person may occupy both HITL and HOTL roles simultaneously, we argue that this role duality must be treated as a design problem requiring architectural and epistemic mitigation rather than mere acknowledgment.

Building trustworthy AI comes with numerous challenges, ranging from robustness and fairness to explainability for effective human oversight and responsible decision-making. Interdisciplinary collaboration is key for tackling these challenges – fortunately, as the AI community grows, finding shared understanding and common ground between relevant fields becomes easier, because more and more researchers with interdisciplinary backgrounds are entering the field. This paves the way for responsible AI development.My personal AI mission: As a philosopher and computer scientist, I am driven to advance responsible AI development by promoting interdisciplinary dialogue and integrating ethical considerations into core research practices. My mission is to create an environment where appropriate trust assessments in AI becomes the norm, ensuring technology serves humanity in a just and responsible manner.

According to Maximizing Objective Act-Consequentialism (MOAC)—more a family of theories than a specific doctrine—the concepts of the right and the best are closely intertwined. moac theories assert that an action is right if and only if no alternative action has better consequences. This criterion of rightness seems, however, to be an expression of a more general view, according to which the ‘core function’ of morality consists in implicitly coordinating collective actions: those actions that, if carried out, lead to the morally best world that moral agents can collectively bring about are to be designated as right. This idea, prominently referred to as the Principle of Moral Harmony by Fred Feldman (1980), was considered unchallenged dogma within the consequentialist community until the second half of the 20th century (for example, see Baier 1958; Bentham 2007; Castañeda 1974; Mackie 1977). However, whether moac theories can meet this expectation is questionable. Various circumstances—overdetermination and preemption, as well as the apparent existence of effects that, considered in isolation, are negligible but accumulate into significant harm—seem to allow the existence of collective decision situations in which combinations of actions yield collectively suboptimal results, even though no agent could have made a difference for the better by acting differently unilaterally. Consequently, such actions are apparently right according to moac theories, yet they lead to suboptimal outcomes. This puzzle, known as the Challenge of Collective Action, has questioned consequentialism for decades (see Glover and Scott-Taggart 1975; Kagan 2011; Parfit 1984; Pinkert 2015; Regan 1980; Zimmerman 1996). This dissertation aims to reconstruct and understand the Challenge of Collective Action in its various forms and ultimately propose a novel consequentialist solution. Significantly based on game-theoretical considerations, the proposed solution results in a new and generalized moac theory, Multi-Agent Consequentialism, that is well-suited for multiple agents. The overarching aim of this work is to preserve the Principle of Moral Harmony as a fundamental motivation of consequentialist theorizing and, at the same time, to offer a decidedly objective-consequentialist solution to the Challenge of Collective Action.

The AI alignment problem comprises both technical and normative dimensions. While technical solutions focus on implementing normative constraints in AI systems, the normative problem concerns determining what these constraints should be. This paper examines justifications for democratic approaches to the normative problem—where affected stakeholders determine AI alignment—as opposed to epistocratic approaches that defer to normative experts. We analyze both instrumental justifications (democratic approaches produce better outcomes) and non-instrumental justifications (democratic approaches prevent illegitimate authority or coercion). We argue that normative and metanormative uncertainty create a justificatory gap that democratic approaches aim to fill through political rather than theoretical justification. However, we identify significant challenges for democratic approaches, particularly regarding the prevention of illegitimate coercion through AI alignment. Our analysis suggests that neither purely epistocratic nor purely democratic approaches may be sufficient on their own, pointing toward hybrid frameworks that combine expert judgment with participatory input alongside institutional safeguards against AI monopolization.

Legislation and ethical guidelines around the globe call for effective human oversight of AI-based systems in high-risk contexts – that is oversight that reliably reduces the risks otherwise associated with the use of AI-based systems. Such risks may relate to the imperfect accuracy of systems (e.g., inaccurate classifications) or to ethical concerns (e.g., unfairness of outputs). Given the significant role that human oversight is expected to play in the operation of AI-based systems, it is crucial to better understand the conditions for effective human oversight. We argue that the reliable detection of errors (as an umbrella term for inaccuracies and unfairness) is crucial for effective human oversight. We then propose that Signal Detection Theory (SDT) offers a promising framework for better understanding what affects people’s sensitivity (i.e., how well they are able to detect errors) and response bias (i.e., the tendency to report errors given a perceived evidence of an error) in detecting errors. Whereas an SDT perspective on the detection of inaccuracies is straightforward, we demonstrate its broader applicability by detailing the specifics for an SDT perspective on unfairness detection, including the need to choose a standard for (un)fairness. Additionally, we illustrate that an SDT perspective helps to better understand the conditions for effective error detection by showing examples of task-, system-, and person-related factors that may affect the sensitivity and response bias of humans tasked with detecting unfairness associated with the use of AI-based systems. Finally, we discuss future research directions for an SDT perspective on error detection.

Previous research in Explainable Artificial Intelligence (XAI) suggests that a main aim of explainability approaches is to satisfy specific interests, goals, expectations, needs, and demands regarding artificial systems (we call these “stakeholders' desiderata”) in a variety of contexts. However, the literature on XAI is vast, spreads out across multiple largely disconnected disciplines, and it often remains unclear how explainability approaches are supposed to achieve the goal of satisfying stakeholders' desiderata. This paper discusses the main classes of stakeholders calling for explainability of artificial systems and reviews their desiderata. We provide a model that explicitly spells out the main concepts and relations necessary to consider and investigate when evaluating, adjusting, choosing, and developing explainability approaches that aim to satisfy stakeholders' desiderata. This model can serve researchers from the variety of different disciplines involved in XAI as a common ground. It emphasizes where there is interdisciplinary potential in the evaluation and the development of explainability approaches.

We argue that explainable artificial intelligence (XAI), specifically reason-giving XAI, often constitutes the most suitable way of ensuring that someone can properly be held responsible for decisions that are based on the outputs of artificial intelligent (AI) systems. We first show that, to close moral responsibility gaps (Matthias 2004), often a human in the loop is needed who is directly responsible for particular AI-supported decisions. Second, we appeal to the epistemic condition on moral responsibility to argue that, in order to be responsible for her decision, the human in the loop has to have an explanation available of the system’s recommendation. Reason explanations are especially well-suited to this end and we examine whether – and how – it might be possible to make such explanations fit with AI systems. We support our claims by focusing on a case of disagreement between human in the loop and AI system.

We argue that, to be trustworthy, Computa- tional Intelligence (CI) has to do what it is entrusted to do for permissible reasons and to be able to give rationalizing explanations of its behavior which are accurate and gras- pable. We support this claim by drawing par- allels with trustworthy human persons, and we show what difference this makes in a hypo- thetical CI hiring system. Finally, we point out two challenges for trustworthy CI and sketch a mechanism which could be used to gener- ate sufficiently accurate as well as graspable rationalizing explanations for CI behavior.