Boris Babic

As artificial intelligence and machine learning (AI/ML) systems become increasingly pervasive in society, their opacity—i.e., the difficulty, and sometimes impossibility, of understanding why they make the decisions they make—has become a serious problem. This is especially true in sensitive decision-making contexts, such as criminal justice, health care, and finance, or in choices requiring allocation of scarce resources. One attempt to “open up” the AI/ML black box has been the emergence of post hoc explainability algorithms—algorithms which generate post hoc approximations to black box models. However, such algorithms have been criticized as merely providing after the fact rationalizations for the decisions these systems make. In this paper, we defend and articulate a different concept—AI/ML justifiability. We explore several ways in which an algorithm could be justifiable, and we argue that pursuing justifiability is a worthwhile goal. A key to our argument is a distinction from the philosophy of action between motivating and normative reasons: effective explanations require (but are unable to provide) motivating reasons, while effective justifications require (and can indeed provide) normative reasons alone. We conclude that as long as a model is justifiable, it can be trusted even if it cannot be explained.

Are we in position to warrantedly establish whether a given artificial intelligence (AI) system is conscious? In short, can we warrantedly establish whether there is AI Consciousness (AIC)? We argue for a provisionally pessimistic answer—probably not—by attending to the traditional problem of other minds. For each of the main response strategies to this problem, we argue that even if the strategy works to establish that other humans and some non-human animals are conscious, the prospect of the strategy’s working to establish that there is AIC—and more generally, to establish whether there is AIC—is unlikely. The upshot is that there a distinctively difficult problem of other AI minds, whose lack of principled resolution is likely to undermine attempts to answer questions about the personal, moral, and agential status of AI systems.

Direct-to-consumer medical artificial intelligence/machine learning applications are increasingly used for a variety of diagnostic assessments, and the emphasis on telemedicine and home healthcare during the COVID-19 pandemic may further stimulate their adoption. In this Perspective, we argue that the artificial intelligence/machine learning regulatory landscape should operate differently when a system is designed for clinicians/doctors as opposed to when it is designed for personal use. Direct-to-consumer applications raise unique concerns due to the nature of consumer users, who tend to be limited in their statistical and medical literacy and risk averse about their health outcomes. This creates an environment where false alarms can proliferate and burden public healthcare systems and medical insurers. While similar situations exist elsewhere in medicine, the ease and frequency with which artificial intelligence/machine learning apps can be used, and their increasing prevalence in the consumer market, calls for careful reflection on how to effectively regulate them. We suggest regulators should strive to better understand how consumers interact with direct-to-consumer medical artificial intelligence/machine learning apps, particularly diagnostic ones, and this requires more than a focus on the system’s technical specifications. We further argue that the best regulatory review would also consider such technologies’ social costs under widespread use.

Artificial intelligence and machine learning (AI/ML) algorithms are increasingly developed in health care for diagnosis and treatment of a variety of medical conditions (1). However, despite the technical prowess of such systems, their adoption has been challenging, and whether and how much they will actually improve health care remains to be seen. A central reason for this is that the effectiveness of AI/ML-based medical devices depends largely on the behavioral characteristics of its users, who, for example, are often vulnerable to well-documented biases or algorithmic aversion (2). Many stakeholders increasingly identify the so-called black-box nature of predictive algorithms as the core source of users' skepticism, lack of trust, and slow uptake (3, 4). As a result, lawmakers have been moving in the direction of requiring the availability of explanations for black-box algorithmic decisions (5). Indeed, a near-consensus is emerging in favor of explainable AI/ML among academics, governments, and civil society groups. Many are drawn to this approach to harness the accuracy benefits of noninterpretable AI/ML such as deep learning or neural nets while also supporting transparency, trust, and adoption. We argue that this consensus, at least as applied to health care, both overstates the benefits and undercounts the drawbacks of requiring black-box algorithms to be explainable.

The large literature on the ethics of statistical evidence and its use in courtrooms is premised on the assumption that statistical evidence can be highly probabilifying (i.e. that it can support a very high credence). When statistical evidence seems to render a morally problematic proposition highly probable, scholars then divide over how they respond to the problem: some attempt to identify a different epistemic problem with the inference, while others grant the inference’s epistemic legitimacy but object to the conclusion on a moral or political basis. In this chapter, the authors argue that, in fact, it is very difficult to construct realistic cases in which statistical evidence is highly probabilifying. The authors explain how noisy data resulting from misclassification—a common feature of most realistic cases—drastically reduces statistical evidence’s probative value. As a result, it is not necessary to advert to additional theoretical bells and whistles to explain why one ought not form a high credence based on statistical evidence in most realistic cases.

While the literature on putting a “human in the loop” in artificial intelligence (AI) and machine learning (ML) has grown significantly, limited attention has been paid to how human expertise ought to be combined with AI/ML judgments. This design question arises because of the ubiquity and quantity of algorithmic decisions being made today in the face of widespread public reluctance to forgo human expert judgment. To resolve this conflict, we propose that human expert judges be included via appeals processes for review of algorithmic decisions. Thus, the human intervenes only in a limited number of cases and only after an initial AI/ML judgment has been made. Based on an analogy with appellate processes in judiciary decision-making, we argue that this is, in many respects, a more efficient way to divide the labor between a human and a machine. Human reviewers can add more nuanced clinical, moral, or legal reasoning, and they can consider case-specific information that is not easily quantified and, as such, not available to the AI/ML at an initial stage. In doing so, the human can serve as a crucial error correction check on the AI/ML, while retaining much of the efficiency of AI/ML’s use in the decision-making process. In this paper we develop these widely applicable arguments while focusing primarily on examples from the use of AI/ML in medicine, including organ allocation, fertility care, and hospital readmission.

Explainability in artificial intelligence and machine learning (“AI/ML”) is emerging as a leading area of academic research and a topic of significant regulatory concern. Indeed, a near-consensus exists in favor of explainable AI/ML among academics, governments, and civil society groups. In this project, we challenge this prevailing trend. We argue that for explainability to be a moral requirement– and even more so for it to be a legal requirement– it should satisfy certain desiderata which it currently does not, and possibly cannot. In particular, we will argue that the currently prevailing approaches to explainable AI/ML are (1) incapable of guiding our action and planning, (2) incapable of making transparent the actual reasons underlying an automated decision, and (3) incapable of underwriting normative (moral/legal) judgments, such as blame and resentment. This stems from the post hoc nature of the explanations offered by prevailing explainability algorithms. We will explain that these algorithms are “insincere-by-design,” so to speak. And this renders them of very little value to legislators or policymakers who are interested in (the laudable goal of) transparency in automated decision making. There is, however, an alternative– interpretable AI/ML which we will distinguish from explainable AI/ML. Interpretable AI/ML can be useful where it is appropriate, but represents real tradeoffs and in some instances (in medicine and elsewhere) adopting an interpretable AI/ML may mean adopting a less accurate AI/ML. We argue that it is better to face those trade-offs head on, rather than embrace the fool’s gold of explainable AI/ML.

This project represents the first systematic assessment of the US Food and Drug Administration’s postmarket surveillance of legally marketed artificial intelligence and machine learning based medical devices. We focus on the Manufacturer and User Facility Device Experience database—the FDA’s central tool for tracking the safety of marketed AI/ML devices. In particular, we evaluate the data pertaining to adverse events associated with approximately 950 medical devices incorporating AI/ML functions for devices approved between 2010 through 2023, and we find that the existing system is insufficient for properly assessing the safety and effectiveness of AI/ML devices. Inparticular,we make three contributions: (1) characterize the adverse event reports for such devices, (2) examine the ways in which the existing FDA adverse reporting system for medical devices falls short, and (3) suggest changes FDA might consider in its approach to adverse event reporting for devices incorporating AI/ML functions.

This paper suggests a new normative approach for combining beliefs. We call it the evidence-first method. Instead of aggregating credences alone, as the prevailing approaches, we focus instead on eliciting a group’s full probability distribution on the basis of the evidence available to its members. This is an altogether different way of combining beliefs. The method has four main benefits: (1) it captures the weight, or resilience, of a group’s belief; (2) it is sensitive to correlation among its individuals; (3) it is commutative under updating; and (4) it can be seen as a generalization of weighted averaging and likelihood ratio approaches. More broadly, it encourages an overall rethinking of the belief combination problem away from aggregating bare credences and toward appropriately combining evidence.

In this paper we develop a general theory of algorithmic fairness. Drawing on Johnson King and Babic’s work on moral encroachment, on Gary Becker’s work on labor market discrimination, and on Strawson’s idea of resentment and indignation as responses to violations of the demand for goodwill toward oneself and others, we locate attitudes to fairness in an agent’s utility function. In particular, we first argue that fairness is a matter of a decision-maker’s relative concern for the plight of people from different groups, rather than of the outcomes produced for different groups. We then show how an agent’s preferences, including in particular their attitudes to error, give rise to their decision thresholds. Tying these points together, we argue that the agent’s relative degrees of concern for different groups manifest in a difference in decision thresholds applied to these groups.

This paper discusses a novel problem at the intersection of ethics and epistemology: there can be cases in which moral considerations seem to "encroach'' upon belief from multiple directions at once, and possibly to varying degrees, thereby leaving their overall effect on belief unclear. We introduce these cases -- cases of moral encroachment under moral uncertainty -- and show that they pose a problem for all predominant accounts of moral encroachment. We then address the problem, by developing a modular Bayesian framework that, we argue, is sufficiently flexible and scaleable to accommodate the multifaceted uncertainty we describe while still generating clear recommendations for an agent's beliefs. Our framework has several practical upshots, and we close by articulating them: we derive insights about the relationship between moral character and doxastic behavior and make suggestions for how to encourage people to revise their doxastic states in morally laudable ways, without deviating from core Bayesian norms.

Many philosophers have argued that statistical evidence regarding group characteristics (particularly stereotypical ones) can create normative conflicts between the requirements of epistemic rationality and our moral obligations to each other. In a recent article, Johnson-King and Babic argue that such conflicts can usually be avoided: what ordinary morality requires, they argue, epistemic rationality permits. In this article, we show that as data get large, Johnson-King and Babic’s approach becomes less plausible. More constructively, we build on their project and develop a generalized model of reasoning about stereotypes under which one can indeed avoid normative conflicts, even in a big data world, when data contain some noise. In doing so, we also articulate a general approach to rational belief updating for noisy data.

My goal in this dissertation is to start a conversation about the role of risk in the decision-theoretic assessment of partial beliefs or credences in formal epistemology. I propose a general theory of epistemic risk in terms of relative sensitivity to different types of graded error. The approach I develop is broadly inspired by the pragmatism of the American philosopher Charles Sanders Peirce and his notion of the ``economy of research.'' I express this framework in information-theoretic terms and show that epistemic risk, so understood, is dual to information entropy. As a result, every unit increase in risk comes with a corresponding unit decrease in information entropy and epistemic risk may be expressed in terms of entropic change. I explain the significance of this for the selection of priors and the Laplacian principle of indifference. I also extend this notion of epistemic risk to the assessment of updating rules, where a similar duality between risk and information holds. In the dynamic context, epistemic risk is given by cross-entropic change. Here I explore the relationship between risk, the Value of Knowledge Theorem, dynamic coherence, and the role of expected accuracy in the selection of update rules. Finally, I apply these considerations to a social institution where attitudes to error are especially salient -- namely, legal decision-making -- and argue that considerations regarding the relative severity of different types of error are central to understanding evidentiary burdens of proof and the probative value of statistical evidence.

Approximate coherentism suggests that imperfectly rational agents should hold approximately coherent credences. This norm is intended as a generalization of ordinary coherence. I argue that it may be unable to play this role by considering its application under learning experiences. While it is unclear how imperfect agents should revise their beliefs, I suggest a plausible route is through Bayesian updating. However, Bayesian updating can take an incoherent agent from relatively more coherent credences to relatively less coherent credences, depending on the data observed. Thus, comparative rationality judgments among incoherent agents are unduly sensitive to luck.

I propose a general alethic theory of epistemic risk according to which the riskiness of an agent’s credence function encodes her relative sensitivity to different types of graded error. After motivating and mathematically developing this approach, I show that the epistemic risk function is a scaled reflection of expected inaccuracy. This duality between risk and information enables us to explore the relationship between attitudes to epistemic risk, the choice of scoring rules in epistemic utility theory, and the selection of priors in Bayesian epistemology more generally.