Kathleen A. Creel

Mistakes are inevitable, but fortunately human mistakes are typically heterogenous. Using the same machine learning model for high stakes decisions creates consistency while amplifying the weaknesses, biases, and idiosyncrasies of the original model. When the same person re-encounters the same model or models trained on the same dataset, she might be wrongly rejected again and again. Thus algorithmic monoculture could lead to consistent ill-treatment of individual people by homogenizing the decision outcomes they experience. Is it wrong to allow the quirks of an algorithmic system to consistently exclude a small number of people from consequential opportunities? This paper will argue that consistent and arbitrary exclusion is often unfair, even when it is divorced from group-based bias or discrimination.

The Rashomon set of equally-good models promises less discriminatory algorithms, reduced outcome homogenization, and fairer decisions through model ensembles or reconciliation. However, we argue from the perspective of allocation multiplicity that these promises may remain unfulfilled. When there are more qualified candidates than resources available, many different allocations of scarce resources can achieve the same utility. This space of equal-utility allocations may not be faithfully reflected by the Rashomon set, as we show in a case study of healthcare allocations. We attribute these unfulfilled promises to several factors: limitations in empirical methods for sampling from the Rashomon set, the standard practice of deterministically selecting individuals with the lowest risk, and structural biases that cause all equally-good models to view some qualified individuals as inherently risky.

We present a structural approach toward achieving equal opportunity in systems of algorithmic decision-making called algorithmic pluralism. Algorithmic pluralism describes a state of affairs in which no set of algorithms severely limits access to opportunity, allowing individuals the freedom to pursue a diverse range of life paths. To argue for algorithmic pluralism, we adopt Joseph Fishkin's theory of bottlenecks, which focuses on the structure of decision-points that determine how opportunities are allocated. The theory contends that each decision-point or bottleneck limits access to opportunities with some degree of severity and legitimacy. We extend Fishkin's structural viewpoint and use it to reframe existing systemic concerns about equal opportunity in algorithmic decision-making, such as patterned inequality and algorithmic monoculture. In proposing algorithmic pluralism, we argue for the urgent priority of alleviating severe bottlenecks in algorithmic decision-making. We contend that there must be a pluralism of opportunity available to many different individuals in order to promote equal opportunity in a systemic way. We further show how this framework has several implications for system design and regulation through current debates about equal opportunity in algorithmic hiring.

Scientific hedges are communicative devices used to qualify and weaken scientific claims. Gregor Betz has argued—unconvincingly, we think—that hedging can rescue the value-free ideal for science. Nevertheless, Betz is onto something when he suggests there are political principles that recommend scientists hedge public-facing claims. In this article, we recast this suggestion using the notion of public justification. We formulate and reject a Rawlsian argument that locates the justification for hedging in its ability to forge consensus. On our alternative proposal, hedging is often justified because it renders scientific claims as publicly accessible reasons.

While feminist critiques of AI are increasingly common in the scholarly literature, they are by no means new. Alison Adam’s Artificial Knowing (1998) brought a feminist social and epistemological stance to the analysis of AI, critiquing the symbolic AI systems of her day and proposing constructive alternatives. In this paper, we seek to revisit and renew Adam’s arguments and methodology, exploring their resonances with current feminist concerns and their relevance to contemporary machine learning. Like Adam, we ask how new AI methods could be adapted for feminist purposes and what role new technologies might play in addressing concerns raised by feminist epistemologists and theorists about algorithmic systems. In particular, we highlight distributed and federated learning as providing partial solutions to the power-oriented concerns that have stymied efforts to make machine learning systems more representative and pluralist.

This article examines the complaint that arbitrary algorithmic decisions wrong those whom they affect. It makes three contributions. First, it provides an analysis of what arbitrariness means in this context. Second, it argues that arbitrariness is not of moral concern except when special circumstances apply. However, when the same algorithm or different algorithms based on the same data are used in multiple contexts, a person may be arbitrarily excluded from a broad range of opportunities. The third contribution is to explain why this systemic exclusion is of moral concern and to offer a solution to address it.