Juan Manuel Durán

Human-AI complementarity is the claim that a human supported by an AI system can outperform either alone in a decision-making process. Since its introduction in the human-AI interaction literature, it has gained traction by generalizing the reliance paradigm and by offering a more practical alternative to the contested construct of 'trust in AI.' Yet complementarity faces key theoretical challenges: it lacks precise theoretical anchoring, it is formalized just as a post hoc indicator of relative predictive accuracy, it remains silent about other desiderata of human-AI interactions and it abstracts away from the magnitude-cost profile of its performance gain. As a result, complementarity is difficult to obtain in empirical settings. In this work, we leverage epistemology to address these challenges by reframing complementarity within the discourse on justificatory AI. Drawing on computational reliabilism, we argue that historical instances of complementarity function as evidence that a given human-AI interaction is a reliable epistemic process for a given predictive task. Together with other reliability indicators assessing the alignment of the human-AI team with the epistemic standards and socio-technical practices, complementarity contributes to the degree of reliability of human-AI teams when generating predictions. This supports the practical reasoning of those affected by these outputs -- patients, managers, regulators, and others. In summary, our approach suggests that the role and value of complementarity lies not in providing a relative measure of predictive accuracy, but in helping calibrate decision-making to the reliability of AI-supported processes that increasingly shape everyday life.

Epistemic transparency is often proposed as a solution to algorithmic opacity, wherein revealing the inner logic of an algorithm provides reasons or supporting evidence for the justification of its outputs. I argue that transparency is defective in belief formation and thus inadequate as an epistemology of algorithms. Two objections are developed: \textit{transparency regress} and \textit{bootstrapping}, both rooted in transparency’s status as a time-sliced, `outsourced' inferentialist epistemology. While weaker forms---such as contextual transparency---are briefly considered, the central argument remains that transparency fails to provide justification and should be abandoned as an epistemology of algorithms.

In a reliabilist epistemology of algorithms, a high frequency of accurate output representations is indicative of the algorithm’s reliability. Recently, Humphreys challenged this assumption, arguing that reliability depends not only on frequency but also on the quality of outputs. Specifically, he contends that radical and egregious misrepresentations have a distinct epistemic impact on our assessment of an algorithm’s reliability, regardless of the frequency of their occurrence. He terms these statistically insignificant but serious errors (SIS-Errors) and maintains that their occurrence warrants revoking our epistemic attitude towards the algorithm’s reliability. This article seeks to defend reliabilist epistemologies of algorithms against the challenge posed by SIS-Errors. To this end, I draw upon computational reliabilism as a foundational framework and articulate epistemological conditions designed to prevent SIS-Errors and thus preserve algorithmic reliability.

The previous chapter made a distinction between knowing and understanding. In computer simulations, this distinction allows us to set apart knowing when researchers trust the results and when they understand them. In this chapter, we explore different forms of understanding by means of using computer simulations. To this end, I have divided the chapter between epistemic functions that have a linguistic form, from those that are characterized for having a non-linguistic form. This distinction is meant to better help categorize the different ways in which researchers obtain understanding of the world that surround us by means of using computer simulations. Indeed, sometimes computer simulations open up the world to us in the form of symbols (e.g., by the use of mathematics, computer code, logic, numerical representation), whereas sometimes the world is accessed through visualizations and sounds. In the following, I analyze studies on scientific explanation, predictions, and exploratory strategies as linguistic forms that provide understanding of the world, and visualization as a case for non-linguistic forms.

In previous chapters, we have discussed how philosophers, scientists, and engineers alike construct the idea that computer simulations offer a ‘new epistemology’ for scientific practice. By this they meant that computer simulations introduce new—and perhaps unprecedented—forms of knowing and understanding the surrounding world, forms that were not available before. Whereas scientists and engineers emphasize the scientific novelty of computer simulations, philosophers try to appraise computer simulations for their philosophical virtues. The truth of the former claim is out of question, the latter, however, is more controversial.

Relying on computer simulations and trusting their results is key for the epistemic future of this new research methodology. The questions that interest us in this chapter are how do researchers typically build reliability on computer simulations? and what exactly would it mean to trust results of computer simulations? When we attempt to answer these questions, a dilemma is raised. On the one hand, it seems that a machine cannot be entirely reliable in the sense that they are not capable of rendering absolutely correct results.

Theories, models, experimental set-ups, prototypes: these are some of the typical units of analysis found in standard scientific and engineering work. Science and engineering are of course populated with other, equally decisive units of analysis that facilitate our description and knowledge of the world. These include hypotheses, conjectures, postulates, and a host of theoretical machinery. Computer simulations are the new acquisition in the scientific and engineering arena that count as novel units of analysis.

In this paper, we discuss epistemic and ethical concerns brought about by machine learning (ML) systems implemented in medicine. We begin by fleshing out the logic underlying a common approach in the specialized literature (which we call the informativeness account). We maintain that the informativeness account limits its analysis to the impact of epistemological issues on ethical concerns without assessing the bearings that ethical features have on the epistemological evaluation of ML systems. We argue that according to this methodological approach, epistemological issues are instrumental to and autonomous of ethical considerations. This means that the informativeness account considers epistemological evaluation uninfluenced and unregulated by an ethical counterpart. Using an example that does not square well into the informativeness account, we argue for ethical assessments that have a substantial influence on the epistemological assessment of ML and that such influence should not be understood as merely informative but rather regulatory. Drawing on the case analyzed, we claim that within the theoretical framework of the informativeness approach, forms of epistemic injustice—especially epistemic objectification—remain unaddressed. Our analysis should motivate further research investigating the regulatory role that ethical elements play in the epistemology of ML.

Achieving trustworthy AI is increasingly considered an essential desideratum to integrate AI systems into sensitive societal fields, such as criminal justice, finance, medicine, and healthcare, among others. For this reason, it is important to spell out clearly its characteristics, merits, and shortcomings. This article is the first survey in the specialized literature that maps out the philosophical landscape surrounding trust and trustworthiness in AI. To achieve our goals, we proceed as follows. We start by discussing philosophical positions on trust and trustworthiness, focusing on interpersonal accounts of trust. This allows us to explain why trust, in its most general terms, is to be understood as reliance plus some “extra factor”. We then turn to the first part of the definition provided, i.e., reliance, and analyze two opposing approaches to establishing AI systems’ reliability. On the one hand, we consider transparency and, on the other, computational reliabilism. Subsequently, we focus on debates revolving around the “extra factor”. To this end, we consider viewpoints that most actively resist the possibility and desirability of trusting AI systems before turning to the analysis of the most prominent advocates of it. Finally, we take up the main conclusions of the previous sections and briefly point at issues that remain open and need further attention.

In this paper, we discuss epistemic and ethical concerns brought about by machine learning (ML) systems implemented in medicine. We begin by fleshing out the logic underlying a common approach in the specialized literature (which we call the _informativeness account_). We maintain that the informativeness account limits its analysis to the impact of epistemological issues on ethical concerns without assessing the bearings that ethical features have on the epistemological evaluation of ML systems. We argue that according to this methodological approach, epistemological issues are _instrumental_ to and _autonomous_ of ethical considerations. This means that the informativeness account considers epistemological evaluation uninfluenced and unregulated by an ethical counterpart. Using an example that does not square well into the _informativeness account_, we argue for ethical assessments that have a substantial influence on the epistemological assessment of ML and that such influence should not be understood as merely informative but rather regulatory. Drawing on the case analyzed, we claim that within the theoretical framework of the informativeness approach, forms of epistemic injustice—especially _epistemic objectification_—remain unaddressed. Our analysis should motivate further research investigating the regulatory role that ethical elements play in the epistemology of ML.

Techniques from artificial intelligence (AI) can be used in forensic evidence evaluation and are currently applied in biometric fields. However, it is generally not possible to fully understand how and why these algorithms reach their conclusions. Whether and how we should include such ‘black box’ algorithms in this crucial part of the criminal law system is an open question that has not only scientific but also ethical, legal, and philosophical angles. Ideally, the question should be debated by people with diverse backgrounds. Here, we present a view on the question from the philosophy of science angle: computational reliabilism (CR). CR posits that we are justified in believing the output of an AI system, if we have grounds for believing its reliability. Under CR, these grounds are classified into ‘reliability indicators’ of three types: technical, scientific, and societal. This framework enables debates on the suitability of AI methods for forensic evidence evaluation that take a wider view than explainability and validation. We argue that we are justified in believing the AI’s output for forensic comparison of voices and forensic comparison of faces. Technical indicators include the validation of the AI algorithm in itself, validation of its application in the forensic setting, and case-based validation. Scientific indicators include the simple notion that we know faces and voices contain identifying information along with operationalizing well-established metrics and forensic practices. Societal indicators are the emerging scientific consensus on the use of these methods, as well as their application and interpretation by well-educated and certified practitioners. We expect expert witnesses to rely more on technical indicators to be justified in believing AIsystems, and triers-of-fact to rely more on societal indicators to believe the expert witness supported by the AIsystem.

The social aspects of causality in medicine and healthcare have been emphasized in recent debates in the philosophy of science as crucial factors that need to be considered to enable, among others, appropriate interventions in public health. Therefore, it seems central to recognize the bearing of social causes (broadly understood, e.g., social inequalities and socio-economic status) in bringing about certain concrete pathologies. Being aware of the relevance of social causes in medicine and healthcare is particularly important in the face of the role that artificial intelligence-based systems (such as machine learning algorithms) are increasingly playing in these high-stakes fields. In fact, these systems bear the dangerous potential of concealing relevant social causes. This is highly problematic not only because it reinforces issues of distributive injustice but also because it can pave the way for issues of epistemic injustice. The central aim of this chapter is to make a first effort to point out possible connections between the importance of recognizing social causes in medicine and healthcare and forms of epistemic injustice.

This chapter is interested in the epistemology of algorithms. As I intend to approach the topic, this is an issue about epistemic justification. Current approaches to justification emphasize the transparency of algorithms, which entails elucidating their internal mechanisms –such as functions and variables– and demonstrating how (or that) these produce outputs. Thus, the mode of justification through transparency is contingent on what can be shown about the algorithm and, in this sense, is internal to the algorithm. In contrast, I advocate for an externalist epistemology of algorithms that I term computational reliabilism (CR). While I have previously introduced and examined CR in the field of computer simulations ([42, 53, 4]), this chapter extends this reliabilist epistemology to encompass a broader spectrum of algorithms utilized in various scientific disciplines, with a particular emphasis on machine learning applications. At its core, CR posits that an algorithm’s output is justified if it is produced by a reliable algorithm. A reliable algorithm is one that has been specified, coded, used, and maintained utilizing reliability indicators. These reliability indicators stem from formal methods, algorithmic metrics, expert competencies, cultures of research, and other scientific endeavors. The primary aim of this chapter is to delineate the foundations of CR, explicate its operational mechanisms, and outline its potential as an externalist epistemology of algorithms.

We would like to thank the authors of the commentaries for their critical appraisal of our feature article, Who is afraid of black box algorithms?1 Their comments, suggestions and concerns are various, and we are glad that our article contributes to the academic debate about the ethical and epistemic conditions for medical Explanatory AI. We would like to bring to attention a few issues that are common worries across reviewers. Most prominently are the merits of computational reliabilism —in particular, when promoted as an alternative to transparency—and CR as necessary but not sufficient for delivering trust. We finalise our response by addressing concerns about the place and role of artificial intelligence in medical decision-making and the physician’s responsibilities. We understand the concerns and reservations that some of the reviewers express regarding the epistemic merits of CR. We believe that, in part, this is due to a practice too deeply rooted in transparency. But on …

The use of black box algorithms in medicine has raised scholarly concerns due to their opaqueness and lack of trustworthiness. Concerns about potential bias, accountability and responsibility, patient autonomy and compromised trust transpire with black box algorithms. These worries connect epistemic concerns with normative issues. In this paper, we outline that black box algorithms are less problematic for epistemic reasons than many scholars seem to believe. By outlining that more transparency in algorithms is not always necessary, and by explaining that computational processes are indeed methodologically opaque to humans, we argue that the reliability of algorithms provides reasons for trusting the outcomes of medical artificial intelligence. To this end, we explain how computational reliabilism, which does not require transparency and supports the reliability of algorithms, justifies the belief that results of medical AI are to be trusted. We also argue that several ethical concerns remain with black box algorithms, even when the results are trustworthy. Having justified knowledge from reliable indicators is, therefore, necessary but not sufficient for normatively justifying physicians to act. This means that deliberation about the results of reliable algorithms is required to find out what is a desirable action. Thus understood, we argue that such challenges should not dismiss the use of black box algorithms altogether but should inform the way in which these algorithms are designed and implemented. When physicians are trained to acquire the necessary skills and expertise, and collaborate with medical informatics and data scientists, black box algorithms can contribute to improving medical care.

When one wants to use citizen input to inform policy, what should the standards of informedness on the part of the citizens be? While there are moral reasons to allow every citizen to participate and have a voice on every issue, regardless of education and involvement, designers of participatory assessments have to make decisions about how to structure deliberations as well as how much background information and deliberation time to provide to participants. After assessing different frameworks for the relationship between science and society, we use Philip Kitcher's framework of Well-Ordered Science to propose an epistemic standard on how citizen deliberations should be structured. We explore what potential standards follow from this epistemic framework focusing on significance versus scientific and engineering expertise. We argue that citizens should be tutored on the historical context of why scientific questions became significant and deemed scientifically and socially valuable, and if citizens report that they are capable of weighing in on an issue then they should be able to do so. We explore what this standard can mean by looking at actual citizen deliberations tied to the 2014 NASA ECAST Asteroid Initiative Citizen forums. We code different vignettes of citizens debating alternative approaches for Mars exploration based upon what level of information seemed to be sufficient for them to feel comfortable in making a policy position. The analysis provides recommendations on how to design and assess future citizen assessments grounded in properly conveying the historical value context surrounding a scientific issue and trusting citizens to seek out sufficient information to deliberate.

Many studies in big data focus on the uses of data available to researchers, leaving without treatment data that is on the servers but of which researchers are unaware. We call this dark data, and in this article, we present and discuss it in the context of high-performance computing facilities. To this end, we provide statistics of a major HPC facility in Europe, the High-Performance Computing Center Stuttgart. We also propose a new position tailor-made for coping with dark data and general data management. We call it the scientific data officer and we distinguish it from other standard positions in HPC facilities such as chief data officers, system administrators, and security officers. In order to understand the role of the SDO in HPC facilities, we discuss two kinds of responsibilities, namely, technical responsibilities and ethical responsibilities. While the former are intended to characterize the position, the latter raise concerns—and proposes solutions—to the control and authority that the SDO would acquire.

Many philosophical accounts of scientific models fail to distinguish between a simulation model and other forms of models. This failure is unfortunate because there are important differences pertaining to their methodology and epistemology that favor their philosophical understanding. The core claim presented here is that simulation models are rich and complex units of analysis in their own right, that they depart from known forms of scientific models in significant ways, and that a proper understanding of the type of model simulations are fundamental for their philosophical assessment. I argue that simulation models can be distinguished from other forms of models by the many algorithmic structures, representation relations, and new semantic connections involved in their architecture. In this article, I reconstruct a general architecture for a simulation model, one that faithfully captures the complexities involved in most scientific research with computer simulations. Furthermore, I submit that a new methodology capable of conforming such architecture into a fully functional, computationally tractable computer simulation must be in place. I discuss this methodology—what I call recasting—and argue for its philosophical novelty. If these efforts are heading towards the right interpretation of simulation models, then one can show that computer simulations shed new light on the philosophy of science. To illustrate the potential of my interpretation of simulation models, I briefly discuss simulation-based explanations as a novel approach to questions about scientific explanation.

A chronicled approach to the notion of computer simulations shows that there are two predominant interpretations in the specialized literature. According to the first interpretation, computer simulations are techniques for finding the set of solutions to a mathematical model. I call this first interpretation the problem-solving technique viewpoint. In its second interpretation, computer simulations are considered to describe patterns of behavior of a target system. I call this second interpretation the description of patterns of behavior viewpoint of computer simulations. This article explores these two interpretations of computer simulations from three different angles. First, I collect a series of definitions of computer simulation from the historical record. I track back definitions to the early 1960s and show how each viewpoint shares similar interpretations of computer simulations—ultimately clustering into the two viewpoints aforementioned. This reconstruction also includes the most recent literature. Second, I unpack the philosophical assumptions behind each viewpoint, with a special emphasis on their differences. Third, I discuss the philosophical implications of each viewpoint in the context of the recent discussion on the logic of scientific explanation for computer simulations.

There is a widely extended image of computer software as some sort of ‘black box,’ where it does not matter how it internally works, but rather what sort of results are obtained given certain input values. By approaching computer software this way, many philosophical issues are hidden, neglected, or simply misunderstood. This article discusses three units of analysis of computer software, namely, specifications, algorithms, and computer processes. The aim is to understand the scientific and engineering practices supporting each unit of analysis, as well as to analyze their methodology, ontology, and epistemology.

Many studies in big data focus on the uses of data available to researchers, leaving without treatment data that is on the servers but of which researchers are unaware. We call this dark data, and in this article, we present and discuss it in the context of high-performance computing facilities. To this end, we provide statistics of a major HPC facility in Europe, the High-Performance Computing Center Stuttgart. We also propose a new position tailor-made for coping with dark data and general data management. We call it the scientific data officer and we distinguish it from other standard positions in HPC facilities such as chief data officers, system administrators, and security officers. In order to understand the role of the SDO in HPC facilities, we discuss two kinds of responsibilities, namely, technical responsibilities and ethical responsibilities. While the former are intended to characterize the position, the latter raise concerns—and proposes solutions—to the control and authority that the SDO would acquire.

Several philosophical issues in connection with computer simulations rely on the assumption that results of simulations are trustworthy. Examples of these include the debate on the experimental role of computer simulations :483–496, 2009; Morrison in Philos Stud 143:33–57, 2009), the nature of computer data Computer simulations and the changing face of scientific experimentation, Cambridge Scholars Publishing, Barcelona, 2013; Humphreys, in: Durán, Arnold Computer simulations and the changing face of scientific experimentation, Cambridge Scholars Publishing, Barcelona, 2013), and the explanatory power of computer simulations :277–292, 2008; Durán in Int Stud Philos Sci 31:27–45, 2017). The aim of this article is to show that these authors are right in assuming that results of computer simulations are to be trusted when computer simulations are reliable processes. After a short reconstruction of the problem of epistemic opacity, the article elaborates extensively on computational reliabilism, a specified form of process reliabilism with computer simulations located at the center. The article ends with a discussion of four sources for computational reliabilism, namely, verification and validation, robustness analysis for computer simulations, a history of successful implementations, and the role of expert knowledge in simulations.

Una imagen muy generalizada a la hora de entender el software de computador es la que lo representa como una “caja negra”: no importa realmente saber qué partes lo componen internamente, sino qué resultados se obtienen de él según ciertos valores de entrada. Al hacer esto, muchos problemas filosóficos son ocultados, negados o simplemente mal entendidos. Este artículo discute tres unidades de análisis del software de computador, esto es, las especificaciones, los algoritmos y los procesos computacionales. El objetivo central es entender las prácticas cientficas e ingenieriles detrás de cada unidad de software, así como analizar su metodología, ontología y epistemología.

This book addresses key conceptual issues relating to the modern scientific and engineering use of computer simulations. It analyses a broad set of questions, from the nature of computer simulations to their epistemological power, including the many scientific, social and ethics implications of using computer simulations. The book is written in an easily accessible narrative, one that weaves together philosophical questions and scientific technicalities. It will thus appeal equally to all academic scientists, engineers, and researchers in industry interested in questions related to the general practice of computer simulations.