Leslye Denisse Dias Duran

The development and implementation of new technologies in healthcare has always been a complex task to undertake. Clinical settings are spaces where multiple convergences occur. There is a multiplicity of stakeholders with a plurality of values and potentially conflicting interests, intricate regulatory protocols, and frameworks, financial constraints, and incentives from public and private actors, and the sensitive matter of dealing with human health and life, which adds a particularly strong normative relevance to existing and emerging challenges.

In Chap. 1, I presented and discussed the state-of-the-art ML applications in medical diagnosis and the opportunities they have to bring about beneficial outcomes to patients, clinicians and perhaps even to the healthcare systems at large. This was done from a medical and socio-technical perspective grounded in the actual (in the sense of both current and concrete) technical possibilities of the models and the headways made in terms of their development and deployment. Research evidence showed that some ML models in diagnosis meet and sometimes exceed benchmarks that rival those of human doctors and seemed to suggest that the models could, at the least, assist them in the diagnostic process.

The integration of Artificial Intelligence (AI) in healthcare is reshaping clinical practice, offering both opportunities for enhanced decision-making and risks of skill degradation among medical professionals. This growing impact calls for a comprehensive evaluation of its effects on medical expertise. This study presents a mixed-method literature review, combining systematic analysis with narrative synthesis to examine AI-induced deskilling and upskilling inhibition-the erosion of medical expertise and the reduction of opportunities for skill acquisition due to AI-driven decision support systems. Anchoring the discussion in the core medical competencies outlined by the Federation of Royal Colleges of Physicians of the UK-Practical Assessment of Clinical Examination Skills (PACES-MRCPUK), the systematic review identifies key vulnerabilities in physical examination, differential diagnosis, clinical judgment, and physician-patient communication. The narrative review explores broader themes related to Human–AI Interaction and the Impact of AI on Human Skills in Organizations. In response to concerns about the Second Singularity-a scenario in which decision-making autonomy is increasingly ceded to AI, weakening human oversight-this review advocates for a research agenda that prioritizes longitudinal studies, real-time monitoring of AI’s impact, and the development of frameworks to mitigate skill erosion, ensuring the preservation of professional autonomy and the safeguarding of the irreplaceable elements of human judgment in medicine and beyond.

This book seeks to navigate between the optimism that has arisen from the promise of the potential of machine learning (ML) in healthcare, and the lack of clarity about what realistic risks and benefits we can foresee. Its main aim is to develop a relational, rights-based normative approach to evaluating the distribution of burdens and benefits of implementing ML in medical diagnosis. This framework, called the "Ecosystem of Moral Constellations", assumes that every person has an equal claim to the fundamental rights necessary to lead one’s life, but recognizes that there may be conflicting interests that risk violating or infringing the rights of an individual or individuals, and that therefore an assessment of these tensions requires a situational prioritization of certain rights over others. This framework proposes to consider the normative relevance of relationships at different points of moral engagement to assess the potential tensions between these burdens and benefits of these technologies. The author argues that decisions about the implementation of AI systems require more than an assessment of technical feasibility. Instead, it is imperative to consider the different normative goals and interests of the actors involved, the material capabilities of the tools, and the role they should play in the clinical workflow.

The alignment problem in the context of large language models must consider the plurality of human values in our world. Whilst there are many resonant and overlapping values amongst the world’s cultures, there are also many conflicting, yet equally valid, values. It is important to observe which cultural values a model exhibits, particularly when there is a value conflict between input prompts and generated outputs. We discuss how the co- creation of language and cultural value impacts large language models (LLMs). We explore the constitution of the training data for GPT-3 and compare that to the world’s language and internet access demographics, as well as to reported statistical profiles of dominant values in some Nation-states. We stress tested GPT-3 with a range of value-rich texts representing several languages and nations; including some with values orthogonal to dominant US public opinion as reported by the World Values Survey. We observed when values embedded in the input text were mutated in the generated outputs and noted when these conflicting values were more aligned with reported dominant US values. Our discussion of these results uses a moral value pluralism (MVP) lens to better understand these value mutations. Finally, we provide recommendations for how our work may contribute to other current work in the field.