Karina Vold

Background: The integration of artificial intelligence (AI) into health care has become a crucial element in the digital transformation of health systems worldwide. Despite the potential benefits across diverse medical domains, a significant barrier to the successful adoption of AI systems in health care applications remains the prevailing low user trust in these technologies. Crucially, this challenge is exacerbated by the lack of consensus among experts from different disciplines on the definition of trust in AI within the health care sector. Objective: We aimed to provide the first consensus-based analysis of trust in AI in health care based on an interdisciplinary panel of experts from different domains. Our findings can be used to address the problem of defining trust in AI in health care applications, fostering the discussion of concrete real-world health care scenarios in which humans interact with AI systems explicitly. Methods: We used a combination of framework analysis and a 3-step consensus process involving 18 international experts from the fields of computer science, medicine, philosophy of technology, ethics, and social sciences. Our process consisted of a synchronous phase during an expert workshop where we discussed the notion of trust in AI in health care applications, defined an initial framework of important elements of trust to guide our analysis, and agreed on 5 case studies. This was followed by a 2-step iterative, asynchronous process in which the authors further developed, discussed, and refined notions of trust with respect to these specific cases. Results: Our consensus process identified key contextual factors of trust, namely, an AI system’s environment, the actors involved, and framing factors, and analyzed causes and effects of trust in AI in health care. Our findings revealed that certain factors were applicable across all discussed cases yet also pointed to the need for a fine-grained, multidisciplinary analysis bridging human-centered and technology-centered approaches. While regulatory boundaries and technological design features are critical to successful AI implementation in health care, ultimately, communication and positive lived experiences with AI systems will be at the forefront of user trust. Our expert consensus allowed us to formulate concrete recommendations for future research on trust in AI in health care applications. Conclusions: This paper advocates for a more refined and nuanced conceptual understanding of trust in the context of AI in health care. By synthesizing insights into commonalities and differences among specific case studies, this paper establishes a foundational basis for future debates and discussions on trusting AI in health care.

Neurotechnologies are rapidly advancing in the past few years, such that neural prostheses and brain-computer interfaces are no longer things that only appear in science fiction movies. As interactions with neurotechnologies deepen, users have reported feeling that these tools are becoming part of their own selves and minds. The hypothesis of extended cognition can accommodate this intuition, as it maintains that artifacts can become a part of their users’ minds. However, there have also been some stark examples where users have abruptly lost access to their sophisticated tools, demonstrating the sometimes vulnerable and precarious nature of certain advanced technologies. These features seem to challenge the idea that users can stand in the right relation to their tools to meet the criteria of parity that support the extended cognition theorists’ arguments. Notably, these technologies seem to violate a condition of ownership that has been appealed to on various occasions over the past two decades of literature on extended cognition. In addition to arguing that neuroprosthetics can be a part of one’s extended cognitive system, despites apparent challenges to an ownership condition, we will also review the disparate history and evaluate the current status of the ownership criterion in the literature on extended cognition. To proceed, we first review the reasons for the origin of the proposed “ownership” criterion and, second, introduce new challenges arising from emerging technologies such as neuroprosthetics. We argue that the ownership condition has at least three shortcomings: (i) ambiguity of meaning, (ii) ethical pitfalls, and (iii) it fails to capture people’s intuitions about the cognitive status of emerging neurotechnologies. Ultimately, we argue that introducing the concept of co-ownership is a necessary distinction and is better suited to explain how advanced cognitive technologies function.

Artificial Intelligence (AI) and machine learning (ML) are rapidly evolving and have already had major impacts on military capabilities in the battlefield, making new kinds of tools and tactics available. A less examined area of application for AI in a military context, however, is its impact on human strategic decision making. This article focuses on the more subtle cognitive influences of AI and how they can be strategically deployed to aid decision making around the state-level resort to force, in particular. I will argue that AI-driven technologies can be used to improve certain critical cognitive resources (e.g. memory, planning, mind-modelling, etc.) of decision makers, thereby providing valuable strategic advantages to those actors who use them successfully. At the same time, I will also caution against the risks of human decision makers becoming overly reliant on AI-support systems. Both the potential advantages and risks are areas that demand further study and consideration.

Neurotechnologies are rapidly advancing in the past few years, such that neural prostheses and brain-computer interfaces are no longer things that only appear in science fiction movies. As interactions with neurotechnologies deepen, users have reported feeling that these tools are becoming part of their own selves and minds. The hypothesis of extended cognition can accommodate this intuition, as it maintains that artifacts can become a part of their users’ minds. However, there have also been some stark examples where users have abruptly lost access to their sophisticated tools, demonstrating the sometimes vulnerable and precarious nature of certain advanced technologies. These features seem to challenge the idea that users can stand in the right relation to their tools to meet the criteria of parity that support the extended cognition theorists’ arguments. Notably, these technologies seem to violate a condition of ownership that has been appealed to on various occasions over the past two decades of literature on extended cognition. In addition to arguing that neuroprosthetics can be a part of one’s extended cognitive system, despites apparent challenges to an ownership condition, we will also review the disparate history and evaluate the current status of the ownership criterion in the literature on extended cognition. To proceed, we first review the reasons for the origin of the proposed “ownership” criterion and, second, introduce new challenges arising from emerging technologies such as neuroprosthetics. We argue that the ownership condition has at least three shortcomings: (i) ambiguity of meaning, (ii) ethical pitfalls, and (iii) it fails to capture people’s intuitions about the cognitive status of emerging neurotechnologies. Ultimately, we argue that introducing the concept of co-ownership is a necessary distinction and is better suited to explain how advanced cognitive technologies function.

Even with obvious deficiencies, large prompt-commanded multimodal models are proving to be flexible cognitive tools representing an unprecedented generality. But the directness, diversity, and degree of user interaction create a distinctive “human-centred generality” (HCG), rather than a fully autonomous one. HCG implies that —for a specific user— a system is only as general as it is effective for the user’s relevant tasks and their prevalent ways of prompting. A human-centred evaluation of general-purpose AI systems therefore needs to reflect the personal nature of interaction, tasks and cognition. We argue that the best way to understand these systems is as highly-coupled cognitive extenders, and to analyse the bidirectional cognitive adaptations between them and humans. In this paper, we give a formulation of HCG, as well as a high-level overview of the elements and trade-offs involved in the prompting process. We end the paper by outlining some essential research questions and suggestions for improving evaluation practices, which we envision as characteristic for the evaluation of general artificial intelligence in the future.

Objective: To examine the role of explainability in machine learning for healthcare (MLHC), and its necessity and significance with respect to effective and ethical MLHC application. Study Design and Setting: This commentary engages with the growing and dynamic corpus of literature on the use of MLHC and artificial intelligence (AI) in medicine, which provide the context for a focused narrative review of arguments presented in favour of and opposition to explainability in MLHC. Results: We find that concerns regarding explainability are not limited to MLHC, but rather extend to numerous well-validated treatment interventions as well as to human clinical judgment itself. We examine the role of evidence-based medicine in evaluating unexplainable treatments and technologies, and highlight the analogy between the concept of explainability in MLHC and the related concept of mechanistic reasoning in evidence-based medicine. Conclusion: Ultimately, we conclude that the value of explainability in MLHC is not intrinsic, but is instead instrumental to achieving greater imperatives such as performance and trust. We caution against the uncompromising pursuit of explainability, and advocate instead for the development of robust empirical methods to successfully evaluate increasingly inexplicable algorithmic systems.

Technological advances are bringing new light to privacy issues and changing the reasons for why privacy is important. These advances have changed not only the kind of personal data that is available to be collected, but also how that personal data can be used by those who have access to it. We are particularly concerned with how information about personal attributes inferred from collected data (such as online behaviour), can be used to tailor messages and services to specific individuals or groups. This kind of ‘personalised targeting’ has the potential to influence individuals’ perceptions, attitudes, and choices in unprecedented ways. In this paper, we argue that because it is becoming easier for companies to use collected data for influence, threats to privacy are increasingly also threats to personal autonomy—an individual’s ability to reflect on and decide freely about their values, actions, and behaviour, and to act on those choices.4 While increasing attention is directed to the ethics of how personal data is collected, we make the case that a new ethics of privacy needs to also think more rigorously about how personal data may be used, and its potential impact on personal autonomy.

The extended mind thesis prompted philosophers to think about the different shapes our minds can take as they reach beyond our brains and stretch into new technologies. Some of us rely heavily on the environment to scaffold our cognition, reorganizing our homes into rich cognitive niches, for example, or using our smartphones as swiss-army knives for cognition. But the thesis also prompts us to think about other varieties of minds and the unique forms they take. What are we to make of the exotic distributed nervous systems we see in octopuses, for example, or the complex collectives of bees? In this paper, I will argue for a robust version of the extended mind thesis that includes the possibility of extended consciousness. This thesis will open up new ways of understanding the different forms that conscious minds can take, whether human or nonhuman. The thesis will also challenge the popular belief that consciousness exists exclusively in the brain. Furthermore, despite the attention that the extended mind thesis has received, there has been relatively less written about the possibility of extended consciousness. A number of prominent defenders of the extended mind thesis have even called the idea of extended consciousness implausible. I will argue, however, that extended consciousness is a viable theory and it follows from the same ‘parity argument’ that Clark and Chalmers first advanced to support the extended mind thesis. What is more, it may even provide us with a valuable paradigm for how we understand some otherwise puzzling behaviors in certain neurologically abnormal patients as well as in some nonhuman animals.

In 2019, the IEEE launched the P7000 standards projects intended to address ethical issues in the design of autonomous and intelligent systems. This move came amidst a growing public concern over the unintended consequences of artificial intelligence (AI), compounded by the lack of an anticipatory process for attending to ethical impact within professional practice. However, the difficulty in moving from principles to practice presents a significant challenge to the implementation of ethical guidelines. Herein, we describe two complementary frameworks for integrating ethical analysis into engineering practice to help address this challenge. We then provide the outcomes of an ethical analysis informed by these frameworks, conducted within the specific context of internetdelivered therapy in digital mental health. We hope both the frameworks and analysis can provide tools and insights, not only for the context of digital healthcare, but for data-enabled and intelligent technology development more broadly.

The extended mind thesis maintains that the functional contributions of tools and artefacts can become so essential for our cognition that they can be constitutive parts of our minds. In other words, our tools can be on a par with our brains: our minds and cognitive processes can literally ‘extend’ into the tools. Several extended mind theorists have argued that this ‘extended’ view of the mind offers unique insights into how we understand, assess, and treat certain cognitive conditions. In this chapter we suggest that using AI extenders, i.e., tightly coupled cognitive extenders that are imbued with machine learning and other ‘artificially intelligent’ tools, presents both new ethical challenges and opportunities for mental health. We focus on several mental health conditions that can develop differently by the use of AI extenders for people with cognitive disorders and then discuss some of the related opportunities and challenges.

Humans and AI systems are usually portrayed as separate sys- tems that we need to align in values and goals. However, there is a great deal of AI technology found in non-autonomous systems that are used as cognitive tools by humans. Under the extended mind thesis, the functional contributions of these tools become as essential to our cognition as our brains. But AI can take cognitive extension towards totally new capabil- ities, posing new philosophical, ethical and technical chal- lenges. To analyse these challenges better, we define and place AI extenders in a continuum between fully-externalized systems, loosely coupled with humans, and fully-internalized processes, with operations ultimately performed by the brain, making the tool redundant. We dissect the landscape of cog- nitive capabilities that can foreseeably be extended by AI and examine their ethical implications. We suggest that cognitive extenders using AI be treated as distinct from other cognitive enhancers by all relevant stakeholders, including developers, policy makers, and human users.

Many authors have proposed constraining the behaviour of intelligent systems with ‘machine ethics’ to ensure positive social outcomes from the development of such systems. This paper critically analyses the prospects for machine ethics, identifying several inherent limitations. While machine ethics may increase the probability of ethical behaviour in some situations, it cannot guarantee it due to the nature of ethics, the computational limitations of computational agents and the complexity of the world. In addition, machine ethics, even if it were to be ‘solved’ at a technical level, would be insufficient to ensure positive social outcomes from intelligent systems.

Vehicle externalism maintains that the vehicles of our mental representations can be located outside of the head, that is, they need not be instantiated by neurons located inside the brain of the cogniser. But some disagree, insisting that ‘non-derived’, or ‘original’, content is the mark of the cognitive and that only biologically instantiated representational vehicles can have non-derived content, while the contents of all extra-neural representational vehicles are derived and thus lie outside the scope of the cognitive. In this paper we develop one aspect of Menary’s vehicle externalist theory of cognitive integration—the process of enculturation—to respond to this longstanding objection. We offer examples of how expert mathematicians introduce new symbols to represent new mathematical possibilities that are not yet understood, and we argue that these new symbols have genuine non-derived content, that is, content that is not dependent on an act of interpretation by a cognitive agent and that does not derive from conventional associations, as many linguistic representations do.

The extended mind thesis maintains that while minds may be centrally located in one’s brain-and-body, they are sometimes partly constituted by tools in our environment. Critics argue that we have no reason to move from the claim that cognition is embedded in the environment to the stronger claim that cognition can be constituted by the environment. I will argue that there are normative reasons, both scientific and ethical, for preferring the extended account of the mind to the rival embedded account.

Andy Clark and David Chalmers (1998) argue that certain mental states and processes can be partially constituted by objects located beyond one’s brain and body: this is their extended mind thesis (EM). But they maintain that consciousness relies on processing that is too high in speed and bandwidth to be realized outside the body (see Chalmers, 2008, and Clark, 2009). I evaluate Clark’s and Chalmers’ reason for denying that consciousness extends while still supporting unconscious state extension. I argue that their reason is not well grounded and does not hold up against foreseeable advances in technology. I conclude that their current position needs re-evaluation. If their original parity argument works as a defence of EM, they have yet to identify a good reason why it does not also work as a defence of extended consciousness. I end by advancing a parity argument for extended consciousness and consider some possible replies.

The extended mind thesis maintains that while minds may be centrally located in one?s brain-and-body, they are sometimes partly constituted by tools in our environment. Critics argue that we have no reason to move from the claim that cognition is embedded in the environment to the stronger claim that cognition can be constituted by the environment. I will argue that there are normative reasons, both scientific and ethical, for preferring the extended account of the mind to the rival embedded account. nema

Your smartphone is much more than just a phone. It can tell a more intimate story about you than your best friend. No other piece of hardware in history, not even your brain, contains the quality or quantity of information held on your phone: it ‘knows’ whom you speak to, when you speak to them, what you said, where you have been, your purchases, photos, biometric data, even your notes to yourself – and all this dating back years. In this piece I ask whether, given the role they play in our lives, our devices deserve the same legal protections as our brains and bodies?

Alan Turing, one of the fathers of computing, warned that Artificial Intelligence (AI) could one day pose an existential risk to humanity. Today, recent advancements in the field AI have been accompanied by a renewed set of existential warnings. But what exactly constitutes an existential risk? And how exactly does AI pose such a threat? In this chapter we aim to answer these questions. In particular, we will critically explore three commonly cited reasons for thinking that AI poses an existential threat to humanity: the control problem, the possibility of global disruption from an AI race dynamic, and the weaponization of AI.

Despite there being little consensus on what intelligence is or how to measure it, the media and the public have become increasingly preoccupied with the concept owing to recent accomplishments in machine learning and research on artificial intelligence (AI). Governments and corporations are investing billions of dollars to fund researchers who are keen to produce an ever‐expanding range of artificial intelligent systems. More than 30 countries have announced such research initiatives over the past 3 years 1. For example, the EU Commission pledged to increase the investment in AI research to €1.5 billion by 2020 (from €500 million in 2017), while China has committed $2.1 billion towards an AI technology park in Beijing alone 1. This global investment in AI is astonishing and prompts several questions: What are the true possibilities and limitations of AI? What do AI researchers and developers mean by “intelligence”? How does this compare to the everyday concept of intelligence and how the term is other branches of cognitive science? And can machine learning produce anything that is truly “intelligent”?

We present nine facets for the analysis of the past and future evolution of AI. Each facet has also a set of edges that can summarise different trends and contours in AI. With them, we first conduct a quantitative analysis using the information from two decades of AAAI/IJCAI conferences and around 50 years of documents from AI topics, an official database from the AAAI, illustrated by several plots. We then perform a qualitative analysis using the facets and edges, locating AI systems in the intelligence landscape and the discipline as a whole. This analytical framework provides a more structured and systematic way of looking at the shape and boundaries of AI.

Autonomy has been central to moral and political philosophy for millenia, and has been positioned as a critical aspect of both justice and wellbeing. Research in psychology supports this position, providing empirical evidence that autonomy is critical to motivation, personal growth and psychological wellness. Responsible AI will require an understanding of, and ability to effectively design for, human autonomy (rather than just machine autonomy) if it is to genuinely benefit humanity. Yet the effects on human autonomy of digital experiences are neither straightforward nor consistent, and are complicated by commercial interests and tensions around compulsive overuse. This multi-layered reality requires an analysis that is itself multidimensional and that takes into account human experience at various levels of resolution. We borrow from HCI and psychological research to apply a model (“METUX”) that identifies six distinct spheres of technology experience. We demonstrate the value of the model for understanding human autonomy in a technology ethics context at multiple levels by applying it to the real-world case study of an AI-enhanced video recommender system. In the process we argue for the following three claims: 1) There are autonomy-related consequences to algorithms representing the interests of third parties, and they are not impartial and rational extensions of the self, as is often perceived; 2) Designing for autonomy is an ethical imperative critical to the future design of responsible AI; and 3) Autonomy-support must be analysed from at least six spheres of experience in order to approriately capture contradictory and downstream effects.