Hannah van Kolfschooten

Governments across Europe are increasingly deploying artificial intelligence (AI) tools to fulfil their obligations under the right to health and advance the core functions of public health: prevention, promotion, and protection. Yet despite their promise, these technologies risk reinforcing health inequities, especially for those marginalized by intersecting forms of disadvantage such as race, gender, disability, migration status, and class. This chapter applies an intersectional lens to examine how AI-driven public health interventions affect states’ legal duties under international and European human rights law. It analyzes how AI is transforming public health governance and assesses whether current EU regulatory frameworks, particularly the Artificial Intelligence Act, can safeguard the right to health. Focusing on use cases such as population cancer screening, health chatbots, and epidemic surveillance, the chapter shows how algorithmic systems often replicate structural exclusions and undermine the principles of availability, accessibility, acceptability, and quality (AAAQ). It concludes by proposing legal and policy tools to align AI governance with intersectional health justice.

AI chatbots such as ChatGPT, Gemini, and Claude are rapidly becoming part of adolescents’ everyday lives. Yet despite their growing role in providing companionship, advice, and even mental health support, these systems remain largely absent from public health discussions on digital regulation. We argue that AI chatbots should be understood as emerging public health technologies. By "public health technologies," we refer to technologies that, due to their scale, design, and capacity to influence health-related behaviours, wellbeing, and access to care, warrant governance based on public health objectives, rather than only consumer protection or innovation concerns. AI chatbots increasingly meet these criteria in terms of scale, design, and influence on behaviour.

Artificial intelligence (AI) is increasingly embedded in clinical decision-making, yet recent reforms to the European Union (EU) Artificial Intelligence Act (AI Act) risk creating a category of high-risk medical AI systems without corresponding safeguards. In the current framework, AI-enabled medical devices are regulated through a combination of sectoral medical device legislation, the Medical Devices Regulation (MDR) and In Vitro Diagnostic Medical Devices Regulation (IVDR), and additional AI-specific safeguards in the AI Act. Recent EU “simplification” initiatives, including the Digital Omnibus and the Health Innovation Package, propose to remove these safeguards. This narrative review analyses recent EU legislative developments and their impact on the relationship between the AI Act and medical device law. It shows that the proposed reforms would retain the high-risk classification of AI-enabled medical devices while removing most associated AI Act obligations. In effect, this decouples risk classification from the safeguards that give it regulatory meaning, including requirements on data governance, risk management, human oversight, and post-market monitoring. Rather than resolving regulatory overlap, the reforms shift the centre of gravity back to product-focused medical device law without ensuring equivalent AI-specific safeguards. These changes may narrow attention to fundamental rights, weaken oversight in clinical use, and increase legal uncertainty regarding accountability and responsibility. In a domain where AI systems directly shape clinical decisions and patient outcomes, these changes risk undermining the conditions for safe, equitable, and accountable deployment of medical AI.

Generative Artificial Intelligence (genAI) chatbots have become an important outlet for many people around the world who are experiencing mental health issues. Of its 800 million weekly users, around 10 percent use ChatGPT for emotional support while more than one million use the chatbot to talk about issues such as depression, psychosis and suicidal ideation. For many users, these conversations feel intimate, tailored and supportive, like talking to a good friend or therapist. Yet these interactions take place within commercial digital platforms whose business models rely on collecting, analysing and commodifying user data. As Big Tech companies explore the use of behavioural advertising in genAI systems, influenced by the global market for health data being worth an estimated $74.41 bn, profound ethical tensions arise between the commodification of user data and the protection of vulnerable individuals seeking mental health support. On one hand, the World Health Organization states that mental health is a fundamental human right, which means that everyone has the right to available, accessible, acceptable, and good-quality mental healthcare. As more than 1 billion people live with mental health issues and many do not receive adequate care, more and more turn to genAI chatbots, which are often free, accessible and highly interactive. As AI chatbots are becoming quasi-therapeutic spaces, they exert an influence over the right to mental health. However, unlike traditional mental healthcare contexts where confidentiality and professional duties restrict the use of patient information, AI chatbots operate within a commercial ecosystem that is designed to maximise engagement and monetise user attention. This means that users pay a high yet often invisible price for their mental health. The commodification of mental health data works as follows: tech companies collect and analyse any information that appear in prompts, chat logs and other user interactions. While some chatbots, such as Grok, claim that “we do not sell your personal information or use it for marketing”, their privacy policies clearly show that aggregated and de-identified data is being ‘shared’ with affiliates and third parties, including advertisers and other business partners (Table 1). Grok started to run ads in August 2025 and ChatGPT has been trialling ads since February 2026.

This paper examines how Article 5 of the EU Artificial Intelligence Act prohibits certain AI uses in healthcare, including emotion recognition and biometric categorisation, highlighting their legal implications for fundamental rights, regulatory compliance, and the protection of vulnerable populations. The EU Artificial Intelligence Act (AI Act), adopted in 2024, is the world’s first comprehensive legal framework for artificial intelligence (AI). Its impact extends beyond the EU, since all providers, importers, distributors, and deployers placing AI systems on the EU market or using them in the EU must comply with role-specific obligations. What is often overlooked in scholarship and policy debates is that the AI Act does not only impose new requirements – it also prohibits certain uses of AI. Article 5 of the AI Act identifies AI practices incompatible with fundamental rights. Article 5 stands apart from the rest of the AI Act. While most provisions set conditions for “high-risk” AI, this provision classifies a small number of uses as “unacceptable” and therefore prohibited. However, several of these bans contain narrowly defined exceptions, mainly for medical or safety purposes. Understanding how these categorical prohibitions and their exceptions interact is crucial for sectors like healthcare, where both risk and necessity are high.

Members of the European Parliament are about to take a decision that will fundamentally shape how artificial intelligence (AI) is governed in European healthcare. The Internal Market and Consumer Protection (IMCO) and Civil Liberties, Justice and Home Affairs (LIBE) compromise amendments propose deleting Annex I, Section A of the AI Act and shift medical devices into Annex I, Section B. If adopted, AI medical devices would remain labelled as “high-risk”, but would no longer be subject to meaningful high-risk obligations. In a domain where AI directly affects diagnosis, treatment, and patient health, this creates a serious regulatory gap with real risks for patients. The proposed solution – “mirroring” AI Act requirements into the Medical Devices Regulation (MDR) and In Vitro Diagnostic Medical Devices Regulation (IVDR) – does not address this gap, as it makes safeguards dependent on future delegated acts and standards, with no guarantee as to their timing or scope. This risks delaying and weakening protections while medical AI systems are already in use. This is not “simplication”, but a structural weakening of safeguards for medical AI.

ChatGPT Health and other health-focused generative AI chatbots increasingly function as alternative first points of contact that may mediate - and in some cases substitute - engagement with regulated healthcare systems. At scale, these systems can shape care-seeking behavior, system capacity, trust in clinical expertise, and health equity. We describe this development as the emergence of shadow health systems: privately governed infrastructures that perform health-system functions without being subject to healthcare-specific safeguards. EU law currently regulates such tools based on declared medical purpose rather than their real-world effects, creating gaps in legal protection. We propose the adoption of stronger consumer and health data safeguards alongside extension of the scope of the Medical Devices Regulation and AI Act.

Artificial intelligence (AI) is now firmly embedded in radiology practice. From automated abnormality detection on chest radiographs to workflow optimization in triage, AI is increasingly shaping diagnostic processes. Its promise is substantial: improved efficiency, faster reporting, and better diagnostic accuracy. Yet these benefits come with risks that extend beyond technical performance. For radiologists, the critical challenge is to ensure that integration of AI into clinical practice does not compromise patients’ fundamental rights. This is not only a professional duty, but also a legal obligation. Under the recently adopted EU Artificial Intelligence Act (AI Act), hospitals and clinicians deploying “high-risk” AI systems, including most radiology applications, are required to demonstrate adequate AI literacy. Radiologists must be equipped not only to understand technical aspects but also to assess risks for patients.

Against a backdrop of rapidly expanding health artificial intelligence (AI) development, this paper examines how the European Union’s (EU) stringent digital regulations may incentivise the outsourcing of personal health data collection to low- and middle-income countries (LMICs), fuelling a new form of AI ethics dumping. Drawing on parallels with the historical offshoring of clinical trials, we argue that current EU instruments, such as the General Data Protection Regulation (GDPR), Artificial Intelligence Act (AI Act) and Medical Devices Regulation, impose robust internal safeguards but do not prevent the use of health data collected unethically beyond EU borders. This regulatory gap enables data colonialism, whereby commercial actors exploit weaker legal environments abroad without equitable benefit-sharing. Building on earlier EU responses to ethics dumping in clinical trials, we propose legal and policy pathways to prevent similar harms in the context of AI.

The European Court of Human Rights (ECtHR) plays an increasingly influential role in shaping health-related human rights in Europe, despite the absence of an explicit right to health in the European Convention on Human Rights (ECHR). This article examines how the Court adjudicates morally and legally contested health issues across six domains: physician-assisted dying, abortion, vaccination, place of birth, posthumous reproduction, and climate-related health risks. Drawing on comparative doctrinal analysis, the article identifies recurring patterns in health-related cases. These include an emphasis on procedural fairness, a tendency to frame harm in individualised terms, and deference to national scientific authorities. By tracing these patterns, the article clarifies how the Court navigates competing rights and interests in contested health cases. It also considers emerging challenges, such as litigation over artificial intelligence in healthcare, highlighting the ECtHR’s growing influence on the development of health law and policy through its interpretation of the ECHR.

This study explores the opportunities, risks, and ethical considerations surrounding the use of artificial intelligence (AI) systems in mental healthcare and provides recommendations for their responsible implementation and regulation. Healthcare forms one of the most popular sectors for AI deployment in the EU. In mental healthcare, AI systems are used in diverse ways, from administrative tasks to communication on platforms, professional decision support, digital therapies like chatbots and personal sensing technologies, and patient monitoring. Beyond healthcare settings, AI applications that use mental health-related data also extend to criminal justice, consumer products, education, and employment. AI systems offer significant potential benefits, including improved accessibility to mental health support, particularly for underserved populations, and reducing administrative burdens in healthcare systems. Clinically, proponents advance AI for personalising treatments, improving diagnostic accuracy, and supporting interventions. Additionally, AI can accelerate mental health research and facilitate the development of novel therapeutic approaches, such as virtual reality therapies. However, AI systems in mental health also pose serious risks, which can be identified at three levels: individual, collective, and societal. At the individual level, concerns include safety risks, privacy violations and inadequate informed consent. Broader challenges include strengthening inequalities or creating new ones, oversurveillance, reinforcing individualistic views of mental health, depersonalization of care, and limited resources. The report highlights key principles and possible courses of action for policymakers to address these risks, by balancing innovation with ethical responsibility and human-centered care in AI-driven mental health systems. It is argued that AI tools need to be developed with ethics, inclusivity, accuracy, safety and the genuine needs of end users in mind. Possible solutions mainly include robust regulation and oversight, transparency and explainability, as well as human rights-centric and co-creation approaches. The active participation and leadership of the most affected communities - people with lived experience – should be regarded as a fundamental ethical and political principle guiding all discussions and decisions on digitalization and mental health. Ensuring that they have an equal voice in shaping policies and innovation is essential to achieving fair and balanced outcomes and preventing the deepening of health inequalities in society.

With digital technologies, your patients have a ‘doctor in their pocket’. But something new is happening when they search online for medical advice. Typing a question such as “Can I take ibuprofen with blood pressure tablets?” or “What helps against chest pain?” into Google no longer produces the familiar list of links. Instead, a confident, AI-generated box appears at the top of the page, offering what looks like an authoritative answer. Google calls this feature an AI Overview. Microsoft’s Copilot provides similar AI-generated summaries through its Edge browser. These systems mark a shift in how people find and interpret health information online. By design, these summaries reduce click-through rates to real websites by 40–60%, replacing the process of browsing diverse sources with a single, seemingly definitive response. First launched in the United States in 2024, where it drew criticism for misleading health advice, AI Overviews are now expanding across Europe. AI Overviews are not just another way of “Googling symptoms.” Until recently, users were presented with a variety of sources: public health agencies, hospital websites, patient forums, news outlets, and wellness blogs. Although the quality of these sources varied, the diversity itself enabled patients to cross-check information, prepare questions, and participate more actively in their care. Searching online can empower patients when they have access to multiple perspectives, allowing them to assess information critically. With AI Overviews, that step disappears.

Machine learning-based AI chatbots are increasingly used to promote health and encourage individuals to adopt healthier behaviors. Chatbots driven by generative AI (genAI) simulate human interactions through text or voice to generate personalized content with guidance on topics such as smoking cessation, nutrition, managing stress, and sleep improvement. The use of AI chatbots for health promotion and wellness has been growing since 2023. While empirical evidence suggests their effectiveness in supporting behavioral change and mental health, the legal, ethical, and societal implications remain largely unexplored. This article presents a qualitative case study of S.A.R.A.H., a genAI chatbot developed by the World Health Organization (WHO), analyzed against the six ethical principles outlined in the WHO’s 2021 Guidance on Ethics and Governance of AI for Health. We also gathered exploratory insights from adolescent focus groups. These findings are descriptive and not based on formal thematic analysis. Drawing on this analysis, we identify key gaps between high-level ethical principles and practice and offer policy recommendations to guide responsible use of AI chatbots for health promotion.

Aan de hand van twee casus wordt in kaart gebracht hoe AI in het diagnostisch proces door hulpverleners kan worden gebruikt, en wat de mogelijke gevolgen zijn van onjuist gebruik van AI of gebruik van een ongeschikt AI-systeem. Daarmee wordt verduidelijkt hoe de patiënt eventuele schade door een onjuiste diagnose gesteld met behulp van AI kan verhalen en waar hij bij dat verhaal tegenaan kan lopen. Geconcludeerd wordt dat verhaal van AI-gerelateerde schade niet fundamenteel verschilt van verhaal van schade veroorzaakt door ‘gewone’ technologie. Er is, net als bij andere technische apparaten, sprake van een zaak die de hulpverlener gebruikt ter uitvoering van zijn verbintenis zoals het stellen van een diagnose. De hulpverlener heeft ook, net als bij andere technische apparaten, verplichtingen in het kader van de geneeskundige behandeling maar ook bijkomende specifieke verplichtingen ten aanzien van het toepassen van AI. Wel worden mogelijke bewijsproblemen voor de patiënt gesignaleerd in het aantonen van de ongeschiktheid van het AI-systeem, wat een drempel kan opleveren voor het verhaal van schade.

Picture a typical teenager waking up. Before even getting out of bed, they’ve already scrolled through TikTok, checked Instagram, and responded to Snapchat notifications. Each swipe delivers content fine-tuned by algorithms designed to maximize attention and engagement. Autoplay keeps the feed going. Notifications prompt more interaction. Autoplay keeps content flowing. Sleep, focus, and mental health are subtly undermined. The mental health consequences of these design features – autoplay, infinite scroll, push notifications – are becoming hard to ignore. In December 2023, the European Parliament issued a resolution calling out these harms and proposing something bold: a digital “right not to be disturbed.” Soon after, the European Commission launched a review of EU consumer law: the Digital Fairness Fitness Check. The conclusion? Existing rules are outdated and unfit for today’s attention-hacking digital environments. In May 2025, the Commission announced plans for a new Digital Fairness Act aimed at updating consumer protections for the algorithmic age. A formal legislative proposal is expected in 2026. The timing couldn’t be more urgent.

Artificial intelligence (AI) translation tools, such as Google Translate and ChatGPT, are increasingly used in healthcare for medical communication to overcome language barriers between patients and providers. While these tools offer accessible and efficient translation, their use raises significant legal, ethical, and policy concerns. Key patients’ rights, including the rights to privacy, informed consent, and equitable access to care, may be compromised. Current European regulations, including the EU AI Act, General Data Protection Regulation, and Medical Devices Regulation, offer only partial protection, leaving important regulatory gaps. This study employs a mixed-methods approach combining legal doctrinal analysis of EU regulatory frameworks with manual content analysis of platform terms of service. It integrates interdisciplinary perspectives from bioethics, digital health, and health communication to evaluate the implications of AI-mediated translation in clinical care. Findings reveal persistent and overlapping risks: violations of data privacy, inaccuracies in translation, bias and discrimination, and unclear liability when errors occur. To mitigate these risks, we propose targeted policy interventions, including developing guidelines for AI translation use in healthcare settings. This article contributes to digital health policy debates by identifying legal pathways to regulate AI translation tools in healthcare, ensuring their use supports patients’ rights and promotes health equity.

Artificial Intelligence (AI) presents unprecedented opportunities to transform healthcare worldwide, from improving diagnostic accuracy to expanding access in underserved regions. Despite this potential and growing investment, a significant gap persists between AI’s theoretical promise and its realised benefits in healthcare settings. This article examines the complex barriers impeding AI benefits realization in global health contexts, including ethical uncertainties, data infrastructure limitations, evidence quality concerns, and regulatory ambiguities. We analyze current initiatives addressing these challenges and highlight how technological solutions alone cannot resolve fundamental healthcare inequities. Drawing on the interdisciplinary perspectives and insights presented at the Global Health in the Age of AI Symposium hosted by the Cini Foundation and Yale Digital Ethics Center, we propose five core infrastructure requirements necessary for ethical AI implementation: robust data exchange; epistemic certainty with staff autonomy; actively protected healthcare values; validated outcomes with meaningful accountability; and environmental sustainability. These requirements form the foundation for a systems approach that balances technological advancement with ethical imperatives, contextual adaptability, and global equity considerations. We conclude that the successful integration of AI into healthcare demands coordinated action across sectors and borders, with careful attention to avoiding technological colonialism and ensuring AI serves as a force for health equity rather than widening existing disparities.

The rapid advancement of digital health innovation, including Artificial Intelligence (AI), is transforming healthcare. The growing role the European Union (EU) plays in regulating the use of AI in healthcare renders national laws insufficient to safeguard patients from unique AIrelated risks. This underscores the urgent need for the recognition of a canon of patients' rights in the scope of EU law. This paper proposes the blueprint for an EU Charter for Digital Patients' Rights, consolidating and adapting existing rights for patients to address these specific challenges. Traditional patients' rights such as the right to privacy and the right to informed consent are interpreted in light of the challenges posed by AI. It also proposes novel rights for patients, such as the right not to be subject to automated medical decision-making and the right to meaningful human contact. This paper highlights the EU's central role in protecting patients' rights and outlines strategies for effective implementation. This comprehensive approach aims to address the current shortcomings and enhance the legal framework governing digital health in Europe.

The Netherlands was the first country to legalize euthanasia and physician-assisted suicide. This paper offers a first legal perspective on the prospects of using AI in the Dutch practice of euthanasia and physician-assisted suicide. It responds to the Regional Euthanasia Review Committees’ interest in exploring technological solutions to improve current procedures. The specific characteristics of AI – the capability to process enormous amounts of data in a short amount of time and generate new insights in individual cases – may for example alleviate the increased workload of review committees due to the continuous increase of euthanasia cases. The paper considers three broad categories for the use of AI in the Dutch euthanasia practice: (1) the physician’s assessment of euthanasia requests, (2) the actual execution of euthanasia, and (3) the retrospective reviews of cases by the Regional Euthanasia Review Committees. Exploring the legal considerations around each avenue, both in the EU AI Act and the Dutch legal framework, this paper aims to facilitate the societal discussion on the role of technology in such deeply human decisions. This debate is equally relevant to other countries that legalized euthanasia (e.g. Belgium and Canada) or physician-assisted suicide (e.g. Switzerland and numerous states in the US).