Stacy S. Chen

Objective: To examine the role of explainability in machine learning for healthcare (MLHC), and its necessity and significance with respect to effective and ethical MLHC application. Study Design and Setting: This commentary engages with the growing and dynamic corpus of literature on the use of MLHC and artificial intelligence (AI) in medicine, which provide the context for a focused narrative review of arguments presented in favour of and opposition to explainability in MLHC. Results: We find that concerns regarding explainability are not limited to MLHC, but rather extend to numerous well-validated treatment interventions as well as to human clinical judgment itself. We examine the role of evidence-based medicine in evaluating unexplainable treatments and technologies, and highlight the analogy between the concept of explainability in MLHC and the related concept of mechanistic reasoning in evidence-based medicine. Conclusion: Ultimately, we conclude that the value of explainability in MLHC is not intrinsic, but is instead instrumental to achieving greater imperatives such as performance and trust. We caution against the uncompromising pursuit of explainability, and advocate instead for the development of robust empirical methods to successfully evaluate increasingly inexplicable algorithmic systems.

In their forthcoming article, ‘What makes a medical intervention invasive?’ De Marco, Simons, and colleagues explore the meaning and usage of the term ‘invasive’ in medical contexts. They describe a ‘Standard Account’, drawn from dictionary definitions, which defines invasiveness as ‘incision of the skin or insertion of an object into the body’. They then highlight cases wherein invasiveness is employed in a manner that is inconsistent with this account (eg, in describing psychotherapy) to argue that the term invasiveness is often used to explicitly or implicitly reference interventions that do not fall under the standard account. The authors offer an alternative ‘schematic’ definition of invasiveness, whereby medical interventions entail an underlying context-independent property of ‘basic invasiveness’ and may be understood both relative to each other and to a context-specific threshold of what is ‘invasive’. 1 We propose that the meaning of terms such as ‘invasiveness’ is necessarily heterogeneous,...

The aim of a theoretically ideal process of informed consent is to promote the autonomy of the patient and to limit unethical physician paternalism. However, in practice, the nature of the medical profession requires physicians to act as ontological decision architects—based on the medical knowledge that they acquire through their experience and training, physicians ontologically determine a subset of viable courses of action for their patient. What is observed is not an unethical physician limitation or biasing of the patient towards certain treatment options that violates patient autonomy or consciously undermines informed consent, but rather a more foundational paternalism that is necessarily inherent to the physician–patient relationship. In this article we argue for a recognition of this underlying physician paternalism and posit that this necessary paternalism is not a foil to patient autonomy, but rather a foundational aspect of the duties of the medical professional within the physician–patient relationship.

The aim of a theoretically ideal process of informed consent is to promote the autonomy of the patient and to limit unethical physician paternalism. However, in practice, the nature of the medical profession requires physicians to act as ontological decision architects—based on the medical knowledge that they acquire through their experience and training, physicians ontologically determine a subset of viable courses of action for their patient. What is observed is not an unethical physician limitation or biasing of the patient towards certain treatment options that violates patient autonomy or consciously undermines informed consent, but rather a more foundational paternalism that is necessarily inherent to the physician–patient relationship. In this article we argue for a recognition of this underlying physician paternalism and posit that this necessary paternalism is not a foil to patient autonomy, but rather a foundational aspect of the duties of the medical professional within the physician–patient relationship.

In many jurisdictions, legal frameworks afford patients the opportunity to make prospective medical decisions or to create directives that contain a special provision forfeiting their own ability to object to those decisions at a future time point, should they lose decision‐making capacity. These agreements have been described with widely varying nomenclatures, including Ulysses Contracts, Odysseus Transfers, Psychiatric Advance Directives with Ulysses Clauses, and Powers of Attorney with Special Provisions. As a consequence of this terminological heterogeneity, it is challenging for healthcare providers to understand the terms and uses of these agreements and for ethicists to engage with the nuances of clinical decision‐making with such unique provisions surrounding patient autonomy. In theory, prospective self‐binding agreements may safeguard patient's “authentic” wishes from future “inauthentic” changes of mind. In practice, it is unclear what may be comprised within these agreements or how—and to what effect—they are used. The primary focus of this integrative review is to curate the existing literature describing Ulysses Contracts (and analogous decisions) used in the clinical arena, in order to empirically synthesize their shared essence and provide insights into the traditional components of these agreements when used in practice, the requirements of their consent processes, and the outcomes of their utilization.