Explainable Machine Learning, Patient Autonomy, and Clinical Reasoning

Aug 3, 2026

Upload by Himbo

Clinical decision support systems based on complex machine learning models often provide opaque rationales and value commitments behind diagnoses and recommended treatments. This clashes with medical ethics, which requires patient autonomy through informed consent grounded in understanding relevant evidence alongside patients’ beliefs and values. Algorithmic explainability is promoted as a partial remedy, but success depends on answering what must be explained, to whom, how, and when, within clinical workflows. The chapter argues these depend on deeper issues in the philosophy of clinical reasoning, defended via a Peircean account.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages33

Reading Time1 h 23 min

AccessFree to download

Introduction

Informed consent and shared decision-making

Opacity of machine learning in clinical decisions

Explainability challenge framing

What needs to be explained, to whom, and when

Clinical reasoning and a Peircean account

Abduction and strategic reasoning

Explainable Machine Learning, Patient Autonomy, and Clinical Reasoning Geoff Keeling and Rune Nyrup Abstract: Clinical decision support systems based on complex machine learning models render opaque the rationale and value commitments that underpin diagnoses and suggested treatments. This creates a tension with the prevailing view in medical ethics, which emphasises patients making autonomous decisions based on an understanding of relevant medical evidence alongside their beliefs and values. Calls for algorithmic explainability in clinical settings are partly motivated by this tension. The question is what needs to be explained, to whom, in what way, and when, to integrate machine learning systems into the clinical process in a way that is consistent with patient-centred decision-making. In this chapter, we review the tension and argue that answers to these questions depend on more fundamental issues in the philosophy ofmedicine regarding the logic of clinical reasoning. We outline and defend a broadly Peircean account which, we argue, captures ethically salient aspects of the interplay between clinicians and decision support systems, and use it to shed light on the particulars of the explainability challenge. Key words: Explainability; Algorithmic Opacity; Informed Consent; Shared Decision Making; Patient Autonomy; Clinical Reasoning; Abduction; C.S. Peirce. Acknowledgements: We are extremely grateful to Farbod Akhlaghi, Christopher Burr, Stephen Cave, Daniel P. Jones, William Peden, Sabin Roman, Carissa Véliz, and an anonymous referee for their comments on the manuscript and many helpful discussions. The paper was presented at the Philosophy of Medical AI workshop in Tübingen and the Centre for the Future of Intelligence Weekly Seminar, and we are grateful to the audiences for helpful feedback. This work was funded by the Wellcome Trust [213660/Z/18/Z] and the Leverhulme Trust through the Leverhulme Centre for the Future of Intelligence. 1. Introduction Machine learning is transforming medicine. The last decade has seen the application of data-driven technologies to the detection and diagnosis of medical conditions, and to the design of personalised treatment plans (He et al. 2019; Yu et al. 2018) . These technologies promise better health outcomes. Existing diagnostic algorithms rival expert clinicians in the accuracy, sensitivity, and specificity of their diagnostic predictions (Shen et al. 2019; Gulshan et al. 2016) . The hope is that, in the future, these algorithms will contribute to a reduction in morbidity and mortality arising from delayed, incorrect, or missed diagnoses. Similarly, treatment recommender systems promise personalised evidence-based treatment plans that are intended to reduce preventable harms arising from suboptimal treatments (Somashekhar et al. 2017) . There are, however, concerns about the transparency and interpretability of machine learning algorithms in medicine (Bjerring and Busch 2020; Grote and Berens 2020; McDougall 2019) . What is at issue is a tension between two factors. On the one hand, the prevailing view in medical ethics emphasises patients’ understanding of the rationale behind diagnoses as a precondition for giving informed consent to treatments (Faden and Beauchamp 1986) . It also emphasises clinicians and patients making joint decisions based on a shared understanding of the relevant medical evidence alongside the patient’s beliefs and values. On the other hand, the complexity of contemporary machine learning systems renders opaque the rationale and value commitments that underpin token algorithmic decisions such as diagnostic predictions and suggested treatment plans. The problem is that these algorithms can make it difficult to explain on what grounds a diagnosis is reached or why a treatment is recommended. Hence it is unclear how, ifat all, opaque machine learning algorithms can be integrated into the clinical process in a way that upholds our best practices of i

FAQs

Why does machine learning opacity create ethical tension in clinical decision-making?

Opaque models can obscure the rationale and value commitments behind diagnoses and treatment recommendations, making it difficult to align with informed consent and shared decision-making requirements.

What determines whether explainability can be integrated into clinical processes?

Integration depends on clarifying what must be explained, to whom, in what way, and at what points in the clinical workflow in a way consistent with patient-centred decision-making.

How does the chapter connect explainability to the philosophy of clinical reasoning?

It argues that disputes about explainability cannot be separated from more fundamental questions about the logic of clinical reasoning, including how clinicians and decision support systems jointly reach diagnoses and recommendations.

Aug 3, 2026

Upload by Himbo