What Are Humans Doing in the Loop? - Co-Reasoning and Practical Judgment When Using Machine Learning-Driven Decision Aids

Aug 6, 2026

Within the ethical debate on machine learning-driven decision support systems (ML-DSS), the ideas of “human in the loop” and “meaningful human control” are frequently treated as necessary for ethical legitimacy. Ethical principles are often used as the main reference point, yet conflicts between principles are usually weighed against each other. From a neo-Kantian perspective inspired by Onora O’Neill, the article proposes an interpretation of the “human in the loop” role in health-care AI evaluation. Patients should be understood as “fellow workers” and epistemic partners, enabling meaningful integration of ethical principles rather than balancing.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

AccessFree to download

Abstract

Introduction

Human oversight requirements and uncertainty about roles

Ethical evaluation and the need for relevant ethical considerations

The American Journal of Bioethics ISSN: (Print) (Online) Journal [homepage: ](homepage: www.tandfonline.com/journals/uajb20)[www.tandfonline.com/journals/uajb20](homepage: www.tandfonline.com/journals/uajb20) What Are Humans Doing in the Loop? CoReasoning and Practical Judgment When Using Machine Learning-Driven Decision Aids Sabine Salloch & Andreas Eriksen To cite this article: Sabine Salloch & Andreas Eriksen (20 May 2024): What Are Humans Doing in the Loop? Co-Reasoning and Practical Judgment When Using Machine Learning-Driven Decision Aids, The American Journal of Bioethics, DOI: 10. 1080/15265161 .2024.2353800 To link to this article: [https://doi.org/10.1080/15265161.2024.2353800](https://doi.org/10.1080/15265161.2024.2353800) © 2024 The Author(s) . Published with license by Taylor & Francis Group, LLC. | Published online: 20 May 2024. | | --- | | Submit your article to this journal | | Article views: 329 | | View related articles | | View Crossmark data | Full Terms & Conditions of access and use can be found at [https://www.tandfonline.com/action/journalInformation?journalCode=uajb20](https://www.tandfonline.com/action/journalInformation?journalCode=uajb20) The AmericAn JournAl of BioeThics [https://doi.org/10.1080/15265161.2024.2353800](https://doi.org/10.1080/15265161.2024.2353800) TargeT arTicle What Are Humans Doing in the Loop? Co-Reasoning and Practical Judgment When Using Machine Learning-Driven Decision Aids Sabine Sallocha* and andreas eriksenb* ahannover medical school; boslo metropolitan university ABSTRACT Within the ethical debate on Machine learning-driven decision support systems (Ml_cDSS), notions such as “human in the loop” or “meaningful human control” are often cited as being necessary for ethical [legitimacy. in](legitimacy. in) addition, ethical principles usually serve as the major point of reference in ethical guidance documents, stating that conflicts between principles need tobe weighed and balanced against each other. Starting from a neo-Kantian viewpoint inspired by Onora O'Neill, this article makes a concrete suggestion of how to interpret the role of the“human in the loop” and to overcome the perspective of rivaling ethical principles in the evaluation of ai in health care. We argue that patients should be perceived as “fellow workers”and epistemic partners in the interpretation of Ml_cDSS outputs. We further highlight that a meaningful process of integrating (rather than weighing and balancing) ethical principles is most appropriate in the evaluation of medical ai. KEYWORDS clinical Decision Support Systems; Machine learning; ethical principles; human in the loop INTRODUCTION Computerized Clinical Decision Support Systems (CDSS) have gained increased importance in healthcare since their first use in the 1980s. According to a widely shared definition, CDSS provide “clinicians, staff, patients, or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care” (Osheroff et al. 2007) . Progress in data science and in the availability of clinical data has recently fueled a rapid development and the introduction of Machine Learning-driven CDSS (ML_CDSS) in various clinical workflows (London 2018) . Radiology (Hwang et al. 2019), pathology (Kiani et al. 2020; Bulten et al. 2022) and ophthalmology (Gulshan et al. 2016; Ting et al. 2017) are among the pioneering disciplines in the introduction of ML_CDSS because they deal with large quantities of visual data. But also surgical disciplines (Standiford et al. 2022), dermatology (Esteva et al. 2017), pediatrics (Liang et al. 2019) and other clinical fields (Neugebauer et al. 2020; Jia et al. 2020) potentially profit from ML_CDSS. In response to the rapid development of AI-based tools, there has been an increased concern with regulatory frameworks and responsibility. The legal slogan is that there must alwa

FAQs

Why does the article focus on the “human in the loop” concept in ML-driven decision support?

Because ethical debates about ML-DSS often assume that “human in the loop” is required for legitimacy, yet it remains unclear what humans are actually supposed to do. The article aims to interpret this role in a concrete, evaluative way.

How does the article propose patients should relate to ML-DSS systems?

It argues that patients should be perceived as “fellow workers” and epistemic partners in interpreting ML-DSS outputs. This reframes oversight and involvement as collaborative epistemic participation.

What does the article suggest about handling ethical principles during evaluation?

Rather than weighing and balancing conflicting principles against each other, it highlights a meaningful process of integrating ethical principles as most appropriate for evaluating medical AI.

Aug 6, 2026

Upload by Emma Wilson