Between Knowledge and Care: A Mixed-Methods Evaluation of Generative AI for T2DM Self-Management from Patient and Physician Perspectives

Jul 16, 2026

Upload by Eliana

Generative AI increasingly supports everyday health guidance, yet its clinical appropriateness for chronic disease remains unclear. This paper presents a two-part mixed-methods study on Type 2 Diabetes Mellitus (T2DM) assessing how patients and physicians evaluate AI-generated health information. Study 1 categorizes 784 patient queries and builds a five-dimensional physician rubric (Accuracy, Safety, Clarity, Integrity, Action Orientation). Study 2 tests seven physicians across four models and interviews identify gaps, including weaker medication reasoning and emotional support.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages35

Reading Time1 h 28 min

Introduction

Study Overview

Study 1: Patient Query Analysis

Study 2: Physician Evaluation and Interviews

Analytic Concepts and Design Implications

arXiv :2607 .03720v 1 [ cs .HC] 4 Jul 2026 Between Knowledge and Care: A Mixed-Methods Evaluation of Generative AI for T2DM Self-Management from Patient and Physician Perspectives RUIQI CHEN∗ , University of Michigan, USA YIBO MENG∗ , Weill Cornell Medicine, Cornell University, USA HUIDI LU, University of Oxford, United Kingdom XIAOLAN DING, North China University of Science and Technology Health Science Center, China Generative AI is increasingly used for everyday health guidance, yet its clinical appropriateness in chronic disease contexts remains poorly understood. This paper presents a two-part mixed-methods study on Type 2 Diabetes Mellitus (T2DM), examining how patients and physicians assess AI-generated health information. Study 1 analyzes 784 participant reported patient queries to characterize seven informational need categories and develops a structured five dimensional physician rating rubric informed by patient query categories and clinician priorities (Accuracy, Safety, Clarity, Integrity, Action Orientation) . Study 2 engages seven physicians scoring responses from four AI models and discussing evaluative reasoning through in-depth interviews. Models perform well on factual explanation and lifestyle guidance but consistently underperform on medication reasoning and emotional support. Two analytic concepts emerge from the data. The pre-visit primer frames AI as preparation for clinical encounters rather than as a replacement for physicians. The fluency illusion describes how polished language may convey epistemic authority that the clinical content does not support. Patients and physicians converged on three shared limitations (role boundaries, emotional inadequacy, personalization gaps) while diverging in evaluative emphasis, which informed four design directions, task-aware orchestration, risk-aware fallback, dynamic personalization, and emotionally attuned interaction. CCS Concepts: • Do Not Use This Code → Generate the Correct Terms for Your Paper; Generate the Correct Terms for Your Paper; Generate the Correct Terms for Your Paper; Generate the Correct Terms for Your Paper. Additional Key Words and Phrases: generative AI, type 2 diabetes, health information quality, chronic disease self management, patient clinician perspectives, human-AI interaction ACM Reference Format: Ruiqi Chen, Yibo Meng, Huidi Lu, and Xiaolan Ding. 2018. Between Knowledge and Care: A Mixed-Methods Evaluation of Generative AI for T2DM Self-Management from Patient and Physician Perspectives. In Proceedings of Make sure to enter the correct conference title from your rights confirmation email (Conference acronym ’XX). ACM, New York, NY, USA, 35 pages. [https://doi.org/XXXXXXX.XXXXXXX](https://doi.org/XXXXXXX.XXXXXXX) 1 INTRODUCTION As generative AI systems increasingly permeate consumer health contexts, their potential role in chronic disease management has garnered substantial interest from the HCI and medical informatics communities [20] . Large language ∗ Ruiqi Chen and Yibo Meng contributed equally to this research. Authors’ Contact Information: Ruiqi Chen, [ruiqich@umich.edu](ruiqich@umich.edu), University of Michigan, Ann Arbor, Michigan, USA; Yibo Meng, [yim4007@med.cornell.edu](yim4007@med.cornell.edu), Weill Cornell Medicine, Cornell University, New York, New York, USA; Huidi Lu, [huidi.lu@sbs.ox.ac.uk](huidi.lu@sbs.ox.ac.uk), University of Oxford, Oxford, England, United Kingdom; Xiaolan Ding, North China University of Science and Technology Health Science Center, Tangshan, Hebei, China. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy o

FAQs

What question does the paper address about generative AI in chronic care?

It investigates whether and how generative AI-generated health information is appropriate for chronic disease self-management, focusing on patient and physician evaluations of contextual safety, actionability, and emotional appropriateness.

How are patient needs and physician evaluation criteria derived in the study?

Study 1 analyzes 784 patient queries to form seven informational need categories and then develops a structured five-dimensional physician rating rubric. The rubric reflects clinician priorities mapped to the patient query categories.

Which aspects do the evaluated AI models perform well on, and where do they underperform?

The models do well on factual explanation and lifestyle guidance but underperform on medication reasoning and emotional support, as reflected by physician scoring and interview reasoning.

Jul 16, 2026

Upload by Eliana