Conceptualizing Machine Learning for Dynamic Information Retrieval of Electronic Health Record Notes - A proof of concept for dynamic note relevance prediction

Aug 3, 2026

Upload by Anda

Clinicians spend substantial time reading and sifting through electronic health record (EHR) notes while documenting, contributing to burnout and workflow inefficiency. This work proposes a machine-learning supervision signal derived from EHR audit logs to predict note relevance within a specific clinical context and time point. Evaluation targets dynamic retrieval in the emergency department, showing AUC 0.963 for predicting which notes will be read in individual note-writing sessions, supported by a clinician user study demonstrating more efficient retrieval.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages29

Reading Time1 h 13 min

AccessFree to download

Introduction

Problem: EHR note search and clinician burnout

Approach: dynamic retrieval from audit-log supervision

Evaluation: emergency department dynamic retrieval

Generalizable insights for ML in healthcare

arXiv :2308 .08494v 1 [ cs .IR] 9 Aug 2023 Conceptualizing Machine Learning for Dynamic Information Retrieval of Electronic Health Record Notes Sharon Jiang 1 Shannon Shen 1 Monica Agrawal 1 Barbara Lam 2 ,3 Nicholas Kurtzman 4 Steven Horng 3 ,4 David Karger 1 David Sontag 1 [jiangs@mit.edu](jiangs@mit.edu) [zjshen@mit.edu](zjshen@mit.edu)[ ](zjshen@mit.edu)[monica.n.agrawal@gmail.com](monica.n.agrawal@gmail.com)[blam@bidmc.harvard.edu](blam@bidmc.harvard.edu)[nkurtzma@bidmc.harvard.edu](nkurtzma@bidmc.harvard.edu)[ ](nkurtzma@bidmc.harvard.edu)[shorng@bidmc.harvard.edu](shorng@bidmc.harvard.edu)[ ](shorng@bidmc.harvard.edu)[karger@mit.edu](karger@mit.edu) [dsontag@csail.mit.edu](dsontag@csail.mit.edu) 1 Department of Electrical Engineering & Computer Science, MIT, Cambridge, MA, USA 2 Division of Hematology & Oncology, Department of Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA 3 Division of Clinical Informatics, Department of Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA 4 Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA Abstract The large amount of time clinicians spend sifting through patient notes and documenting in electronic health records (EHRs) is a leading cause of clinician burnout. By proactively and dynamically retrieving relevant notes during the documentation process, we can reduce the effort required to find relevant patient history. In this work, we conceptualize the use of EHR audit logs for machine learning as a source of supervision of note relevance in a specific clinical context, at a particular point in time. Our evaluation focuses on the dynamic retrieval in the emergency department, a high acuity setting with unique patterns of information retrieval and note writing. We show that our methods can achieve an AUC of 0.963 for predicting which notes will be read in an individual note writing session. We additionally conduct a user study with several clinicians and find that our framework can help clinicians retrieve relevant information more efficiently. Demonstrating that our framework and methods can perform well in this demanding setting is a promising proof of concept that they will translate to other clinical settings and data modalities (e.g. , labs, medications, imaging) . 1. Introduction Electronic health records (EHRs) serve as a central repository of a patient’s past medical history, containing both structured data and free text notes (Burton et al. , 2004; Li et al. , 2022) . These data are crucial across multiple stages of the medical decision making process (Muhiyaddin et al. , 2022) . Over the course of a patient encounter, clinicians describe time-varying purposes of information retrieval from the EHR: rapid sense-making of a new patient, re-familiarizing oneself with an existing patient, searching for a particular factoid, © 2023 S. Jiang, S. Shen, M. Agrawal, B. Lam, N. Kurtzman, S. Horng, D. Karger & D. Sontag. Conceptualizing ML for Dynamic Information Retrieval of EHR Notes and looking for unspecified evidence to support the differential diagnosis process (Nygren and Henriksson, 1992) . However, retrieval of this relevant information is a time-consuming process, given the volume of data in EHRs. This data gathering process is complicated by the fact that much of the information required for medical decision making is found only in free text notes (Li et al. , 2008; Shivade et al. , 2014) . These notes have become bloated and unwieldy to manually sift through due to their multitude of aims (clinical communication, compliance, and billing) . Consequently, clinicians are spending more time navigating and documenting in EHRs than in face-to-face encounters with patients, a phenomenon considered a leading cause of clinician burnout (Menachemi and Collum, 2011; Moy et al. , 2021) . Given the inefficiencies in clinical workflows, there is significant interest in better characterizin

FAQs

How does the method use EHR data to supervise note relevance?

It conceptualizes EHR audit logs as a supervision source for predicting whether notes will be relevant in a given clinical context at a particular time point.

Where is the approach evaluated and what performance is reported?

The evaluation focuses on dynamic information retrieval in the emergency department, achieving an AUC of 0.963 for predicting which notes will be read during individual note-writing sessions.

Does the work include clinician feedback, and what does it show?

A user study with several clinicians indicates the framework helps retrieve relevant information more efficiently, supporting the practical value of the approach in a demanding setting.

Aug 3, 2026

Upload by Anda