Sensitive Data Detection with High-Throughput Machine Learning Models in Electrical Health Records

Aug 4, 2026

Upload by Miles

The work addresses the growing need to share healthcare data while protecting sensitive personal information under HIPAA. Existing solutions lack efficient ways to detect or remove protected health information (PHI) before sharing, especially because PHI fields vary heterogeneously across different holders’ databases. The study leverages a key observation that PHI metadata distributions differ from non-PHI fields. Using engineered metadata features and machine learning classification models, it automatically identifies PHI fields in structured EHR data and achieves 99% accuracy on unseen datasets.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

Introduction

De-identification and privacy requirements

Direct and indirect identifiers

De-identification methods

The Texas Medical Center Library DigitalCommons@TMC | Faculty, Staff and Student Publications | McWilliams School of Biomedical Informatics | | --- | --- | | 1-1-2023
Sensitive Data Detection with High-Throughput Machine Learning Models in Electrical Health Records
Kai Zhang Xiaoqian Jiang
Follow this and additional works at: [https://digitalcommons.library.tmc.edu/uthshis_docs](https://digitalcommons.library.tmc.edu/uthshis_docs)
Part of the Bioinformatics Commons, Biomedical Informatics Commons, Data Science Commons, and the Translational Medical Research Commons | | Recommended Citation Zhang, Kai and Jiang, Xiaoqian, "Sensitive Data Detection with High-Throughput Machine Learning Models in Electrical Health Records" (2023) . Faculty, Staff and Student Publications. 476. [https://digitalcommons.library.tmc.edu/uthshis_docs/476](https://digitalcommons.library.tmc.edu/uthshis_docs/476) This Article is brought to you for free and open access by the McWilliams School of Biomedical Informatics at DigitalCommons@TMC. It has been accepted for inclusion in Faculty, Staff and Student Publications by an authorized administrator of DigitalCommons@TMC. For more information, please contact [digcommons@library.tmc.edu](digcommons@library.tmc.edu). Sensitive Data Detection with High-Throughput Machine Learning Models in Electrical Health Records Kai Zhang, PhD, Xiaoqian Jiang, PhD University of Texas Health Science Center, Houston, TX, USA Abstract: In the era of big data, there is an increasing need for healthcare providers, communities, and researchers to share data and collaborate to improve health outcomes, generate valuable insights, and advance research. The Health Insurance Portability and Accountability Act of 1996 (HIPAA) is a federal law designed to protect sensitive health information by defining regulations for protected health information (PHI) . However, it does not provide efficient tools for detecting or removing PHI before data sharing. One of the challenges in this area of research is the heterogeneous nature of PHI fields in data across different parties. This variability makes rule-based sensitive variable identification systems that work on one database fail on another. To address this issue, our paper explores the use of machine learning algorithms to identify sensitive variables in structured data, thus facilitating the de-identification process. We made a key observation that the distributions of metadata of PHI fields and non-PHI fields are very different. Based on this novel finding, we engineered over 30 features from the metadata of the original features and used machine learning to build classification models to automatically identify PHI fields in structured Electronic Health Record (EHR) data. We trained the model on a variety of large EHR databases from different data sources and found that our algorithm achieves 99% accuracy when detecting PHI-related fields for unseen datasets. The implications of our study are significant and can benefit industries that handle sensitive data. Keywords: De-identification, Protected health information (PHI), Electronic health records (EHR), Machine learning algorithms 1. Introduction Because of improvements in online data tracking and sharing techniques, healthcare data privacy has become a major issue in recent years. In the fields of medicine and research, the sharing of data with other parties and for secondary uses is a frequent practice1. Patients, however, have voiced worries regarding the lack of control they have over how their data is used and shared2. There are many difficulties in maintaining data confidentiality in the context of healthcare research3. Although there are legal ways to share data, there is always space for improvement in terms of speeding up and securing data transmissions. To encourage the safe and responsible use of healthcare data, it is critical to address these concerns as soon as they arise. De-

FAQs

Why is sensitive data detection needed before healthcare data sharing?

HIPAA protects protected health information (PHI), but it does not provide efficient tools to detect or remove PHI prior to sharing. Efficient detection supports safer and faster data transmission.

What challenge do rule-based PHI identification systems face?

PHI fields are heterogeneous across different parties and databases, so rule-based identification that works for one dataset may fail on another.

How does the proposed method identify PHI fields in structured EHR data?

It uses an observation about differing metadata distributions between PHI and non-PHI fields. The approach engineers features from PHI metadata and trains machine learning classification models to automatically identify PHI-related fields.

Aug 4, 2026

Upload by Miles