Assessing models for de-identification of Electronic Discharge Summary using Machine Learning tools - Master of Science Project

Aug 3, 2026

Upload by Miles

De-identification removes identifying information from Electronic Discharge Summary clinical records to safeguard individual privacy and reduce misuse of personal data across collection, processing, distribution, and publication. The Electronic Discharge Summary contains Protected Health Information (PHI), making de-identification mandatory. This project applies machine learning to determine which model best de-identifies a dataset. Using an open-source Harvard Medical School dataset, Conditional Random Fields, LSTM, and Random Forest are evaluated on token-level F-measure, Recall, and Precision. LSTM achieves the highest micro averages and strong macro performance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages58

Reading Time2 h 26 min

AccessFree to download

Introduction

Background

Problem Statement

Research Question

Research Aims and Objectives

Limitations

Overview

Literature Review

Related Work

Conclusion

Protection of Health Information in South Africa

How Personal Health Information is collected

How Personal Health Information is stored and protected

Logistics behind getting South African dataset

De-identification and its Rationale in USA

Sufficient degree of identification risk for a professional assessment

Validity period determined by experts for a given dataset

Research Methodology

Research design

Models

Data

Methods

Performance metrics

Analysis

Results and Discussion

Descriptive statistics

Experimental Results

Practical implications as a result of limitation

Conclusions and Future Work

Conclusions

Future Work

Assessing models for de-identification of Electronic Discharge Summary using Machine Learning tools Tshilisanani Mudau (11640828) Supervisor(s): Professor Winston Garira Dr Rendani Netshikweta A Project report submitted in partial fulfillment of the requirements for the degree of Master of Science in the field of e-Science in the Department of Computer Science and Applied Mathematics University of Venda 18 July 2024 i Dedicated to my late mom. How i wish you were here to witness God’s grace upon your only daughter. ii Declaration I, Tshilisanani Mudau (11640828), declare that this report is my own, unaided work. It is being submitted for the degree of Master of Science in the field of e-Science atthe University of Venda. It has not been submitted for any degree or examination at any other university. Tshilisanani Mudau (11640828) 18 July 2024 iii Abstract Background: De-identification is a technique that eliminates identifying information from Clinical Records in order to protect individual privacy. This procedure decreases the chance of personal information being collected, processed, distributed, and published from being used to identify the person. When Machine Learning techniques were included in the de-identification process, it substantially improved over the previous method. Research Problem: The Electronic Discharge Summary(EDS) has evolved into a significantly improved technique of providing discharge summaries though this information contains Protected Health Information (PHI), which poses a risk to patients’ privacy. This makes the process of de-identification to be mandatory. There have lately been several Machine Learning approaches to de-identify data. This study focuses on applying Machine Learning techniques to figure out which model can best de-identify a data set. Methods: The open source data set from Harvard Medical School was used. This data set contains 899 Electronic Health Records (EHR), 669 for training and 220 for test purpose. The Conditional Random Fields (CRF), Long Short Term Memory (LSTM) and Random Forest models were used, and the performance of each model was assessed. Findings: In order to assess each model’s performance, evaluation metrics were used to compare F-measure, Recall and Precision at token level to determine which Machine Learning model performed best. The Long Short Term Memory was found to outperform both Conditional Random Fields and Random Forest with micro average F-measure, Recall and precision of 99%, and macro average F-measure of 77%, Recall of 73% and Precision of 90% . iv Acknowledgements Above all, I would like to thank the Almighty God for everything, his unconditional love, mercy and grace. If it wasn’t of him, this would not have been possible. I’d like to express my sincere gratitude to my supervisor, professor Winston Garira and my co-supervisor Dr Rendani Netshikweta for their assistance. I would also like to thank everyone who assisted and supported me in any way. My heartfelt gratitude goes out to my family for their unfailing support. Funding was provided by the DSI-NICIS NEPTTP and is highly appreciated. v Contents Declaration ii Abstract iii Acknowledgements iv List of Figures vii List of Tables viii List of Abbreviations ix 1 Introduction 1 1.1 Background ................................. 3 1.1.1 The purpose of De-identification ................. 3 1.1.2 Natural Language Processing (NLP) ............... 4 1.1.3 Privacy Requirement ........................ 5 1.1.4 Ethical Considerations ....................... 5 1.2 Problem Statement ............................. 5 1.3 Research Question ............................. 6 1.4 Research Aims and Objectives ....................... 6 1.4.1 Research Aims ........................... 6 1.4.2 Objectives .............................. 7 1.5 Limitations .................................. 7 1.6 Overview ....................

FAQs

Why is de-identification necessary for Electronic Discharge Summary records?

Electronic Discharge Summary information includes Protected Health Information (PHI). De-identification is required to protect patient privacy and reduce the risk of personal information being used to identify individuals.

Which machine learning models were compared in the study?

The project compares Conditional Random Fields (CRF), Random Forest, and Long Short-Term Memory (LSTM). Each model is assessed using token-level evaluation metrics.

How were the models evaluated, and which performed best?

Models are evaluated using token-level F-measure, Recall, and Precision. LSTM outperformed CRF and Random Forest, reaching 99% micro-average scores and 77% macro-average F-measure, with Recall at 73% and Precision at 90%.

Aug 3, 2026

Upload by Miles