An automated method of identifying incorrectly labelled images based on the sequences of loss functions of deep learning networks - Abstract - Medical image label quality improvement

Sep 21, 2026

Uploaded by Violet

Deep learning medical image classification depends on correctly labelled data for training, validation, and testing, yet manual labelling can introduce errors in up to about 10% of images. An automated approach is proposed that uses the sequence of loss values across multiple training epochs to flag incorrectly labelled samples. Identified images are then reviewed and relabelled by senior physicians, improving labelled dataset quality and enhancing deep learning model performance. Experiments use a fundus dataset for referable diabetic retinopathy screening.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages9

Reading Time23 min

AccessFree to download

Introduction

Motivation: label quality impact

Causes of incorrect medical image labels

An automated method of identifying incorrectly labelled images based on the sequences of loss functions of deep learning networks Zhipeng Zhang1, Wenhui Shou1, Wengting Ma1, Dongjia Xing1, Qingqing Xu1, Li-Qun Xu1, Qingxia Fan 2, Ling Xu2 1 China Mobile Research Institute, Beijing 100032, China 2 Shenyang He Eye Hospital, Shenyang 110034, China [shouwenhui@chinamobile.com](shouwenhui@chinamobile.com) Abstract. Deep learning has been widely applied to medical image analysis tasks. Since the labelled medical images are the foundation of the training, validation, and test of deep learning classification models, the quality of labelling process could directly affect the performance of the models. However, it was estimated that up to ten percent of manually labelled medical images may be incorrectly labelled. In this paper, by utilizing the sequences of loss functions of deep learning classification networks through multiple training epochs, an automated method of identifying incorrectly labelled medical images was proposed. For those identified images, their labels could be further reviewed and updated by senior and experienced physicians, ultimately improving the quality of labelled medical image datasets, as well as the performance of the deep learning models. Two experiments were carried out to validate the effectiveness of the proposed method, based on a specific fundus image dataset for referable diabetic retinopathy screening. a) In the first experiment, the effectiveness of the method to accurately identify the incorrectly labelled samples from the whole labelled dataset was verified. For a fundus image dataset comprising 10788 samples with gold-standard labels (5394 non- referable diabetic retinopathy samples and 5384 referable diabetic retinopathy samples), the labels of a small part (6%, 648) of the images were intentionally changed to the opposite, in order to simulate the real-world situation. By utilizing the proposed method, 75.31%(488) of the incorrectly labelled samples were successfully identified, and only 4.85% (492) of the correctly labelled samples were wrongly identified as the incorrectly labelled ones. b) In the second experiment, by further reviewing those 980 samples (only 9.1% of the whole dataset) that were identified as incorrectly labelled from the dataset and updating their labels to the correct ones, the deep learning classification model for referable diabetic retinopathy screening was retrained. Tested on an independent test dataset with completely correct labels (700 non-referable diabetic retinopathy samples and 700 referable diabetic retinopathy samples), the best accuracy of the model was increased from 95.93%(trained on the dataset with 6% incorrectly labelled samples) to 96.50%(trained on the revised dataset with 1.5% incorrectly labelled samples), approaching the ideal value 96.57%(trained on the original dataset with 0% incorrectly labelled 2 samples), demonstrating the effectiveness of the proposed method to improve the performance of the deep learning models. Keywords: Deep Learning, Medical Image Classification, Incorrectly Labelled Sample Identification 1 Introduction Artificial intelligence (AI) has drawn intense interests around the world. In the healthcare industry, self-diagnostic symptom checkers and clinical decision support systems become a research topic and deep learning techniques-enabled medical image analysis has been applied to almost every medical specialty [1-6]. To realize this vision, it is necessary to obtain a large amount of medical images which were dedicatedly labelled by a group of physicians. These images would be used as either the training samples to construct a deep convolutional neural network (CNN) model, orthe validation and test samples to verify the performance of the model. However, when physicians manually label the medical images, especially a large amount of real-world images with varying levels of image quality, which wer

FAQs

Why do incorrectly labelled medical images affect deep learning models?

Because labelled images are the foundation for training, validation, and testing of classification models, label quality directly influences model performance.

How does the proposed method detect incorrectly labelled images?

It analyzes the sequences of loss functions across multiple training epochs from deep learning classification networks to identify samples likely to be incorrectly labelled.

What evidence supports the method’s effectiveness?

Two experiments on a fundus dataset for referable diabetic retinopathy screening show high detection accuracy and improved model accuracy after updating flagged labels.

Sep 21, 2026

Uploaded by Violet