An Empirical Study of Self-Admitted Technical Debt in Machine Learning Software - research findings

Aug 5, 2026

Machine learning software development often produces technical debt through sub-optimal design and implementation choices that developers explicitly acknowledge in code comments, known as self-admitted technical debt (SATD). This study analyzes 318 open-source ML projects across five domains and compares them with 318 non-ML projects. It detects SATD in comment snapshots, performs manual characterization of sampled SATD, and applies survival analysis to model evolution dynamics. Results show ML has higher SATD rates, greater susceptibility in preprocessing and pipeline stages, earlier appearance, and long-lasting SATD linked to large, low-complexity multi-file code changes.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages41

Reading Time1 h 43 min

AccessFree to download

Introduction

Motivation and context for SATD in ML

Study Design

Research goals and dataset

SATD detection and manual analysis

Survival analysis approach

Observations and Findings

SATD rate comparison: ML vs non-ML

Pipeline-stage susceptibility patterns

Timing of SATD introduction

Characteristics of long-lasting SATD

Contributions and Implications

arXiv :2311 . 12019v3 [ cs . SE] 27 Nov 2025 An Empirical Study of Self-Admitted Technical Debt in Machine Learning Software AADITYA BHATIA, SAIL, Queen’s University, Canada FOUTSE KHOMH, Polytechnique Montréal, Canada BRAM ADAMS, MCIS, Queen’s University, Canada AHMED E HASSAN, SAIL, Queen’s University, Canada The emergence of open-source ML libraries such as TensorFlow and Google Auto ML has enabled developers to harness state-of-the-art ML algorithms with minimal overhead. However, during this accelerated ML development process, said developers may often make sub-optimal design and implementation decisions, leading to the introduction of technical debt that, if not addressed promptly, can significantly impact on the quality of ML-based software. Developers frequently acknowledge these sub-optimal design and development choices through code comments written during development. These comments, which often highlight areas requiring additional work or refinement in the future are known as self-admitted technical debt (SATD). While prior research has demonstrated that SATD can serve as a reliable indicator of technical debt and has extensively studied SATD in traditional (non-ML) software, little attention has been given to this issue in the context of ML. This paper aims to investigate the occurrence of SATD in ML code by analyzing 318 open-source ML projects across five domains, along with 318 non-ML projects. We detected SATD in source code comments in various snapshots of the studied projects, conducted a manual analysis of a sample of the identified SATD to comprehend the nature of technical debt in the ML code, and performed a survival analysis of the SATD to understand the evolution dynamics of such debts. Our analyses yielded the following observations: (i) Machine learning projects have a median percentage of SATD that is twice that of non-machine learning projects. (ii) ML pipeline stages for data preprocessing and model generation logic are more susceptible to debt than model validation and deployment stages. (iii) SATDs appear in ML projects earlier in the development process compared to non-ML projects. (iv) Long-lasting SATDs are typically introduced during extensive code changes that span multiple files, which exhibit low complexity. Our research contributes to the understanding of technical debt in an ML context and underscores the need for targeted debt management strategies. This contribution is particularly relevant for developers and stakeholders in ML projects by aiding them in identifying and addressing technical debt proactively and paving the way for future research in developing automated tools and methodologies for managing SATD inan ML environment. Additional Key Words and Phrases: Self Admitted Technical Debt, Machine Learning, Temporal Analysis ACM Reference Format: Aaditya Bhatia, Foutse Khomh, Bram Adams, and Ahmed E Hassan. 2025. An Empirical Study of Self-Admitted Technical Debt in Machine Learning Software. ACM Trans. Softw. Eng. Methodol. 1, 1 (December 2025), 41 pages. [https://doi.org/10.1145/nnnnnnn.nnnnnnn](https://doi.org/10.1145/nnnnnnn.nnnnnnn) Authors’ Contact Information: Aaditya Bhatia, [aaditya.bhatia@queensu.ca](aaditya.bhatia@queensu.ca), SAIL, Queen’s University, Kingston, Canada; Foutse Khomh, [foutse.khomh@polymtl.ca](foutse.khomh@polymtl.ca), Polytechnique Montréal, Canada; Bram Adams, [bram.adams@queensu.ca](bram.adams@queensu.ca), MCIS, Queen’s University, Kingston, Canada; Ahmed E Hassan, [ahmed@cs.queensu.ca](ahmed@cs.queensu.ca), SAIL, Queen’s University, Kingston, Canada. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with cred

FAQs

What is self-admitted technical debt (SATD) in this study?

SATD refers to technical debt that developers acknowledge in code comments during development, typically highlighting areas needing future work or refinement.

How was SATD investigated across machine learning and non-machine learning projects?

The study analyzes 318 open-source ML projects and 318 non-ML projects by detecting SATD in source-code comments, manually analyzing a sample for nature of debt, and running survival analysis to study evolution over time.

What stages in ML pipelines are most prone to SATD?

SATD is more susceptible in ML pipeline stages related to data preprocessing and model generation logic than in model validation and deployment stages.

Aug 5, 2026

Upload by Emma Wilson