Detecting Refactoring Commits in Machine Learning Python Projects - A Machine Learning-Based Approach

Aug 2, 2026

Upload by Ezra

Refactoring improves software quality without changing functional behavior, yet developers’ refactoring practices are hard to study because maintenance documentation is often incomplete or vague. In machine learning (ML) Python ecosystems, refactoring follows a data-driven development process and includes ML-specific refactoring types that existing rule-based detectors for Python cannot reliably find. MLRefScanner, a machine-learning prototype, detects refactoring-related commits in ML Python libraries beyond state-of-the-art coverage, evaluated on 199 open-source projects and validated against Python refactoring tools.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages22

Reading Time55 min

AccessFree to download

Introduction

Refactoring and the need for detection

Limitations of existing tools in Python ML projects

Proposed approach

MLRefScanner design using commit messages

Evaluation

Dataset and comparison setup

Results for precision and recall

Results and implications

Combining MLRefScanner with PyRef

Future directions

arXiv :2404 .06572v 1 [ cs . SE] 9 Apr 2024 Detecting Refactoring Commits in Machine Learning Python Projects: A Machine Learning-Based Approach SHAYAN NOEI, Queen’s University, Canada HENG LI, Polytechnique Montréal, Canada YING ZOU, Queen’s University, Canada Refactoring aims to improve the quality of software without altering its functional behaviors. Understanding developers’ refactoring activities is essential to improve software maintainability. The use of machine learning (ML) libraries and frameworks in software systems has significantly increased in recent years, making the maximization of their maintainability crucial. Due to the data-driven nature of ML libraries and frameworks, they often undergo a different development process compared to traditional projects. As a result, they may experience various types of refactoring, such as those related to the data. The state-of-the-art refactoring detection tools have not been tested in the ML technical domain, and they are not specifically designed to detect ML-specific refactoring types (e.g., data manipulation) in ML projects; therefore, they may not adequately find all potential refactoring operations, specifically the ML-specific refactoring operations. Furthermore, a vast number of ML libraries and frameworks are written in Python, which has limited tooling support for refactoring detection. PyRef, a rule-based and state-of-the-art tool for Python refactoring detection, can identify 11 types of refactoring operations with relatively high precision. In contrast, for other languages such as Java, state-of-the-art tools are capable of detecting a much more comprehensive list of refactorings. For example, Rminer can detect 99 types of refactoring for Java projects. Inspired by previous work that leverages commit messages to detect refactoring, we introduce MLRefScanner, a prototype tool that applies machine-learning techniques to detect refactoring commits in ML Python projects. MLRefScanner detects commits involving both ML-specific refactoring operations and additional refactoring operations beyond the scope of state-of-the-art refactoring detection tools. To demonstrate the effectiveness of our approach, we evaluate MLRefScanner on 199 ML open-source libraries and frameworks and compare MLRefScanner against other refactoring detection tools for Python projects. Our findings show that MLRefScanner outperforms existing tools in detecting refactoring-related commits, achieving an overall precision of 94% and recall of 82% for identifying refactoring-related commits. MLRefScanner can identify commits with ML-specific and additional refactoring operations compared to state-of-the-art refactoring detection tools. When combining MLRefScanner with PyRef, we can further increase the precision and recall to 95% and 99%, respectively. MLRefScanner provides a valuable contribution to the Python ML community, as it allows ML developers to detect refactoring-related commits more effectively in their ML Python projects. Our study sheds light on the promising direction of leveraging machine learning techniques to detect refactoring activities for other programming languages or technical domains where the commonly used rule-based refactoring detection approaches are not sufficient. CCS Concepts: • Software and its engineering; Additional Key Words and Phrases: code refactoring, refactoring detection, python refactoring, code quality, refactoring evolution ACM Reference Format: Shayan Noei, Heng Li, and Ying Zou. 2018. Detecting Refactoring Commits in Machine Learning Python Projects: A Machine Learning-Based Approach. 1, 1 (April 2018), 22 pages. [https://doi.org/XXXXXXX.XXXXXXX](https://doi.org/XXXXXXX.XXXXXXX) Authors’ addresses: Shayan Noei, [s.noei@queensu.ca](s.noei@queensu.ca), Queen’s University, Kingston, Canada; Heng Li, [heng.li@polymtl.ca](heng.li@polymtl.ca), Polytechnique Montréal, Montréal, Canada; Ying Zou, [ying.zou@queensu.ca](ying.zou@queensu.ca),

FAQs

Why is detecting refactoring commits especially important in ML Python projects?

ML Python libraries evolve through a data-driven process, producing refactoring types that may be ML-specific and not well captured by existing Python tooling, making accurate detection crucial for maintainability insights.

How does MLRefScanner detect refactoring commits?

MLRefScanner applies machine-learning techniques inspired by prior work using commit messages, aiming to identify commits with ML-specific refactoring operations and additional refactorings beyond rule-based tool coverage.

What performance did MLRefScanner achieve and how can it be improved?

On 199 ML open-source libraries, MLRefScanner achieved 94% precision and 82% recall for refactoring-related commits; combining it with PyRef further increased precision and recall to 95% and 99%, respectively.

Aug 2, 2026

Upload by Ezra