Cost-Effective Retraining of Machine Learning Models - Cara Algorithm - Optimizes Retraining Trade-off

Aug 4, 2026

Cost-effective retraining of machine learning (ML) models is critical as data drift occurs over time, but frequent retraining is expensive and infrequent retraining yields stale, inaccurate models. The work proposes automated, cost-aware systems that decide when to retrain by explicitly accounting for both staleness costs and retraining costs, including monetary and energy/resource expenses. The paper introduces a Cost-Aware Retraining Algorithm, Cara, and evaluates it on synthetic and real-world datasets, showing strong adaptation to data drifts, better accuracy than drift baselines, fewer retraining actions, and lower total cost.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages27

Reading Time1 h 8 min

AccessFree to download

Introduction

Data drift and the need for retraining decisions

Existing drift detection limitations (queries and costs)

Two decision costs: staleness vs retraining

Contributions and proposed approach

Cost-Aware Retraining Algorithm (Cara)

Oracle baseline and experimental evaluation

arXiv :2310 .042 16v 1 [ cs .LG] 6 Oct 2023 Cost-Effective Retraining of Machine Learning Models Ananth Mahadevan, Michael Mathioudakisa Department of Computer Science, University of Helsinki, Finland Abstract It is important to retrain a machine learning (ML) model in order to maintain its performance as the data changes over time. However, this can be costly as it usually requires processing the entire dataset again. This creates a trade-off between retraining too frequently, which leads to unnecessary computing costs, and not retraining often enough, which results in stale and inaccurate ML models. To address this challenge, we propose ML systems that make automated and cost-effective decisions about when to retrain an ML model. We aim to optimize the trade-off by considering the costs associated with each decision. Our research focuses on determining whether to retrain or keep an existing ML model based on various factors, including the data, the model, and the predictive queries answered by the model. Our main contribution is a Cost-Aware Retraining Algorithm called Cara, which optimizes the trade-off over streams of data and queries. To evaluate the performance of Cara, we analyzed synthetic datasets and demonstrated that Cara can adapt to different data drifts and retraining costs while performing similarly to an optimal retrospective algorithm. We also conducted experiments with real-world datasets and showed that Cara achieves better accuracy than drift detection baselines while making fewer retraining decisions, ultimately resulting in lower total costs. Keywords: machine learning, resource-aware computing 1. Introduction Retraining a machine learning (ML) model is essential in the presence of data drift [1], i.e., continuous changes in the data due to factors such as system modifications, seasonality, or changes in user preferences. As a data drift occurs, the performance of an ML model trained on old data typically decreases – or, more generally, is not as high as it could be if it took advantage of the new data. As a result, maintaining the performance of an ML model calls for an online decision on whether to Retrain or Keep the existing ML model. In what follows, we refer to any algorithm that makes such a decision as a retraining algorithm. Several methods exist to detect data drift [2, 3 , 4] by monitoring the errors of an ML model on a stream of labeled samples, i.e. , data. These methods decide to retrain the ML model whenever a drift is detected. However, note that (i) ML models are typically used to predict a stream of unlabeled samples [5], i.e. , queries, and (ii) (re)training is associated with costs, such as the monetary cost for (re)training a model on a cloud service or the energy cost for executing the training algorithm [6] . Considering queries and costs in addition Preprint submitted to Arxiv October 9, 2023 to data is essential in deciding whether to retrain a model. To see why, consider a real estate firm that uses an ML model to predict housing prices. Suppose, the data used to train the ML model represents the price distribution of all houses in the market. However, the customer queries concern only a niche of this market, for example, mansions. In such a case, if new data suggests prices have changed for studio apartments but not for mansions, retraining the model will not improve its performance on queries significantly enough to justify the retraining cost. By contrast, if the new data indicates that prices for mansions have changed significantly, then the potential drop in performance on queries will warrant a Retrain decision. Unfortunately, many existing drift detection methods do not consider queries or retraining costs, which may lead to suboptimal decisions for scenarios like the above. The above discussion gives rise to two costs when deciding whether to retrain. The first is a model staleness cost, i.e. , the performance loss due to keeping an ML model trained on

FAQs

Why is retraining an ML model necessary over time?

Retraining is needed because data drift changes the underlying data distribution, reducing model performance when using the old training data.

What are the two main costs considered in retraining decisions?

The approach balances staleness cost (performance loss from keeping an outdated model) and retraining cost (resources such as monetary cost or energy needed to retrain).

How does Cara improve retraining decisions compared with drift detection baselines?

Cara optimizes the trade-off using costs and factors from both data and query context, achieving better accuracy with fewer retraining decisions, which reduces total cost.

Aug 4, 2026

Upload by Ava Thompson