Adaptive Machine Learning for Dynamic Environments: Evaluating Data Drift-Triggered Retraining in COVID-19 Severity Prediction

Aug 5, 2026

Machine learning (ML) models deployed in operational systems often suffer performance degradation when incoming data changes over time. This study evaluates four adaptive retraining strategies—naïve, periodic, clinically guided context-driven, and data drift-triggered—for predicting severe COVID-19 outcomes. Using a large-scale CDC dataset, drift-triggered retraining achieves predictive performance comparable to periodic retraining while reducing retraining frequency and enabling automated adaptive learning. Context-driven retraining also performs well but depends on expert input and lacks automation. Results offer practical guidance for maintaining ML systems under evolving healthcare data.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

AccessFree to download

1. Introduction

2. Dynamically Changing Environments and COVID-19

Formalizing the dynamic setting and risk minimization

Proceedings of the 59th Hawaii International Conference on System Sciences | 2026 Adaptive Machine Learning for Dynamic Environments: Evaluating Data Drift-Triggered Retraining in COVID-19 Severity Prediction Young U. Ryu Jindal School of Management 􀹠e University of Texas at Dallas [ryoung@utdallas.edu](ryoung@utdallas.edu) [Varghese S. Jacob](Varghese S. Jacob)[ ](Varghese S. Jacob)Jindal School of Management 􀹠e University of Texas at Dallas [vjacob@utdallas.edu](vjacob@utdallas.edu) Mehmet Ulvi SaygiAyvaci Jindal School of Management 􀹠e University of Texas at Dallas [Mehmet.Ayvaci@utdallas.edu](Mehmet.Ayvaci@utdallas.edu) HarshalAgrawal Erik Jonsson School of Engineering & Computer Science 􀹠e University of Texas at Dallas [Harshal.Agrawal@utallas.edu](Harshal.Agrawal@utallas.edu) Abstract Machine learning (ML) models deployed in operational systems often face performance degradation due to data drift. Designing eﬀective, scalable strategies for adaptive model maintenance remains an open challenge. This study evaluates four retraining strategies—naïve (static), periodic, clinically guided context-driven, and data drift-triggered—in a dynamic, high-stakes environment: predicting severe COVID-19 outcomes. Using a large-scale CDC dataset, we ﬁnd that data drift-triggered retraining strikes aneﬀective tradeoﬀ between predictive performance andretraining cost. It matches the performance of periodic retraining while requiring far fewer retraining cyclesand oﬀers a fully automated mechanism for adaptive learning. In contrast, context-driven retraining performs well and requires fewer retraining cycles but depends on expert input and lacks automation. Our ﬁndings provide empirical insights and practical guidance for designing adaptive ML systems in dynamic environments, with implications for both researchers and practitioners deploying ML models in healthcare and other domains with evolving data landscapes. Keywords: adaptive machine learning, data drift, COVID-19 severity prediction, automated re-training 1. Introduction Machine learning (ML) models are increasingly being used for operationally important decisions across domains such as healthcare, ﬁnance, and public policy. With this growing adoption, a critical challenge is maintaining the performance of these models over time. Real-world environments are dynamic. Shifts in data distributions and outcome relationships—respectively known as data drift and concept drift—can degrade the performance of deployed models. Determining when and how to update deployed models has become an area of increasing research interest with signiﬁcant practical relevance. A straightforward approach to model maintenance is to periodically retrain models on new data, or to rely on domain experts to trigger retraining based on external events. Both strategies have limitations. Periodic retraining is resource-intensive and can introduce instability. Expert-driven retraining may miss subtle shifts or rely on heuristics that are diﬃcult to generalize across settings. 􀹠ese challenges point to the need for scalable, data-driven approaches to monitor model validity and guide retraining decisions in operational information systems. Data drift detection oﬀers a promising mechanism to address this gap. While methods for detecting shiftsin data distributions have been explored in the ML literature (Gama et al., 2014), their application in live settings is still an active area of research. We empirically evaluate the use of data drift-triggered retraining in a complex, high-stakes environment: predicting severe COVID-19 outcomes using patient data. 􀹠e COVID-19 pandemic provides a natural testbed, with multiple sequential data drifts caused by virus variants, vaccination campaigns, and evolving treatment practices. We compare four retraining strategies: naive (static) models, periodic retraining, clinically guided contextdriven retraining (based on Ayvaci et al., 2025), and

FAQs

Why does model performance degrade in operational ML systems?

Performance can degrade when the data distribution and the relationships between features and outcomes change over time, known as data drift and concept drift. These shifts make models trained earlier less accurate.

Which retraining strategies are compared in the study?

The study compares four strategies: naïve (static) models, periodic retraining, clinically guided context-driven retraining, and data drift-triggered retraining.

What is the main finding about data drift-triggered retraining?

Data drift-triggered retraining reaches predictive performance comparable to periodic retraining while requiring far fewer retraining cycles, providing a more automated mechanism for adaptive learning.

Aug 5, 2026

Upload by Theodora