Self-Adaptive Anomaly Detection with Reinforcement Learning and Human Feedback in Connected Vehicles

Jul 16, 2026

Connected vehicles require continuous monitoring to detect deviations before they propagate into failures, yet static diagnostics degrade as the system evolves through over-the-air updates, configuration changes, and shifting workloads that introduce concept drift. The framework addresses the gap between isolated automated adaptation and operator integration by combining coordinated online mechanisms. It uses a factorized deep Q-network with self-attention for service-specific detector selection, an ensemble of three drift detectors with a conjunctive alarm rule, and human-in-the-loop retraining with pending transition buffering and prioritized replay to incorporate expert knowledge.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages8

Reading Time20 min

Introduction

Challenges of continuous behavior evaluation under evolution

Limitations of existing automated adaptation and operator-in-the-loop methods

Related Work

Anomaly detection, concept drift, and human-in-the-loop adaptation

Self-Adaptive Anomaly Detection with Reinforcement Learning and Human Feedback in Connected Vehicles Matthias Weiß, Athreya Hosahalli Prakash, Maurice Artelt, Falk Dettinger, Nasser Jazdi and Michael Weyrich Institute of Industrial Automation and Software Engineering (IAS) University of Stuttgart Pfaffenwaldring 47, 70550 Stuttgart, Germany [E-Mail:](E-Mail: {vorname.nachname}@ias.uni-stuttgart.de)[ {](E-Mail: {vorname.nachname}@ias.uni-stuttgart.de)[vorname.nachname](E-Mail: {vorname.nachname}@ias.uni-stuttgart.de)[}](E-Mail: {vorname.nachname}@ias.uni-stuttgart.de)[@ias.uni-stuttgart.de](E-Mail: {vorname.nachname}@ias.uni-stuttgart.de) arXiv :2607 .08373v 1 [ cs .LG] 9 Jul 2026 Abstract—Connected vehicles are autonomous cyber-physical systems whose behavior must be continuously monitored during operation to detect deviations from normal operation before they propagate into failures. Such evaluation is challenging because the systems themselves evolve: over-the-air updates, configuration changes, and shifting workloads alter the definition of normal behavior, causing static diagnostic methods to degrade silently over time. Existing approaches typically address either automated model adaptation or operator integration in isolation, rather thanas a single coordinated supervisory loop. This paper presents an online anomaly detection framework for autonomous CPS that integrates three coordinated mechanisms. A factorized deep Q-network with self-attention selects the most suitable detector from a candidate pool for each monitored service, exploiting inter-service dependencies in the microservice topology. An ensemble of three statistical drift detectors monitors the input distribution and raises an alarm only when all three concur, prioritizing precision over recall. A human-in-the-loop retraining mechanism, built around a pending transition buffer and a 60/40 prioritized replay strategy, allows the operator to incorporate expert knowledge while preserving the system’s learned response to prior data distributions. The framework is evaluated on a connected-vehicle testbed running an automated valet parking application across seven backend microservices. The attention-augmented agent achievesan F1 score of 0.69, compared to at most 0.11 for any single detector applied uniformly. Following a real software update that induces measurable concept drift, F1 drops to 0.52; after operator-triggered retraining, performance recovers to 0.65 on the new distribution while remaining at 0.69 on the prior one, demonstrating sustained adaptation without catastrophic forgetting. Index Terms—Connected Vehicles, Anomaly Detection, Diagnosis, Reinforcement Learning, Human Feedback I. INTRODUCTION Connected vehicle (CV) functions such as over-the-air updates, remote diagnostics, real-time traffic optimization, or cooperative maps, are increasingly deployed in production vehicles, transforming the vehicle from a standalone product into one node in a distributed cyber-physical system spanning cloud services, edge nodes, and other vehicles [1]–[3] . These systems operate with growing autonomy: they coordinate among themselves, allocate computational resources, and respond to environmental conditions with limited operator involvement [4], [5] . As autonomy grows, so does the need to continuously evaluate operational behavior, in order to detect deviations before they propagate into service-affecting or safety-relevant failures [6], [7] . Behavior evaluation in such systems is challenging because the systems themselves evolve. Frequent over-the-air updates, configuration changes, and shifting workloads alter the statistical properties of monitored signals, creating concept drift that silently degrades static diagnostic methods over time [8] . Compounding this, the volume and heterogeneity of signals across cloud, edge, and vehicle domains exceed what manual operator inspection can reasonably cover [9] . Diagnostic methods must

FAQs

Why do static anomaly detection methods fail in connected vehicle systems over time?

Over-the-air updates, configuration changes, and shifting workloads alter the statistical properties of monitored signals, creating concept drift that silently degrades fixed diagnostic behavior.

How does the proposed framework decide which detector to use for each service?

A factorized deep Q-network with self-attention selects an appropriate detector from a candidate pool for each monitored service while leveraging inter-service dependencies in the microservice topology.

What role does human feedback play in maintaining performance after updates?

A human-in-the-loop retraining mechanism uses a pending transition buffer and a 60/40 prioritized replay strategy so operators can incorporate expert knowledge while preserving previously learned response behavior.

Jul 16, 2026

Upload by Valentina