Can LLMs Really Recover Microservice Failures? A Recovery-Aware Evaluation of Diagnosis-to-Action Reasoning

Jul 16, 2026

Upload by Maeve

Large language models (LLMs) are used to interpret operational evidence and support incident response in cloud-native microservice systems, yet recovery-oriented scenarios require more than root-cause identification. Operators must convert diagnosis into a concrete recovery action, select an admissible target, and verify service health restoration. R2Act is proposed as a recovery-action evaluation framework, including an incident schema, actionspace modeling, recovery-validity metrics, an offline evaluator, and a live-replay protocol.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

Introduction

Background and problem motivation

Related work and identified gaps

Can LLMs Really Recover Microservice Failures? A Recovery-Aware Evaluation of Diagnosis-to-Action Reasoning arXiv :2607 .04623v 1 [ cs . SE] 6 Jul 2026 Jiaxing Qi∗ , Zhongzhi Luan∗ , Hongyu Zhang†, Shaohan Huang‡ Carol Fung§ , Yongxin Tong∗ , Hailong Yang∗ , and Depei Qian∗ ∗ Beihang University [jiaxingqi@buaa.edu.cn](jiaxingqi@buaa.edu.cn), [luan.zhongzhi@buaa.edu.cn](luan.zhongzhi@buaa.edu.cn), [yxtong@buaa.edu.cn](yxtong@buaa.edu.cn), [hailongyang@buaa.edu.cn](hailongyang@buaa.edu.cn), [depeiq@buaa.edu.cn](depeiq@buaa.edu.cn) †Chongqing University [hyzhang@cqu.edu.cn](hyzhang@cqu.edu.cn) ‡Microsoft Research Asia [shaohanh@microsoft.com](shaohanh@microsoft.com) § Concordia University carol.fung@concordia.ca Abstract—Large language models (LLMs) are increasingly used to interpret operational evidence and assist incident response in cloud-native microservice systems. However, recoveryoriented use cases require more than identifying a root cause. After observing symptoms and diagnosing a fault, an operator or agent must translate the diagnosis into a concrete recovery action, apply it to an admissible target, and verify that service health has been restored. Existing RCA and log-analysis evaluations are well-suited to diagnosis, but they do not characterize this subsequent action decision. This paper presents R2Act, a recovery-action evaluation framework for post-diagnosis incident response. R2Act defines an incident schema, quality gate, actionspace representation, recovery-validity metrics, offline evaluator, and live-replay protocol. We instantiate the framework as a benchmark dataset of 302 quality-audited Kubernetes incidents from Online Boutique. Each incident provides synchronized multi-modal observations, root-cause labels, an incident-specific action space, and annotated valid and invalid recovery plans. We evaluate heuristic, supervised, RCA-oriented, deep log, and LLMbased methods. The strongest RAG-based LLMs reach 91.4%– 99.7% root-cause service accuracy, yet their recovery validity remains only 36.8%–60.3% . Even when both the root-cause service and fault type are correct, recovery-oriented methods still choose invalid actions for 39.5%–62.0% of correctly diagnosed incidents. In validity-gated live replay, 146 of 302 Qwen-RAG predictions are both offline-valid and recovered in live execution. Overall, this work reveals that many recovery failures arise not from missing diagnostic knowledge, but from the difficulty of translating diagnostic evidence into valid recovery actions and admissible targets. This work provides a reproducible, simplified starting point for research and evaluation. Index Terms—microservices, fault diagnosis, recovery planning, recovery-aware evaluation, LLMs I. INTRODUCTION AI-assisted operations are advancing rapidly along several axes. LLM-based operational systems can summarize logs, retrieve similar incidents, localize likely root causes, and draft mitigation steps [1], [2] . At the same time, cloud-native microservice systems have become harder to recover because Fig. 1. R2Act adds a recovery-action evaluation layer after diagnosis, checking whether methods can select a valid operation and target under incident-specific constraints. independent deployment, elastic scaling, and dense service dependencies create more ways for failures to propagate [3],[4], [5] . A user-visible incident may leave evidence scattered across logs, Kubernetes events, metrics, and resource state. These trends change incident response from a diagnosis-only task into an action-oriented decision problem: an automated method must not only explain what failed, but also decide which recovery action should be taken and where that action should be applied. Yet there have been relatively few efforts in this recovery process. Real microservice recovery begins with partial, noisy, and multi-modal operational evidence, while the required outcome is a concrete decision that may modify a l

FAQs

What problem does this paper focus on for LLMs in microservice incident response?

It focuses on whether LLM-based methods can translate post-diagnosis information into valid recovery actions, including selecting an admissible target and parameters, not just identifying root causes.

What is R2Act and what components does it include?

R2Act is a recovery-action evaluation framework that defines an incident schema, represents action spaces, introduces recovery-validity metrics, and supports both offline evaluation and a live-replay protocol.

What do the experiments show about diagnostic accuracy versus recovery validity?

Even when root-cause service and fault type are correct, recovery-oriented methods still select invalid actions frequently, indicating that translating diagnostic evidence into valid recovery decisions is the main challenge.

Jul 16, 2026

Upload by Maeve