SWE-Doctor: Guiding Software Engineering Agents with Runtime Diagnosis from Multi-Faceted Bug Reproduction Tests

Jul 16, 2026

Upload by Finn

LLM-based software engineering agents generate patches from issue reports and code repositories, yet bug reproduction tests (BRTs) are not clearly understood as guidance for patch generation. A preliminary study shows that using advanced BRT generators directly is ineffective: fail-to-fail BRTs can mislead agents, while even fail-to-pass BRTs yield limited or negative gains due to partial coverage. SWE-Doctor instead uses multi-faceted BRT executions to build runtime-grounded diagnosis records, improving patch guidance and reducing partial patches. Evaluations on SWE-bench Verified and SWE-bench Pro across five LLM backends show consistent superiority and higher resolution rates.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

Introduction

Bug reproduction tests in agent-based patch generation

Limitations of direct BRT guidance

Proposed approach: runtime-grounded diagnoses

Evaluation

SWE-Doctor: Guiding Software Engineering Agents with Runtime Diagnosis from Multi-Faceted Bug Reproduction Tests Yaoqi Guo∗ , Yang Liu∗ , Jie M. Zhang†, Yun Ma‡, Yiling Lou§ , Zhenpeng Chen¶ ∗ Nanyang Technology University, †King’s College London, ‡Peking University, § University of Illinois Urbana-Champaign, ¶ Tsinghua University [yaoqi001@e.ntu.edu.sg](yaoqi001@e.ntu.edu.sg), [yangliu@ntu.edu.sg](yangliu@ntu.edu.sg), [jie.zhang@kcl.ac.uk](jie.zhang@kcl.ac.uk), [mayun@pku.edu.cn](mayun@pku.edu.cn), [yilingl@illinois.edu](yilingl@illinois.edu), [zpchen@tsinghua.edu.cn](zpchen@tsinghua.edu.cn) arXiv :2607 .00990v 1 [ cs . SE] 1 Jul 2026 Abstract—Large language model (LLM)-based software engineering agents are increasingly developed to resolve software issues by generating patches from issue reports and code repositories. Bug reproduction tests (BRTs) are an important building block for such agents and have been shown useful for patch validation. However, it remains unclear whether BRTs can also help the more central stage of patch generation. We first conduct a preliminary study and find that directly using advanced BRT generators to guide patch generation is not beneficial: fail-to-fail BRTs can mislead agents, while even fail-topass BRTs bring limited or negative gains. Our analysis reveals two reasons: fail-to-pass BRTs may cover only one manifestation of the reported issue, leading to partial patches, whereas failto-fail BRTs are unreliable as direct patch-generation targets. Motivated by these insights, we propose SWE-Doctor, a software issue resolution agent that guides patch generation with runtime diagnoses derived from multi-faceted BRT executions. SWEDoctor first generates multi-faceted BRTs for different behavioral requirements stated in the issue, then executes and debugs these BRTs to construct runtime-grounded diagnosis records, and finally uses the diagnoses together with localization information inferred during BRT generation to guide patch generation and reduce partial patches. We evaluate SWE-Doctor on Python bug-fixing issues from the widely adopted SWE-bench Verified and SWE-bench Pro across five LLM backends. SWE-Doctor consistently outperforms existing agents across all 10 LLM– benchmark combinations, achieving average resolution rates of 75.7% on SWE-bench Verified and 59.4% on SWE-bench Pro. In particular, on the more challenging SWE-bench Pro, SWE-Doctor improves the average resolution rate by 8.0–8.9 percentage points over the baseline agents. Further analyses show that both multi-faceted BRT generation and runtime-grounded diagnosis contribute substantially to SWE-Doctor’s effectiveness. Index Terms—Software Issue Resolution, Agent, Bug Reproduction Test, Runtime Diagnosis I. INTRODUCTION Large language models (LLMs) have accelerated the development of software engineering agents for resolving realworld software issues [1]–[3] . Given a reported issue and its corresponding buggy repository, such agents can navigate the codebase, inspect relevant files, edit source code, run tests, and Corresponding author: Zhenpeng Chen. submit patches in an end-to-end manner, with LLMs serving as their reasoning and decision-making backends. Bug reproduction test (BRT) generation is the task of generating tests that reproduce the bugs described in issue reports [4] . It is an important building block for software issue resolution because a BRT turns a textual bug report into executable feedback [1], [5] . In real-world issue resolution tasks, however, such tests are usually not provided [2]: an agent must infer the bug-triggering behavior from the issue report and the repository itself. As a result, effective BRT generation can provide agents with concrete executions that expose the reported bug and support downstream resolution. Recent studies [4], [6], [7] have proposed advanced BRT generators and shown that generated BRTs can improve issue resolution when used for patch validation, i.

FAQs

Why are bug reproduction tests important for LLM-based patch generation agents?

Bug reproduction tests turn a textual bug report into executable feedback by exposing bug-triggering behavior through concrete executions, which can support downstream resolution.

What problems arise when directly using advanced BRT generators to guide patch generation?

Fail-to-pass BRTs may cover only one manifestation of the reported issue, leading to partial patches, while fail-to-fail BRTs can mislead agents as direct patch-generation targets.

How does SWE-Doctor guide patch generation differently from prior BRT usage?

SWE-Doctor generates multi-faceted BRTs, executes and debugs them to produce runtime-grounded diagnosis records, and then uses these diagnoses together with localization information to guide patch generation and reduce partial patches.

Jul 16, 2026

Upload by Finn