[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82377-en":3,"doc-seo-82377-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82377,687197207919,"Theodora","https://ap-avatar.wpscdn.com/avatar/a000253d6f5f7c60be?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779446848396160552",8,"Research & Report","Failure as a Process: An Anatomy of CLI Coding Agent Trajectories","Large language model (LLM) coding agents are increasingly used for autonomous software engineering in terminal environments, but reliability remains uncertain. Existing studies explain why agents fail while treating failure as a final outcome. A process-oriented framework reframes failure as a temporal phenomenon across onset, evolution, and recovery, using 3,843 collected execution trajectories filtered to 1,794 for manual annotation. Analysis over 63,000+ steps yields findings on failure occurrence, causes, recovery dynamics, and cross-system consistency.","Failure as a Process: An Anatomy of CLI Coding Agent Trajectories  \nXiangxin Zhao 1,* , Han Li2,* , Shuaiting Li 1 , Tianyi Zhao 1 , Earl T. Barr 1 , Federica Sarro 1 , He Ye 1,†  \n1University College London, London, United Kingdom 2Nanjing University, Nanjing, China {xiangxin.zhao.21, [shuaiting.li.23](shuaiting.li.23), tianyi.zhao.24, e.barr, f.sarro, [he.ye](he.ye}@ucl.ac.uk)[}](he.ye}@ucl.ac.uk)[@ucl.ac.uk](he.ye}@ucl.ac.uk), [han.li.cs@smail.nju.edu.cn](han.li.cs@smail.nju.edu.cn)  \narXiv :2607 .095 10v 1 [ cs . SE] 10 Jul 2026  \nAbstract—Large language model (LLM) coding agents are increasingly deployed to autonomously perform software engineering tasks in terminal-based environments, making their reliability a growing concern. Existing empirical studies investigate why coding agents fail, yet they largely treat failure as a final outcome rather than a temporal process, providing limited insight into how failures emerge, evolve, and become unrecoverable. We present the first large-scale empirical study of CLI coding-agent failure trajectories, introducing a process-oriented framework that analyzes failure through its onset, evolution, and recovery across execution trajectories. We first collect 3,843 execution trajectories generated by seven frontier models across three coding-agent scaffolds (OpenHands, MiniSWE, and Terminus2) on Terminal-Bench, then carefully filter them to obtain 1,794 complete and valid trajectories for manual annotation (over 63,000 execution steps), from which we derive 14 findings spanning failure occurrence, root causes, recovery, and crosssystem consistency. Our findings show that coding-agent failures are predominantly driven by epistemic errors, typically begin within the first few execution steps, and often remain hidden until recovery is no longer possible, suggesting that improving codingagent reliability requires earlier validation and intervention rather than relying solely on final-outcome evaluation.  \nIndex Terms—LLM agents, empirical study, failure analysis  \nI. INTRODUCTION  \nLarge language model (LLM) agents are increasingly used to automate software engineering tasks, including repositorylevel bug fixing, issue resolution, environment setup, and software maintenance [1], [2], [3], [4] . Many of these tasks are performed in terminal-based environments, where agents interact directly with repositories, dependencies, and build systems through the command line. Products such as Claude Code [5], Codex [6], and Gemini CLI [7] exemplify this trend. However, such autonomy also introduces new reliability challenges: a single incorrect decision can silently propagate through many subsequent actions before the final outcome becomes apparent [8], [9] .  \nTo improve coding agent reliability, a growing body of empirical studies has sought to understand why agents fail by analyzing their execution trajectories and behaviors. Existing work investigates coding-agent failures from several complementary perspectives, including trajectory characterization and behavioral analysis [10], [11], failure attribution and diagnosis [12], [13], [14], [15], and failure taxonomies and evaluation artifacts [16], [17], [18] . Collectively, these studies  \n*These authors contributed equally. †Corresponding author.  \nTABLE I: Comparison with recent empirical studies of codingagent failures. Scale reports the number of analyzed trajectories unless otherwise noted. Setting denotes the evaluation environment.“Failure as Process” indicates whether a study explicitly models failure as a temporal process rather than only analyzing final outcomes.  \n\n| Work | Scale | Setting | Failure as Process |\n| --- | --- | --- | --- |\n| Bouzenia and Pradel [10] | 120 | Issue | ✗ |\n| TrajAudit [14] | 93 | Issue | ✗ |\n| Majgaonkar et al. [11] | 559 | Issue | ✗ |\n| AgentLens [17] | 2,614 | Issue | ✗ |\n| Who&When [12] | 184 | Multi-agent | ✗ |\n| MAST [15] | 1,600 | Multi-agent | ✗ |\n| Our Work | 3,843 | CLI | ✓ |\n\nhave substantially impro","cbCaicD6FKyrN23U","https://ap.wps.com/l/cbCaicD6FKyrN23U","pdf",921191,1,12,"English","en",105,"# Introduction\n## Related Empirical Studies\n## Limitations in Prior Work\n## Our Work and Approach","[{\"question\":\"How does the document define “failure as a process” for CLI coding agents?\",\"answer\":\"It models failure over time along execution trajectories, focusing on when a decisive error first occurs, whether the agent recognizes it, and whether the trajectory recovers or becomes unrecoverable.\"},{\"question\":\"What data and evaluation setup are used in the study?\",\"answer\":\"The study collects 3,843 execution trajectories using seven foundation models across three coding-agent scaffolds (OpenHands, MiniSWE, Terminus2) on Terminal-Bench, then filters them to obtain 1,794 complete and valid trajectories for manual annotation.\"},{\"question\":\"What do the findings suggest about the origins and visibility of failures?\",\"answer\":\"Failures are predominantly driven by epistemic errors, often begin within the first few execution steps, and may remain hidden until recovery is no longer possible.\"}]",1784180015,30,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"failure-as-a-process-an-anatomy-of-cli-coding-agent-trajectories","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/failure-as-a-process-an-anatomy-of-cli-coding-agent-trajectories/82377/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How does the document define “failure as a process” for CLI coding agents?","Question",{"text":75,"@type":76},"It models failure over time along execution trajectories, focusing on when a decisive error first occurs, whether the agent recognizes it, and whether the trajectory recovers or becomes unrecoverable.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What data and evaluation setup are used in the study?",{"text":80,"@type":76},"The study collects 3,843 execution trajectories using seven foundation models across three coding-agent scaffolds (OpenHands, MiniSWE, Terminus2) on Terminal-Bench, then filters them to obtain 1,794 complete and valid trajectories for manual annotation.",{"name":82,"@type":73,"acceptedAnswer":83},"What do the findings suggest about the origins and visibility of failures?",{"text":84,"@type":76},"Failures are predominantly driven by epistemic errors, often begin within the first few execution steps, and may remain hidden until recovery is no longer possible.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":28,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]