[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83917-en":3,"doc-seo-83917-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83917,8796095461610,"Oliver","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Latent Programming Horizons in Coding Agents","A coding agent working on software-engineering tasks executes many reasoning, code-editing, and test-running steps, yet internal representations of the evolving program remain poorly understood. This work shows that language-model residual streams, probed with logistic regression, linearly encode whether the current code parses, passes tests, reduces failing tests, and introduces regressions. Probes also forecast future edits’ outcomes, achieving above-chance prediction up to about 25 steps ahead, indicating a latent programming horizon. Results transfer across benchmarks without retraining.","arXiv :2607 .05 188v 1 [ cs .LG] 6 Jul 2026  \nLatent Programming Horizons in Coding Agents  \nAndr´e Silva, Han Tu, Martin Monperrus  \nKTH Royal Institute of Technology Stockholm, Sweden  \n{andreans, htu, [monperrus](monperrus}@kth.se)[}](monperrus}@kth.se)[@kth.se](monperrus}@kth.se)  \nAbstract  \nA coding agent solving a software-engineering task spends dozens of steps reasoning, editing code, and running tests, yet little is known about what the underlying language model internally represents about the program it is working on. We show that the residual streams of language models under coding agents linearly encode properties of the evolving program:  \na logistic-regression probe on hidden states is able to decode whether the current code parses, passes its test suite, reduces the number of failing tests, and introduces regressions, reaching AUC up to 0.83 for correctness across two models and two benchmarks. Our second finding is more surprising:  \nthese representations run ahead of the agent’s own edits. Probes trained to predict the outcome of future edits (before they are materialized and written on disk) achieve performance above chance up to roughly 25 steps in advance. We call this the agent’s latent programming horizon. As a proof of external validity, we show that the probes transfer across benchmarks without retraining. Our positive results open calls for more research in mechanistic interpretability of coding agents.  \n§ Code |  Data  \n1 Introduction  \nCoding agents are increasingly used to solve complex software engineering tasks. Over many steps, they read files, reason about the problem, edit code, run tests, and revise their changes, iterating until the task is complete. Despite intense interest in building such agents, strikingly little is known about how the underlying language model represents the code it is working on. In this paper, we study the following question: what does a coding agent internally represent about the program it is editing?  \nPrior work in other domains gives reason to expect a rich answer. Transformers trained onboard games develop linearly decodable internal representations of the game state (Li et al., 2023; Nanda et al., 2023; Karvonen, 2024) . Models trained on synthetic grid-world programs encode both the current and future semantic state of those programs (Jin & Rinard, 2024) . In the software domain, prior work (Ribeiro et al., 2025; Bui et al., 2025; Vu et al., 2025) has established that the correctness of a single generated function is linearly decodable from a model’s hidden states. Yet, these papers assume single-step generation where the full program sits in context and never changes. None of them is about coding agents iteratively editing an only partially observed real codebase, over dozens of steps, where the model’s hidden state shifts with every edit. This is the problem we address in this paper.  \nTo study this arguably difficult question, we proceed as follows. We systematically collect agentic trajectories, extract the residual stream of the model at each agentic step, and train linear probes to decode properties of the program being edited. We consider two open-weight models running mini-swe-agent (Yang et al., 2024) on two benchmarks (SWE-Bench-Verified (Jimenez et al., 2024) and SWE-Bench-Pro (Deng et al., 2025)) . Two main findings emerge.  \nP(correct) high  \nFigure 1: A coding agent iteratively edits code and runs tests (top); the hidden states at each step trace a corresponding path through the latent program space (bottom) . The space is shaded by probe-estimated P (correct), with a warm basin marking the final region where the final program is fully correct and the coding task is successful. Our experiments show that linear probes trained on hidden states decode program properties such as full correctness well above chance throughout the agentic trajectories. Beyond current state, the latent program representation captures future programs’ properties: this de","cbCait2F6D8YZrz9","https://ap.wps.com/l/cbCait2F6D8YZrz9","pdf",1639042,1,24,"English","en",105,"# Abstract\n# Introduction\n## Background and Motivation\n## Experimental Approach\n## Main Findings\n## Contributions","[{\"question\":\"How does the paper evaluate what a coding agent’s language model represents about the program?\",\"answer\":\"It collects multi-step agent trajectories, extracts residual streams at each step, and trains linear probes to decode properties of the program being edited.\"},{\"question\":\"Which current-program properties can the linear probes decode?\",\"answer\":\"The probes decode whether the current code parses, passes its test suite, reduces failing tests, and introduces regressions, with performance above chance under controls.\"},{\"question\":\"What is the “latent programming horizon” and how far ahead can it be predicted?\",\"answer\":\"The latent programming horizon is the agent’s internal representation of future programs. Probes can predict outcomes of future edits roughly up to 25 steps in advance, before the edits are written to disk.\"}]",1784191432,60,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"latent-programming-horizons-in-coding-agents","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/latent-programming-horizons-in-coding-agents/83917/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How does the paper evaluate what a coding agent’s language model represents about the program?","Question",{"text":75,"@type":76},"It collects multi-step agent trajectories, extracts residual streams at each step, and trains linear probes to decode properties of the program being edited.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Which current-program properties can the linear probes decode?",{"text":80,"@type":76},"The probes decode whether the current code parses, passes its test suite, reduces failing tests, and introduces regressions, with performance above chance under controls.",{"name":82,"@type":73,"acceptedAnswer":83},"What is the “latent programming horizon” and how far ahead can it be predicted?",{"text":84,"@type":76},"The latent programming horizon is the agent’s internal representation of future programs. Probes can predict outcomes of future edits roughly up to 25 steps in advance, before the edits are written to disk.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":28,"slug":108},5,"Comic","comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]