[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84297-en":3,"doc-seo-84297-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84297,1374391974585,"Genevieve","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",6,"Technology","Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems","Production LLM agents often incur avoidable latency and reliability losses by regenerating code for repeated procedural steps on each request. A self-evolving, agentic tool-making pipeline compiles recurring SOP nodes into validated, versioned tools prior to deployment. Tool synthesis is grounded in the live environment via execution traces, backend schema/value observation, candidate generation, and repair against labeled cases. At runtime, the production agent calls compiled tools and falls back to coding only when necessary, reducing p50 latency and errors in fulfillment-center alarm triage.","Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems  \nKalle Kujanpää Ning Liu Shahnawaz Alam Yeshwanth Reddy Sura Tianyu Yang Kristina Klinkner Shervin Malmasi  \nAmazon, Fulfillment Technologies & Robotics  \n{kujanpaa,[malmasi}@amazon.com](malmasi}@amazon.com)  \narXiv :2607 .080 10v 1 [ cs .CL] 9 Jul 2026  \nAbstract  \nProduction LLM agents often waste latency and reliability by regenerating code for the same procedural steps on every request. We replace this inference-time coding loop with an agentic tool-making pipeline that compiles repeated SOP steps into validated, versioned tools before deployment. The tool-maker grounds synthesis in the live environment as it collects execution traces, observes backend schemas and values, generates candidate tools, and repairs them against labeled cases. At runtime, the production agent calls these tools directly and falls back to code generation only when needed. We deploy the approach in a Fulfillment Center alarm-triage system, where an agent diagnoses alarms against a 44-node SOP over heterogeneous metric backends. In production, tool calls reduce p50 latency by 42% . On 1,500 historical alarms, they reduce end-to-end error rate by up to 53% by suppressing run-to-run variance in repeated steps. Because tools return compact structured verdicts, they also enable a simpler direct-call architecture, reducing p50 latency by a further 62% in a controlled ablation. Versioned tools also improve auditability and expose specification gaps and upstream data drift. Our results show that self-evolving agents can make industrial LLM systems faster, more reliable, and easier to operate.  \n1 Introduction  \nLLM coding agents are becoming a practical interface to complex software and operational systems. Instead of a fixed API or hand-written automation per task, these agents read natural-language instructions, write and execute code, and iterate until they produce an answer or action. This flexibility has enabled deployments in on-call incident triage (Chenet al., 2024b ; Wang et al., 2024b), workflow-guided operations (Ye et al., 2025), and general code-based tool use (Wang et al., 2024a) . In industry settings, many procedures are documented as standard operating procedures (SOPs), while the data needed  \nto execute them resides in metric stores, logs, dashboards, and ticketing systems.  \nThis flexibility, however, creates an efficiency and reliability bottleneck. In the prevailing CodeAct-style paradigm (Wang et al., 2024a), the agent generates and executes fresh code for each request at inference time. When the same workflow is repeated against a stable backend, the agent re-interprets the same instruction, rediscovers the same schema, and regenerates similar code, raising latency, cost, and run-to-run variance. Prior benchmarks show agents still struggle with workflowguided tasks (Nandi et al., 2025 ; Wang et al., 2025b ; Riddell et al., 2026), and even SOP-enhanced multiagent systems leave a substantial accuracy gap on operational diagnostics (Pei et al., 2025) . In our setting, most latency and correctness errors are caused by translating underspecified SOP text into a concrete query against a production metric backend.  \nThis has motivated a shift from inference-time coding agents to self-evolving agents: systems that improve their own action space over time by constructing reusable tools, skills, or programs before they are needed in production. Recent work explores this through single-pass tool synthesis from examples or task descriptions (Cai et al., 2024 ; Yuan et al., 2024 ; Wang et al., 2024c), iterative build-time construction with environmental or test-based validation (Wang et al., 2023, 2024d ; Wölflein et al., 2025), and adjacent operational documents such as policies (Zwerdling et al., 2025) and troubleshooting guides (Mao et al., 2025) . The key idea is to amortize cost: a procedural step executed repeatedly is compiled into a tested tool once, then invoked cheaply an","cbCaia2OIIdAhHUL","https://ap.wps.com/l/cbCaia2OIIdAhHUL","pdf",353326,1,17,"English","en",105,"# Introduction\n## Motivation: latency and reliability bottlenecks in CodeAct-style agents\n## Shift to self-evolving agents and amortized tool construction\n## Production setting: fulfillment center SOP-based alarm diagnosis","[{\"question\":\"Why do production coding agents waste latency and reliability?\",\"answer\":\"They regenerate fresh code for each request in an inference-time loop, reinterpreting the same instructions and rediscovering schemas, which increases latency, cost, and run-to-run variance.\"},{\"question\":\"How does the proposed self-evolving agentic tool-making pipeline work?\",\"answer\":\"It compiles repeated SOP steps into validated, versioned tools using live execution traces and schema/value observation, then repairs candidates against labeled cases before deployment.\"},{\"question\":\"What changes at runtime compared with the baseline approach?\",\"answer\":\"The production agent calls the precompiled tools inline for each SOP node and falls back to code generation only when a tool is unavailable or fails, reducing latency and stabilizing repeated-step outcomes.\"}]",1784194643,43,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"tool-making-and-self-evolving-llm-agents-in-low-latency-systems","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/technology/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/tool-making-and-self-evolving-llm-agents-in-low-latency-systems/84297/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do production coding agents waste latency and reliability?","Question",{"text":75,"@type":76},"They regenerate fresh code for each request in an inference-time loop, reinterpreting the same instructions and rediscovering schemas, which increases latency, cost, and run-to-run variance.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the proposed self-evolving agentic tool-making pipeline work?",{"text":80,"@type":76},"It compiles repeated SOP steps into validated, versioned tools using live execution traces and schema/value observation, then repairs candidates against labeled cases before deployment.",{"name":82,"@type":73,"acceptedAnswer":83},"What changes at runtime compared with the baseline approach?",{"text":84,"@type":76},"The production agent calls the precompiled tools inline for each SOP node and falls back to code generation only when a tool is unavailable or fails, reducing latency and stabilizing repeated-step outcomes.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,113,118,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":111,"slug":112},50,"technology",{"id":114,"doc_module":4,"doc_module_name":45,"category_name":115,"show_sort_weight":116,"slug":117},7,"Healthcare",40,"healthcare",{"id":119,"doc_module":4,"doc_module_name":45,"category_name":120,"show_sort_weight":121,"slug":122},8,"Research & Report",30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]