[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84662-en":3,"doc-seo-84662-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84662,4810365810221,"Aurora","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",6,"Technology","AgentLTL Trace Verification Framework for Measuring, Enforcing, and Training Procedural Compliance","Tool-using LLM agents are commonly judged by final-answer correctness or by LLM-based judges, which ignore how answers are produced. In safety-critical contexts, procedural validity is part of correctness. AGENTLTL introduces an FO-LTL-derived language expressing procedural, semantic, and grounding constraints over agent traces, producing a deterministic judge-free compliance score. A single specification enables both pre-execution harnessing and reinforcement-learning finetuning, improving compliance and accuracy on complex workflow benchmarks.","AgentLTL: A Trace-Verification Framework for Measuring, Enforcing, and Training Procedural Compliance in Tool-Using LLM Agents  \nLaïla Elkoussy, Julien Perez†  \nLRE, EPITA  \n[laila.elkoussy@epita.fr](laila.elkoussy@epita.fr)  \n†Bpifrance  \njulien.perez@bpifrance.fr  \narXiv :2607 .02599v 1 [ cs . SE] 1 Jul 2026  \nAbstract  \nTool-using LLM agents are usually evaluated by final-answer correctness or LLM judges.  \nNeither captures how an answer was produced.  \nIn safety-critical settings, the procedure itself is part of correctness. In this paper, we introduce AGENTLTL, a language derived from FirstOrder Linear Temporal Logic (FO-LTL) that expresses procedural rules over agent traces. It yields a deterministic, judge-free compliance score. In this framework, a single specification drives two usages. The first is harnessing:  \nthe constraints score completed traces, or gate tool calls by checking each prefix online, before execution. The second is finetuning: the score serves as a dense reward. On a benchmark spanning ordering, branching, iteration, and grounding, block-and-warn harnessing improves compliance on five of seven models. Finetuning with the same reward yields +38 and +17.5 percentage point gains in accuracy and compliance on held-out patterns, including unseen tool-name aliases. These findings are consistent with the model acquiring procedural structure rather than memorizing surface tool names and procedures.  \n1 Introduction  \nTool-augmented LLM agents are increasingly used in enterprise and safety-critical settings, yet evaluation still focuses on final-answer accuracy, often with LLM judges. Two traces can produce the same answer while differing in retrieval, branching, or tool grounding. Answer correctness collapses these differences into a single outcome.  \nIn many settings, the procedure is part of correctness. A clinical triage agent that skips a required contraindication check is not correct, even if its recommendation is. Figure 1 illustrates this common shortcut: the agent answers from parametric memory instead of tool outputs. This fails silently when memory and environment diverge.  \nWe introduce AGENTLTL, a language derived from First-Order Linear Temporal Logic (FO-LTL) for expressing procedural, semantic, and grounding constraints over execution traces. Comparing atrace against these constraints yields a deterministic, judge-free compliance score. The same constraints support two uses. When the LTL fragment permits, they gate tool calls before execution, detecting and blocking violating calls. The score also serves as a reinforcement learning reward. A shared constraint language avoids drift between evaluation, deployment, and training.  \nContributions. (i) We introduce AGENTLTLand its compliance score. (ii) We build a benchmark of 12 workflow templates and evaluate 7 language models under three harnesses that intervene on the trace at different strengths, localizing failures in ordering, branching, iteration, and argument grounding. (iii) We show that the same score is an effective reward: finetuning a target model improves both compliance and answer correctness on held-out templates, including unseen tool-name aliases. (iv) We show that grounding constraints can detect parametric-memory hallucination by distinguishing supported answers from unsupported recall. AGENTLTL, the benchmark, training corpus, and scripts are available here.  \n2 Related Work  \nTool-augmented LLM agents. The tool-using paradigm began with ReAct (Yao et al., 2023) and expanded through finetuning (Schick et al., 2023 ; Patil et al., 2023), larger tool inventories (Liu et al., 2025 ; Li et al., 2023), and reliability benchmarks such as τ-bench (Yao et al., 2024) and AgentBoard (Ma et al., 2024) . BFCL (Patil et al., 2025) evaluates function calling against gold abstract syntax trees, while τ2-bench (Barres et al., 2025) studies dual-control settings where agents and users jointly modify shared state. OSWorld (Xie et al.,  \nFigure ","cbCaiogaMPhhJmaO","https://ap.wps.com/l/cbCaiogaMPhhJmaO","pdf",1243785,1,34,"English","en",105,"# Abstract\n# 1 Introduction\n## Contributions\n# 2 Related Work","[{\"question\":\"Why is final-answer correctness insufficient for tool-using LLM agents in safety-critical settings?\",\"answer\":\"Two traces can yield the same final answer while using different retrieval, branching, or tool grounding. In safety-critical scenarios, skipping required checks means the procedure itself is part of correctness, even if the recommendation matches.\"},{\"question\":\"What is AGENTLTL and what does its compliance score measure?\",\"answer\":\"AGENTLTL is a language derived from First-Order Linear Temporal Logic (FO-LTL) that expresses procedural rules over agent traces. Comparing a trace against these constraints yields a deterministic, judge-free compliance score reflecting ordering, presence, and grounding properties.\"},{\"question\":\"How is the same AGENTLTL specification used both for harnessing and for training?\",\"answer\":\"When the LTL fragment permits, the constraints gate tool calls before execution by checking each prefix online and blocking violating calls. The same score is also used as a dense reinforcement-learning reward for finetuning, improving both compliance and accuracy.\"}]",1784197539,86,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"agentltl-trace-verification-framework-for-measuring-enforcing-and-training-procedural-compliance","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/technology/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/agentltl-trace-verification-framework-for-measuring-enforcing-and-training-procedural-compliance/84662/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is final-answer correctness insufficient for tool-using LLM agents in safety-critical settings?","Question",{"text":75,"@type":76},"Two traces can yield the same final answer while using different retrieval, branching, or tool grounding. In safety-critical scenarios, skipping required checks means the procedure itself is part of correctness, even if the recommendation matches.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is AGENTLTL and what does its compliance score measure?",{"text":80,"@type":76},"AGENTLTL is a language derived from First-Order Linear Temporal Logic (FO-LTL) that expresses procedural rules over agent traces. Comparing a trace against these constraints yields a deterministic, judge-free compliance score reflecting ordering, presence, and grounding properties.",{"name":82,"@type":73,"acceptedAnswer":83},"How is the same AGENTLTL specification used both for harnessing and for training?",{"text":84,"@type":76},"When the LTL fragment permits, the constraints gate tool calls before execution by checking each prefix online and blocking violating calls. The same score is also used as a dense reinforcement-learning reward for finetuning, improving both compliance and accuracy.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,113,118,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":111,"slug":112},50,"technology",{"id":114,"doc_module":4,"doc_module_name":45,"category_name":115,"show_sort_weight":116,"slug":117},7,"Healthcare",40,"healthcare",{"id":119,"doc_module":4,"doc_module_name":45,"category_name":120,"show_sort_weight":121,"slug":122},8,"Research & Report",30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]