[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82146-en":3,"doc-seo-82146-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82146,1099514067438,"River Wang","https://ap-avatar.wpscdn.com/avatar/100002539ee87300030?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780474512215547542",8,"Research & Report","SLBench: Evaluating How LLM Agents Follow Logical Relations in Skills","Agent skills extend LLM agents with reusable procedures, tools, and domain-specific workflows, but their safety hinges on resolving dependencies among interacting instructions. SkillLogic analyzes logical relations in skill files and derives executable tests from them, using a taxonomy of eight relation types such as preconditions, constraints, and fallbacks. A scan of over 5,000 public skills finds 70% include at least one logical relation. SLBench (86 cases) evaluates Codex and Claude Code, reporting unsafe rates up to 70%, driven by capability gaps and low-salience text, and SLGuard reduces violations by 63% on targeted cases.","SLBench: Evaluating How LLM Agents Follow Logical Relations in Skills  \nXuan Chen  \nPurdue University [chen4124@purdue.edu](chen4124@purdue.edu)  \nChengpeng Wang  \nPurdue University [wang6590@purdue.edu](wang6590@purdue.edu)  \nLu Yan  \nPurdue University [yan390@purdue.edu](yan390@purdue.edu)  \nXiangyu Zhang  \nPurdue University [xyzhang@purdue.edu](xyzhang@purdue.edu)  \narXiv :2607 .090 16v 1 [ cs .CR] 10 Jul 2026  \nAbstract  \nAgent skills extend LLM agents with reusable procedures, tools, and domain-specific workflows, but their safety depends on resolving dependencies among interacting instructions. We introduce SkillLogic, a framework for analyzing logical relations in skill files and constructing executable tests from them. Our taxonomy covers eight relation types, including preconditions that gate valid actions, constraints that limit how allowed actions may be performed, and fallbacks that specify recovery behavior after failure. Using SkillLogic, we scan over  \n5,000 public skills and find that 70% contain at least one logical relation. We then construct SLBench, an 86-case executable benchmark from high-confidence, high-impact, and locally testable relations. Evaluating Codex and Claude Code across six LLM backbones shows unsafe rates up to 70%, with violations leading to privacy leaks, unsafe configuration changes, and incomplete cleanup. The human audit attributes failures to both agent capability gaps and low-salience skill text. We further show that SLGuard, a lightweight inferencetime scaffold, reduces violations by 63% on targeted cases. Our results establish logicalrelation following as a distinct reliability challenge for skill-guided agents.  \n1 Introduction  \nLLM agents increasingly rely on external skills: reusable instruction files that package domain knowledge, workflows, and operational constraints (Yao et al., 2023 ; Schick et al., 2023 ; Qin et al., 2024 ; Liu et al., 2024 ; Zhou et al., 2024) . Unlike single instructions, skills are procedural documents composed of multiple instructional statements that jointly define valid agent behavior. The challenge is not only that a skill contains many clauses, but that these clauses often form logical relations: actions may depend on prior steps, constraints may narrow permitted actions, and higherpriority clauses may override conflicting ones (Wen  \nAn agent finishes the visible task and silently leaves apostcondition unsatisfied.  \nSCENARIO  \nUser asks the agent to make a leaked clinical upload flow HIPAA-safe before a compliance call, using the file-cleanup skill.  \n\n|  |  Post-Condition \u003Cbr>B must hold after A,\u003Cbr>on every path |  |  |\n| --- | --- | --- | --- |\n|  |  | CLAUSE B · HIDDEN POSTCONDITION\u003Cbr>Delete uploaded notes, JSON artifacts, session files—on success AND error path.\u003Cbr>“The workflow is not finished until every uploaded\u003Cbr>artifact has been removed; the same cleanup runs in  the error path.”  |  |\n\n\n| ✗Agent resolves Clause A only\u003Cbr>Workflow declared done; PHI persists on disk. |  |\n| --- | --- |\n| Final Repo State\u003Cbr>[src/upload_handler.py](src/upload_handler.py)\u003Cbr>data/sessions/incident-884/\u003Cbr>note.txt\u003Cbr>summary.json | patched\u003Cbr> still on disk \u003Cbr> PHI leaked \u003Cbr> PHI leaked \u003Cbr> |\n\n\n| ✓Agent resolves Clause A ∧ Clause B\u003Cbr>Workflow truly done; no residual PHI. |  |\n| --- | --- |\n| Final Repo State\u003Cbr>[src/upload_handler.py](src/upload_handler.py)\u003Cbr>data/sessions/incident-884/\u003Cbr>~~ note~~.~~tx~~t\u003Cbr>~~ summary~~.~~json~~\u003Cbr> |  cleanup both \u003Cbr> removed \u003Cbr> empty  |\n\nExisting benchmarks: limited coverage of logical relations; lack executable, realistic test cases.  \nFigure 1: Example of a logical-relation violation ina skill-guided agent. The agent is required to delete sensitive artifacts after processing clinical notes. If the postcondition clause is missed, the agent may still return a plausible success message while leaving sensitive fileson disk, resulting in a privacy leak.  \net al., 2024 ; Zhang et al., 2025) . An agent that fo","cbCailemg3hJWvvI","https://ap.wps.com/l/cbCailemg3hJWvvI","pdf",511047,1,18,"English","en",105,"# Abstract\n# Introduction\n## SkillLogic framework\n## SLBench benchmark and evaluation\n## Results and SLGuard mitigation","[{\"question\":\"What is SkillLogic and what does it analyze in skill files?\",\"answer\":\"SkillLogic analyzes logical relations embedded in skill files and turns them into executable tests. It uses a taxonomy of eight relation types, including preconditions, constraints, and fallbacks.\"},{\"question\":\"How is SLBench constructed and what does it evaluate?\",\"answer\":\"SLBench is an 86-case executable benchmark built from high-confidence, high-impact, locally testable logical relations. It evaluates Codex and Claude Code across multiple LLM backbones for unsafe skill-following behavior.\"},{\"question\":\"What failure outcomes did the evaluation observe when agents violate logical relations?\",\"answer\":\"Violations can lead to privacy leaks, unsafe configuration changes, and incomplete cleanup. Human audits attribute failures to both agent capability gaps and low-salience skill text, while SLGuard reduces violations by 63% on targeted cases.\"}]",1784178442,45,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"slbench-evaluating-how-llm-agents-follow-logical-relations-in-skills","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/slbench-evaluating-how-llm-agents-follow-logical-relations-in-skills/82146/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is SkillLogic and what does it analyze in skill files?","Question",{"text":75,"@type":76},"SkillLogic analyzes logical relations embedded in skill files and turns them into executable tests. It uses a taxonomy of eight relation types, including preconditions, constraints, and fallbacks.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How is SLBench constructed and what does it evaluate?",{"text":80,"@type":76},"SLBench is an 86-case executable benchmark built from high-confidence, high-impact, locally testable logical relations. It evaluates Codex and Claude Code across multiple LLM backbones for unsafe skill-following behavior.",{"name":82,"@type":73,"acceptedAnswer":83},"What failure outcomes did the evaluation observe when agents violate logical relations?",{"text":84,"@type":76},"Violations can lead to privacy leaks, unsafe configuration changes, and incomplete cleanup. Human audits attribute failures to both agent capability gaps and low-salience skill text, while SLGuard reduces violations by 63% on targeted cases.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]