[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-86046-en":3,"doc-seo-86046-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},86046,1099514067438,"River Wang","https://ap-avatar.wpscdn.com/avatar/100002539ee87300030?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780474512215547542",8,"Research & Report","Auditing Belief-Conditioned LLM Agents in Hidden-Information Social Deduction Games","Evaluating LLM agents in hidden-information multi-agent settings is difficult because final outcomes have high variance and rarely explain why an agent acted as it did. This work studies a 9-player Werewolf environment with strict code-level information isolation and introduces an auditable framework that maintains an external belief state over hidden roles, logs belief updates and belief–action deviations as structured evidence, and enables a defensive offline improvement loop for reviewing failures before any strategy change. Across 1,080 frozen games, active-belief agents achieve substantially better good-side outcomes. In a 200-seed paired A0/A1 comparison, the good-side win rate increases from 0.205 to 0.390 while reducing irreversible witch-poison errors. The mechanism is reported as unresolved, but the audit framework itself is shown to make effects measurable and diagnoses reproducible.","arXiv :2607 . 10814v1 [ cs .MA] 12 Jul 2026  \nAuditing Belief-Conditioned LLM Agents in Hidden-Information Social Deduction Games  \nYuan Gao∗ Jiangyi Yang Yao Zhao Yichi Zhang  \nUniversity of Melbourne  \nAbstract  \nEvaluating LLM agents in hidden-information multi-agent settings is hard: final outcomes are high-variance and rarely reveal why an agent decided as it did. We study this in a 9-player Werewolf environment where agents act under strict, code-level information isolation, and we build an auditable framework that maintains an external belief state over hidden roles, logs belief updates and belief–action deviations as structured evidence, and supports a defensive offline improvement loop that reviews bad cases before any strategy change.  \nAcross 1,080 frozen games spanning belief-disabled, active-belief, kernel-ablation, camprestricted, consumption-policy, and high-load arms, and including a seed-paired A0/A1 comparison, the active-belief condition is associated with substantially better good-side outcomes:  \nin the 200-seed A0/A1 comparison the good-side win rate rises from 0.205 to 0.390 (paired McNemar χ2 = 16 .4 , p \u003C 0.001), with fewer irreversible witch-poison errors. We do not, however, attribute this shift to belief content. Direct action–belief consistency is low (≈ 0 .21), and giving belief only to the werewolves helps the good side more than giving it only to the good side, which argues against a simple holder-benefit account; we therefore report the effect as an association and treat its mechanism as unresolved. The contribution is the audit framework itself: it makes the effect measurable, exposes low direct action–belief consistency, rejects an unreliable forced-consumption intervention with evidence, and separates strategy effects from load confounds. We accordingly position external belief in high-noise hidden-information games primarily as an auditable cognitive baseline that also carries decision-relevant signal, turning opaque agent behavior into replayable evidence for safer, controlled iteration.  \n1 Introduction  \nLarge language model (LLM) agents are increasingly used as autonomous decision makers in interactive environments. Many existing evaluations, however, focus on tasks where the full problem state is either directly observable or can be reconstructed from the prompt. Hidden-information multi-agent games pose a different challenge. In these environments, agents must act under partial observability, reason about other agents’ private information, interpret potentially deceptive communication, and make decisions whose quality may only become clear after the game ends. Final win rate is therefore a noisy and often insufficient signal: a good decision may lose because of downstream teammate errors, while a poor decision may win because of opponent mistakes.  \nSocial deduction games such as Werewolf provide a compact testbed for this problem. Players have asymmetric private roles, public discussion is cheap and potentially deceptive, and the game outcome depends on a sequence of speech, voting, and role-specific night actions. These properties make the setting useful for studying LLM agents under hidden state, adversarial interaction, and  \n∗ Corresponding author: [yuan.gao.2@student.unimelb.edu.au](yuan.gao.2@student.unimelb.edu.au)  \nhigh-variance feedback. At the same time, they expose a core limitation of black-box agent evaluation. If an agent votes for the wrong player, poisons a teammate, withholds a useful claim, or ignores a strong public signal, the final outcome alone does not explain whether the failure came from missing evidence, poor belief tracking, prompt-induced behavior, load-induced LLM failures, or a reasonable strategic override.  \nThis paper studies how to make such agent behavior auditable. We implement a 9-player Werewolf environment in which LLM agents operate under strict information isolation: each agent receives only the public events and private observations availabl","cbCair943nHIaz1y","https://ap.wps.com/l/cbCair943nHIaz1y","pdf",479457,1,29,"English","en",105,"# Introduction\n## Auditable belief-state framework\n## Deviation-guided diagnosis\n## Defensive offline improvement loop\n# Evaluation","[{\"question\":\"Why is auditing LLM agents in hidden-information social deduction games challenging?\",\"answer\":\"Final win rates are noisy and often do not reveal whether decisions failed due to missing evidence, belief-tracking issues, prompt effects, load-induced LLM failures, or strategic overrides by the agent.\"},{\"question\":\"What does the proposed framework record to enable auditing?\",\"answer\":\"It records structured game events, agent decision traces, belief snapshots, and evaluation reports, supporting post-hoc replay and diagnosis. It also maintains an external belief layer over hidden roles.\"},{\"question\":\"What is the reported effect of using active-belief conditions?\",\"answer\":\"Across 1,080 frozen games, active-belief agents show substantially better good-side outcomes. In a 200-seed A0/A1 comparison, the good-side win rate rises from 0.205 to 0.390 and irreversible witch-poison errors are fewer, while the mechanism behind the shift is treated as unresolved.\"}]",1784208059,73,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"auditing-belief-conditioned-llm-agents-in-hidden-information-social-deduction-games","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/auditing-belief-conditioned-llm-agents-in-hidden-information-social-deduction-games/86046/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is auditing LLM agents in hidden-information social deduction games challenging?","Question",{"text":75,"@type":76},"Final win rates are noisy and often do not reveal whether decisions failed due to missing evidence, belief-tracking issues, prompt effects, load-induced LLM failures, or strategic overrides by the agent.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What does the proposed framework record to enable auditing?",{"text":80,"@type":76},"It records structured game events, agent decision traces, belief snapshots, and evaluation reports, supporting post-hoc replay and diagnosis. It also maintains an external belief layer over hidden roles.",{"name":82,"@type":73,"acceptedAnswer":83},"What is the reported effect of using active-belief conditions?",{"text":84,"@type":76},"Across 1,080 frozen games, active-belief agents show substantially better good-side outcomes. In a 200-seed A0/A1 comparison, the good-side win rate rises from 0.205 to 0.390 and irreversible witch-poison errors are fewer, while the mechanism behind the shift is treated as unresolved.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]