[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83309-en":3,"doc-seo-83309-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83309,1374391974585,"Genevieve","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Hallucination Self Play Bootstrapping Reinforced Detector via Evolved Generator","Identifying faithfulness hallucinations in LLM-generated outputs is difficult because high-quality annotated data is scarce. Prior approaches often use advanced LLMs to synthesize training data, but they keep the generator static and limit iterative improvement of the detector. Hallucination Self-Play (HSP) introduces a closed-loop framework where an evolved generator produces increasingly hard-to-detect hallucinations while a detector is trained with RL from AI feedback. Experiments on RAGTruth show progressive gains for a small LLM without external supervision.","arXiv :2607 .07993v 1 [ cs .CL] 8 Jul 2026  \nHallucination Self-Play: Bootstrapping Reinforced Detector via Evolved Generator  \nShiping Yang1,2 ∗ Shining Liang2 Weihao Liu3 Wenbiao Ding2 Linjun Shou2 Lu Cheng3 Angel X. Chang1  \n1Simon Fraser University 2Microsoft 3University of Illinois at Chicago  \nAbstract  \nIdentifying faithfulness hallucinations in LLM-generated outputs remains challenging due to the scarcity of high-quality annotated data. Recent work relies on advanced LLMs to synthesize training data, including rationales, labels, and hallucinated claims. However, these methods treat the generator as a static component, limiting iterative improvement of the detector. To address this limitation, we introduce Hallucination SelfPlay (HSP), a novel framework that enables the detector to bootstrap with an evolved generator. HSP involves two roles initialized from the same base model, a detector that assesses the faithfulness of model outputs, and a generator that produces increasingly hard-to-detect hallucinated responses. Specifically, the detector is first fine-tuned on human-labeled data and then employed as a reward model to train the generator via reinforcement learning from AI feedback (RLAIF) . In turn, the evolved generator synthesizes hallucination data to further optimize the detector through rule-based reinforcement learning. Experiments on RAGTruth benchmark and two model families demonstrate that the proposed framework can progressively enhance a small LLM to match or even outperform advanced LLMs without external supervision. Our code is available at [https://anonymous.4open.science/r/Hallucination-Self-Play-50B5](https://anonymous.4open.science/r/Hallucination-Self-Play-50B5) .  \n1 Introduction  \nDespite the remarkable capabilities of large language models (LLMs) across diverse domains (Li et al., 2025; Yu et al., 2025b; Zhang et al., 2025; Jiang et al., 2026), they remain prone to hallucinate when handling long-tail knowledge or outdated information. Retrievalaugmented generation (RAG) has emerged as an effective paradigm to improve the factuality of model responses by grounding them in retrieved documents (Arslan et al., 2024; Yang et al., 2025b) . However, even with RAG, LLMs still suffer from faithfulness hallucinations, i.e., generating claims that are contradictory to or unsupported by the provided context (Yang et al., 2023; Huang et al., 2025; Jiang & Ferraro, 2026) . Therefore, detecting such hallucinations is critical for providing trustworthy LLM services.  \nPrior work leverages advanced LLMs to determine whether a model response contains hallucinations (Dhuliawala et al., 2024; Jacovi et al., 2025; Seo et al., 2025) . While these methods achieve impressive performance, they are impractical for real-world application due to high inference cost and latency. This has motivated the development of lightweight and specialized detectors for efficient hallucination detection. However, the high cost and scarcity of human annotation limit further performance scaling of detectors. To address this, recent studies directly synthesize hallucinated claims using tailored generation pipelines (Cao et al., 2023; Tang et al., 2024; Tan et al., 2024; Lei et al., 2025) . A key limitation of such approaches is that the hallucination generators are typically static, lacking adaptivity to the evolving capabilities of the detector. As a result, detectors quickly reach a performance plateau, as the synthetic hallucinations become too easy to provide effective training signals for further improvement.  \n∗ Work done during an internship at Microsoft  \nTo overcome this limitation, we propose Hallucination Self-Play (HSP), a closed-loop interaction between two roles: a generator and a detector, both initialized from the same base model. The generator is optimized to produce diverse and challenging hallucinations based on the detector’s feedback, while the detector is trained via RLVR on the resulting synthetic data. This intera","cbCaioe0JAf8IjGT","https://ap.wps.com/l/cbCaioe0JAf8IjGT","pdf",325081,1,16,"English","en",105,"# Introduction\n# Related Work\n## Hallucination Detection","[{\"question\":\"Why is faithfulness hallucination detection challenging for LLM outputs?\",\"answer\":\"It is challenging because high-quality annotated data is limited, making it hard to train reliable detectors.\"},{\"question\":\"How does HSP improve a hallucination detector iteratively?\",\"answer\":\"HSP uses a self-play loop where a detector provides feedback as a reward model to train an evolved generator, which then synthesizes harder hallucination data to further optimize the detector.\"},{\"question\":\"What evidence is used to prevent reward hacking in self-play training?\",\"answer\":\"HSP uses ground truth labels from a QA dataset as a proxy for verification and adds additional safeguards to reduce shortcut exploitation by the generator.\"}]",1784186660,40,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"hallucination-self-play-bootstrapping-reinforced-detector-via-evolved-generator","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/hallucination-self-play-bootstrapping-reinforced-detector-via-evolved-generator/83309/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"Why is faithfulness hallucination detection challenging for LLM outputs?","Question",{"text":74,"@type":75},"It is challenging because high-quality annotated data is limited, making it hard to train reliable detectors.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does HSP improve a hallucination detector iteratively?",{"text":79,"@type":75},"HSP uses a self-play loop where a detector provides feedback as a reward model to train an evolved generator, which then synthesizes harder hallucination data to further optimize the detector.",{"name":81,"@type":72,"acceptedAnswer":82},"What evidence is used to prevent reward hacking in self-play training?",{"text":83,"@type":75},"HSP uses ground truth labels from a QA dataset as a proxy for verification and adds additional safeguards to reduce shortcut exploitation by the generator.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,118,121,126,129,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":28,"slug":117},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":119,"slug":120},30,"research-report",{"id":122,"doc_module":4,"doc_module_name":45,"category_name":123,"show_sort_weight":124,"slug":125},9,"Religion & Spirituality",20,"religion-spirituality",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":124,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]