[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85708-en":3,"doc-seo-85708-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85708,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","Hallucination Detector: A Hybrid LLM and Semantic Scholar Tool Calling","Large language models used in scientific writing introduce a critical failure mode: fabricated references, including made-up authors, bogus DOIs, incorrect identifiers, and citations that mix parts of real papers. Audits show such errors have reached peer-reviewed publications, making automated verification essential at publication scale. The work presents and evaluates the AtomGPT reference checker, combining LLM-based bibliographic field extraction with structured retrieval from Semantic Scholar. It benchmarks against confirmed NeurIPS 2025 hallucinated citations and analyzes when detection succeeds or fails.","arXiv :2607 .09774v 1 [ cs .DL] 7 Jul 2026  \nHallucination Detector: A hybrid LLM and Semantic Scholar tool calling for detecting hallucination in scientific literature on  \n[AtomGPT.org](AtomGPT.org)  \nHarichandana Neralla 1 , Jaehyung Lee2 , Aldo H. Romero4 , Kamal Choudhary2 ,3 ,∗  \n1 Hopkins Extreme Materials Institute, Johns Hopkins University, Baltimore, MD 21218, USA  \n2 Department of Materials Science and Engineering, Johns Hopkins University, Baltimore, MD 21218, USA  \n3 Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA  \n4 Department of Physics and Astronomy, West Virginia University, Morgantown, WV 26506, USA  \n∗ Corresponding author: [kchoudh2@jhu.edu](kchoudh2@jhu.edu)  \nJuly 14, 2026  \nAbstract  \nLarge language models are now commonly used as partners in scientific writing, and this shift has brought a subtler type of failure: made-up references. Fabricated authors, bogus DOIs, wrongly assigned identifiers, and citations that merge elements from multiple genuine articles are now being inserted into manuscripts at a volume that traditional peer review was never meant to handle. Recent audits reveal that such references have already slipped through the review process and made their way into the published literature, including leading journals and conferences. Automated verification that operates at the speed and scale of modern content production has therefore become a necessary safeguard rather than a convenience. This work presents and evaluates the AtomGPT reference checker ([https://atomgpt.org/hallucination_detector](https://atomgpt.org/hallucination_detector)), an open, web-accessible tool that verifies citations against the scholarly literature by combining large-language-model field extraction with structured retrieval from Semantic Scholar. For each reference, the tool extracts the bibliographic fields, retrieves the closest matching real papers, and scores the agreement across title, authorship, and venue to produce a graded judgment of whether a citation is trustworthy, partially supported, or likely fabricated. We benchmark the tool against an externally curated set of confirmed hallucinated citations from accepted NeurIPS 2025 papers and find that it reliably flags the great majority of them. A per-field analysis explains the behavior: fabricated references are most commonly uncovered when the listed authors do not correspond to any actual publication, even if the title appears credible, while similarity in titles alone is a weak indicator, and similarity in venues is an even weaker one. We characterize the small number of missed cases in which a fabricated citation closely resembles a single real paper and position the tool as a lightweight, drop-in component for editorial pipelines, submission systems, and review platforms, where catching fabricated references before publication is increasingly essential to preserving research integrity.  \n1 Introduction  \nCitations are fundamental to the integrity of the scientific record. Each reference signals that a verifiable source exists and supports the statement it accompanies. When a cited source does not exist or cannot be verified, readers, reviewers, and future researchers lose the ability to trace the origin of the information and to evaluate the evidence independently.  \nLarge language models (LLMs) are now routinely used as partners in academic writing, and this shift has introduced a significant new failure mode: reference hallucination, in which an LLM generates citations that appear authentic, often including plausible authors, journal names, publication years, and other bibliographic details, but do not correspond to any real publication. Such fabricated references can be difficult to detect because they closely resemble legitimate citations. Recent studies have begun to quantify how prevalent this behavior is. Walters and Wilder found that a substantial fraction of the bibliographic citations ge","cbCaick7tVbgwGpW","https://ap.wps.com/l/cbCaick7tVbgwGpW","pdf",719408,1,12,"English","en",105,"# Introduction\n## Motivation and problem of reference hallucination\n## Prior work quantifying fabrication rates\n## Evidence that fabricated citations enter published literature\n## Benchmark and paper contributions","[{\"question\":\"What problem does the AtomGPT reference checker address?\",\"answer\":\"It detects reference hallucination in scientific manuscripts, where LLM-generated citations look plausible but do not correspond to real publications.\"},{\"question\":\"How does the tool verify a citation?\",\"answer\":\"For each reference, it extracts bibliographic fields with a large language model, retrieves closest matches from Semantic Scholar, and scores agreement across title, authorship, and venue.\"},{\"question\":\"What did the evaluation show about the tool’s effectiveness?\",\"answer\":\"Benchmarked against an externally curated set of confirmed hallucinated citations from accepted NeurIPS 2025 papers, it reliably flags the great majority of them, with per-field analysis explaining common failure patterns.\"}]",1784205727,30,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"hallucination-detector-a-hybrid-llm-and-semantic-scholar-tool-calling","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/hallucination-detector-a-hybrid-llm-and-semantic-scholar-tool-calling/85708/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does the AtomGPT reference checker address?","Question",{"text":74,"@type":75},"It detects reference hallucination in scientific manuscripts, where LLM-generated citations look plausible but do not correspond to real publications.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does the tool verify a citation?",{"text":79,"@type":75},"For each reference, it extracts bibliographic fields with a large language model, retrieves closest matches from Semantic Scholar, and scores agreement across title, authorship, and venue.",{"name":81,"@type":72,"acceptedAnswer":82},"What did the evaluation show about the tool’s effectiveness?",{"text":83,"@type":75},"Benchmarked against an externally curated set of confirmed hallucinated citations from accepted NeurIPS 2025 papers, it reliably flags the great majority of them, with per-field analysis explaining common failure patterns.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,121,126,129,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":28,"slug":120},"research-report",{"id":122,"doc_module":4,"doc_module_name":45,"category_name":123,"show_sort_weight":124,"slug":125},9,"Religion & Spirituality",20,"religion-spirituality",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":124,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]