[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84731-en":3,"doc-seo-84731-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84731,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","Uncertainty-Aware Abstention in Large Language Models with Provable Alignment Guarantees","Large language models (LLMs) deployed for question answering can still produce hallucinated or misaligned outputs, especially when reliability matters and a wrong answer is worse than no answer. Uncertainty quantification enables selective answering by accepting responses that appear reliable and abstaining otherwise, but heuristic uncertainty scores and naive thresholding lack statistical risk guarantees. This work introduces CIC, a confidence-interval calibration framework that converts uncertainty scores into risk-controlled abstention rules with provable error-rate control.","arXiv :2607 .04430v 1 [ cs .CL] 5 Jul 2026  \nUncertainty-Aware Abstention in Large Language Models with Provable Alignment Guarantees  \nSijin Dong 1* and Hiroyuki Shinnou 1  \n1 Ibaraki University, Japan.  \n*Corresponding author(s). E-mail(s): [dong8529278@gmail.com](dong8529278@gmail.com) ;  \nAbstract  \nLarge language models (LLMs) are increasingly used in question answering (QA) systems, but they can still produce hallucinated or misaligned responses without reliable confidence estimates. Uncertainty quantification (UQ) enables selective answering, where a system responds only when its prediction appears reliable and abstains otherwise. However, uncertainty scores alone are often heuristic, and thresholding them does not provide statistical guarantees on the error rate among accepted answers. We propose CIC, a confidence-interval-based calibration framework that converts arbitrary uncertainty scores into risk-controlled selective answering rules. Using a held-out calibration set, CIC assigns each generated response an uncertainty score and a binary error label based on an application-specific alignment criterion. For each candidate threshold, CIC estimates the error rate among accepted answers and constructs a high-probability upper confidence bound using Hoeffding-style or Clopper–Pearson intervals. It then selects the threshold with the highest answering rate whose upper bound remains below a user-specified risk level α . Under exchangeability, CIC guarantees with probability at least 1 − δ that the selected non-null threshold controls the accepted-answer error rate at level α . Experiments on closed-ended and openended QA benchmarks across seven LLMs and multiple uncertainty estimators show that CIC achieves valid risk control while maintaining strong answering efficiency. These results demonstrate that CIC provides a practical and statistically grounded mechanism for reliable LLM deployment in QA workflows.  \nKeywords: large language models, question answering, selective answering, uncertainty quantification, upper confidence bound  \n1  \n1 Introduction  \nLarge language models (LLMs) have achieved remarkable progress in natural language understanding and generation [1, 2], and are increasingly deployed as question answering (QA) systems, conversational assistants [3], and decision-support tools [4] . Despite their strong empirical performance, LLMs can still generate hallucinated, unsupported, or semantically misaligned responses [5–10] . This issue is particularly problematic in reliability-sensitive applications, where an incorrect answer may be more harmful than no answer at all. Therefore, beyond improving the average accuracy of LLMs, it is crucial to develop principled mechanisms that determine when a model output should be trusted and when the system should abstain [8, 11–17] .  \nUncertainty quantification (UQ) provides a natural route toward this goal [18] . Given a query and a generated response, an uncertainty estimator assigns a score intended to reflect the reliability of the output. Such scores can be used for selective answering: the system accepts a response when its uncertainty is sufficiently low and abstains otherwise. A wide range of uncertainty signals have been proposed for LLMs, including entropy-based scores, sampling-based disagreement measures, and semantic-consistency metrics [19–22] . However, these scores are often heuristic. They may correlate with correctness on average, but they do not perfectly separate correct from incorrect answers. As a result, choosing an uncertainty threshold by intuition or empirical accuracy alone does not provide a statistical guarantee on the error rate among the answers that the system actually returns [23–25] .  \nThis limitation motivates the central question of this work: given an arbitrary uncertainty score for an LLM output, can we calibrate a test-time answering rule that accepts as many responses as possible while provably controlling the risk among accepted ","cbCaiao15h0uxnqD","https://ap.wps.com/l/cbCaiao15h0uxnqD","pdf",796873,1,22,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"Why do LLM QA systems need abstention mechanisms?\",\"answer\":\"LLMs can generate hallucinated, unsupported, or semantically misaligned responses. In reliability-sensitive settings, incorrect answers may be more harmful than abstaining.\"},{\"question\":\"How does CIC turn uncertainty scores into reliable selective answering?\",\"answer\":\"CIC uses a held-out calibration set to assign each response an uncertainty score and a binary alignment label. It scans candidate thresholds, estimates the error rate among accepted responses, and uses upper confidence bounds to ensure risk control.\"},{\"question\":\"What guarantee does CIC provide for the accepted-answer error rate?\",\"answer\":\"Under exchangeability, CIC guarantees with probability at least 1 − δ that the selected non-null threshold controls the accepted-answer error rate at the user-specified risk level α.\"}]",1784197908,55,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"uncertainty-aware-abstention-in-large-language-models-with-provable-alignment-guarantees","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/uncertainty-aware-abstention-in-large-language-models-with-provable-alignment-guarantees/84731/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do LLM QA systems need abstention mechanisms?","Question",{"text":75,"@type":76},"LLMs can generate hallucinated, unsupported, or semantically misaligned responses. In reliability-sensitive settings, incorrect answers may be more harmful than abstaining.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does CIC turn uncertainty scores into reliable selective answering?",{"text":80,"@type":76},"CIC uses a held-out calibration set to assign each response an uncertainty score and a binary alignment label. It scans candidate thresholds, estimates the error rate among accepted responses, and uses upper confidence bounds to ensure risk control.",{"name":82,"@type":73,"acceptedAnswer":83},"What guarantee does CIC provide for the accepted-answer error rate?",{"text":84,"@type":76},"Under exchangeability, CIC guarantees with probability at least 1 − δ that the selected non-null threshold controls the accepted-answer error rate at the user-specified risk level α.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]