[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85015-en":3,"doc-seo-85015-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85015,13056703019404,"Miles","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","Future Confidence Distillation in Large Language Models","Reliable confidence estimation is crucial for deploying large language models in confidence-aware systems where decisions like retrieval, tool use, and adaptive computation require accurate reliability estimates. Existing methods treat confidence as a property of finished answers, ignoring how it changes during answering. This work studies temporal confidence by contrasting pre-solution Feeling-of-Knowing (FOK) and post-solution Judgement-of-Learning (JOL) across frontier and open-source models. Post-solution confidence is better calibrated and more discriminative; future confidence distillation trains pre-solution predictors using teacher signals from post-solution correctness probes, achieving much of the calibration gains with low inference cost and strong cross-dataset transfer.","Future Confidence Distillation in Large Language Models  \nSahil Kale 1  \n1University of California, Los Angeles  \nLos Angeles, USA  \n[sahilrkale@cs.ucla.edu](sahilrkale@cs.ucla.edu)  \narXiv :2607 .07626v 1 [ cs .CL] 8 Jul 2026  \nAbstract  \nReliable confidence estimation is essential for deploying large language models (LLMs) in confidence-aware systems, where downstream decisions such as retrieval, tool use, and adaptive computation depend on accurately estimating answer reliability. Existing approaches, however, largely treat confidence as a property of completed responses, overlooking how confidence-related information evolves throughout the answering process. In this work, we investigate confidence from a temporal perspective by comparing pre-solution Feelingof-Knowing (FOK) and post-solution Judgement-of-Learning (JOL) confidence estimates across frontier and open-source LLMs. We show that post-solution confidence is consistently better calibrated and more discriminative than presolution confidence, while linear probes trained on hidden representations recover substantially richer confidence-related information than models explicitly verbalise. Building on this observation, we introduce future confidence distillation, which trains predictors operating on pre-solution hidden representations using teacher confidence estimates produced by post-solution correctness probes. Despite requiring only presolution representations for inference, distilled predictors recover much of the calibration improvement achieved by postsolution confidence, remain highly sample efficient, and transfer across datasets within the same domain. Together, our findings demonstrate that confidence-related information evolves throughout the answering process and can be anticipated before answer generation is complete, enabling significantly more reliable yet low-cost confidence estimation.  \n1 Introduction  \nLarge language models (LLMs) are increasingly deployed in high-stakes settings including question answering, reasoning systems, retrieval-augmented generation (RAG), autonomous agents, and decision-support pipelines (Li et al. 2024; Cheng et al. 2025) . In such systems, models or downstream components often rely on internal confidence estimates to determine whether additional retrieval should be triggered, external tools should be invoked, more computation should be allocated, or a response should be deferred altogether (Li et al. 2025) . Such a capability is often framedas self-knowledge (Kale and Nadadur 2025) or metacognition  \nCopyright © 2027, Association for the Advancement of Artificial Intelligence ([www.aaai.org](www.aaai.org)). All rights reserved.  \n(Ma et al. 2025) . Consequently, the quality of a model’s selfknowledge has become a critical factor of reliable, effective, and cost-efficient confidence-aware decision policies (Kale and Dhami 2025) .  \nA growing body of work studies self-knowledge via confidence estimation and calibration in LLMs, typically through verbalised confidence scores, token probabilities, or post-hoc calibration techniques (Kale 2025) . Despite recent progress, confidence signals remain imperfectly aligned with actual correctness, limiting their usefulness for downstream decision-making (Ren et al. 2026; Kale and Dhami 2025) . More importantly, regardless of the estimation method, confidence is almost always treated as a property of a completed answer. We instead investigate whether confidence-related information encoded in hidden representations evolves throughout the answering process and can be anticipated before answer generation is complete.  \nIn this work, we study confidence from a temporal perspective. Inspired by classical theories of meta-memory (Nelson 1990; Koriat 1997), we distinguish between two stages of self-assessment: a pre-solution Feeling-of-Knowing (FOK) signal and a post-solution confidence signal analogous toa Judgement-of-Learning (JOL) . The former reflects the model’s estimate of future su","cbCaiddEU9Cnloaq","https://ap.wps.com/l/cbCaiddEU9Cnloaq","pdf",8798550,1,16,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"为什么需要对大语言模型进行可靠的置信度估计？\",\"answer\":\"在检索、工具调用、自适应计算等依赖可靠性判断的系统中，准确估计答案可靠性是关键。置信度用于决定是否触发额外检索、调用外部工具、增加计算或推迟回复。\"},{\"question\":\"文中如何区分 FOK 与 JOL 两类置信度信号？\",\"answer\":\"FOK（Feeling-of-Knowing）表示在开始回答之前对未来成功的预估；JOL（Judgement-of-Learning）表示在产生解答之后的置信度评估。该划分用于从时间角度分析置信度信息如何随回答过程演化。\"},{\"question\":\"“未来置信度蒸馏”具体解决了什么问题？\",\"answer\":\"它解决了仅用回答前（pre-solution）输入进行置信度预测时能力不足的问题。通过用基于回答后正确性的教师探针生成的置信度信号，训练只依赖回答前隐藏表示的预测器，从而在低推理成本下恢复大量校准改进，并在同一领域跨数据集迁移。\"}]",1784200280,40,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"future-confidence-distillation-in-large-language-models","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/future-confidence-distillation-in-large-language-models/85015/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"为什么需要对大语言模型进行可靠的置信度估计？","Question",{"text":75,"@type":76},"在检索、工具调用、自适应计算等依赖可靠性判断的系统中，准确估计答案可靠性是关键。置信度用于决定是否触发额外检索、调用外部工具、增加计算或推迟回复。","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"文中如何区分 FOK 与 JOL 两类置信度信号？",{"text":80,"@type":76},"FOK（Feeling-of-Knowing）表示在开始回答之前对未来成功的预估；JOL（Judgement-of-Learning）表示在产生解答之后的置信度评估。该划分用于从时间角度分析置信度信息如何随回答过程演化。",{"name":82,"@type":73,"acceptedAnswer":83},"“未来置信度蒸馏”具体解决了什么问题？",{"text":84,"@type":76},"它解决了仅用回答前（pre-solution）输入进行置信度预测时能力不足的问题。通过用基于回答后正确性的教师探针生成的置信度信号，训练只依赖回答前隐藏表示的预测器，从而在低推理成本下恢复大量校准改进，并在同一领域跨数据集迁移。","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":28,"slug":118},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]