[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82472-en":3,"doc-seo-82472-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82472,1099513958607,"Jiven","https://ap-avatar.wpscdn.com/avatar/100002390cf8733938c?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778829742770036399",8,"Research & Report","Verifiable Rewards for Calibrated Probabilistic Forecasting","Verifiable Rewards for Calibrated Probabilistic Forecasting studies reinforcement learning for calibrated probabilistic forecasting using proper scoring rules like the Brier score. Practical RLVR degrades calibration due to label noise from stochastic outcomes and corruption of chain-of-thought reasoning. Focusing on aleatoric forecasting, the paper uses NFL win probabilities as a testbed and introduces a verifiable, label-free, state-conditioned empirical win-rate reward estimated from past outcomes. Direct prediction or a gradient mask prevents updating the reasoning while matching betting-market calibration and Brier-score performance.","arXiv :2607 .00164v1 [ cs .LG] 30 Jun 2026  \nVerifiable Rewards for Calibrated Probabilistic Forecasting  \nSadanand Singh Allam Reddy Manan Chopra  \nCascade Research  \nAbstract  \nReinforcement learning with verifiable rewards can in principle train calibrated probabilistic forecasters, since a proper scoring rule such as the Brier score is computed from outcomes alone and is minimized in expectation by the true probability. In practice it degrades calibration, and existing remedies address epistemic uncertainty, where a model’s confidence accompanies a verifiably correct or incorrect answer. We study aleatoric forecasting, where the forecast itself is the output and the label is one stochastic outcome, taking NFL in-game win probability as a testbed with the betting market as a reference. Rewarding the realized per-play outcome fails, because the single outcome is a noisy target and the policy gradient corrupts the chain of thought. We introduce a verifiable, label-free reward, a state-conditioned empirical win rate estimated from past outcomes, that removes the label noise, and we keep the gradient off the reasoning, by direct prediction or a gradient mask, so it cannot be corrupted. Trained with this reward alone, without human labels or supervised fine-tuning, a 7B model reaches the calibration of the betting market by direct prediction and is better calibrated than a zero-shot frontier model. That frontier model and a tabular estimator reach the same Brier score as this model, identifying the market’s small remaining edge as live in-game information beyond their shared inputs. Masking the gradient, rather than dropping the chain of thought, preserves reasoning from which the forecast follows, which ordinary chain-of-thought training corrupts.  \n1 Introduction  \nReinforcement learning with verifiable rewards (RLVR) post-trains a language model against a reward computed from observed outcomes [1, 2] . Calibrated probabilistic prediction is a natural target for it: a proper scoring rule such as the Brier score is computable from outcomes alone and is minimized in expectation by the true probability [3, 4], so optimizing it should produce forecasts whose probabilities match observed frequencies.  \nIn practice, reinforcement learning does the opposite, degrading calibration and leaving models overconfident, whether the reward is human feedback [5] or a verifiable correctness signal [6, 7] . The work that corrects this addresses epistemic uncertainty, where the model answers a question that has a correct answer and the goal is a calibrated confidence that the answer is right [6, 8] .  \nA second kind of uncertainty is left untreated. It isaleatoric: the output is itself a probability, and the label is a single realized outcome of a stochastic event, with no answer that can be called correct. NFL in-game win probability is a clear instance. At any point in a game the forecaster states the probability that the team in possession wins; the realized result is one Bernoulli draw from a rate that no model observes directly; and the betting market provides a strong, independent estimate of that rate. Whether RLVR can train a calibrated forecaster in this regime, and what makes it fail, has not been studied.  \nReinforcement learning against the per-play Brierscore decalibrates such a forecaster through two mechanisms. The label for a play is a single realized outcome, one draw from the rate being estimated, so a reward  \nscored against it has high variance and pulls the policy toward whichever result occurred. When the model reasons in language before answering, a second problem appears: optimizing the final probability rewrites the reasoning into incoherent arguments. We remove the variance by rewarding a state-conditioned empirical win rate estimated from past outcomes, a verifiable and label-free target, and we remove the corruption by keeping the policy gradient off the reasoning, through direct prediction or a gradient mask over the","cbCaiivYRh3Vx8Z6","https://ap.wps.com/l/cbCaiivYRh3Vx8Z6","pdf",734743,1,14,"English","en",105,"# Introduction\n## Problem: calibration degradation in RLVR\n## Aleatoric forecasting and label noise\n## Proposed verifiable, label-free reward and training method","[{\"question\":\"Why does reinforcement learning with verifiable rewards often degrade probability calibration?\",\"answer\":\"Because the reward signal drives the policy using observed outcomes that do not preserve calibration, often leaving models overconfident even when reward correctness is verifiable.\"},{\"question\":\"What is the main challenge in aleatoric forecasting addressed in the paper?\",\"answer\":\"The label is a single realized outcome drawn from an unknown stochastic rate, so there is no inherently “correct” answer; the resulting label noise can inflate variance and mislead per-play training.\"},{\"question\":\"How does the proposed method form a verifiable, label-free reward?\",\"answer\":\"It computes a state-conditioned empirical win rate estimated from past outcomes, replacing the single realized outcome with a verifiable target and reducing label noise.\"}]",1784180723,35,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"verifiable-rewards-for-calibrated-probabilistic-forecasting","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/verifiable-rewards-for-calibrated-probabilistic-forecasting/82472/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why does reinforcement learning with verifiable rewards often degrade probability calibration?","Question",{"text":75,"@type":76},"Because the reward signal drives the policy using observed outcomes that do not preserve calibration, often leaving models overconfident even when reward correctness is verifiable.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is the main challenge in aleatoric forecasting addressed in the paper?",{"text":80,"@type":76},"The label is a single realized outcome drawn from an unknown stochastic rate, so there is no inherently “correct” answer; the resulting label noise can inflate variance and mislead per-play training.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the proposed method form a verifiable, label-free reward?",{"text":84,"@type":76},"It computes a state-conditioned empirical win rate estimated from past outcomes, replacing the single realized outcome with a verifiable target and reducing label noise.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]