[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-121972-en":3,"doc-seo-121972-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},121972,4398048950312,"Violet","https://ap-avatar.wpscdn.com/avatar/400002538284de19e3c?_k=1778320343897328908",8,"Research & Report","A Large-Scale Study of Probabilistic Calibration in Neural Network Regression - Proceedings","Accurate probabilistic predictions are essential for optimal decision making, especially in regression where uncertainty must be quantified reliably. This paper performs the largest empirical study of probabilistic calibration for neural network regression to date, evaluating post-hoc recalibration, conformal prediction, and regularization-based approaches. It introduces differentiable recalibration and regularization methods and analyzes their effectiveness. Results show regularization offers a good tradeoff between calibration and sharpness, while post-hoc methods achieve superior calibration, linked to conformal finite-sample coverage. Quantile recalibration is shown to be a special case of conformal prediction, and the study is fully reproducible with common code for fair comparisons.","A Large-Scale Study of Probabilistic Calibration in Neural Network Regression  \nVictor Dheur 1 Souhaib Ben Taieb 1  \narXiv :2306 .02738v 1 [ cs .LG] 5 Jun 2023  \nAbstract  \nAccurate probabilistic predictions are essential for optimal decision making. While neural network miscalibration has been studied primarily in classification, we investigate this in the lessexplored domain of regression. We conduct the largest empirical study to date to assess the probabilistic calibration of neural networks. We also analyze the peformance of recalibration, conformal, and regularization methods to enhance probabilistic calibration. Additionally, we introduce novel differentiable recalibration and regularization methods, uncovering new insights into their effectiveness. Our findings reveal that regularization methods offer a favorable tradeoff between calibration and sharpness. Post-hoc methods exhibit superior probabilistic calibration, which we attribute to the finite-sample coverage guarantee of conformal prediction. Furthermore, we demonstrate that quantile recalibration can be considered as a specific case of conformal prediction. Our study is fully reproducible and implemented in a common code base for fair comparisons.  \n1. Introduction  \nNeural network predictions affect critical decisions in many applications, including medical diagnostics and autonomous driving (Gulshan et al., 2016; Guizilini et al., 2020) . However, effective decision making often requires accurate probabilistic predictions (Gawlikowski et al., 2021; Abdar et al., 2021) . For example, consider a probabilistic regression model that produces 90% prediction intervals. An important property would be that 90% of these prediction intervals contain the realizations.  \nFor models that output a predictive distribution, probabilis-  \n1Department of Computer Science, University of Mons, Mons, Belgium. Correspondence to: Victor Dheur \u003Cvic[tor.dheur@umons.ac.be](tor.dheur@umons.ac.be)> .  \nProceedings of the 40 th International Conference on Machine Learning, Honolulu, Hawaii, USA. PMLR 202, 2023 . Copyright 2023 by the author(s) .  \ntic calibration is an important property that states that all quantiles must be calibrated, i.e., the frequency of realizations below these quantiles must match the corresponding quantile level. Additionally, predictive distributions should be sufficiently sharp (i.e., concentrated around the realizations) and leverage the information in the inputs.  \nIn the classification setting, Guo et al. (2017) found that common neural architectures trained on image and text data were miscalibrated, sparking increased interest in neural network calibration. In a follow-up study, Minderer et al.(2021) showed that more recent neural architectures demonstrate improved calibration. However, there has been less research on calibration for neural probabilistic regression models compared to classification. Therefore, it remains uncertain whether the same results apply to the regression setting. This paper addresses this gap by conducting a comprehensive study on probabilistic calibration for regression using tabular data. We explore various calibration methods, including quantile recalibration (Kuleshov, Fenner, et al., 2018) and conformalized quantile regression (Romano, Patterson, et al., 2019) . We also consider regularization methods, which have been shown to perform well in the classification setting (Karandikar et al., 2021; Popordanoska et al., 2022; Yoon et al., 2023) .  \nWe make the following main contributions:  \n1. We conduct the largest empirical study to date on probabilistic calibration of neural regression models using 57 tabular datasets (Sections 4 and 6) . We consider multiple state-of-the-art calibration methods (Section 5), including post-hoc recalibration, conformal prediction, and regularization methods, with various scoring rules and predictive models.  \n2. Building on quantile recalibration, we propose a new differentiable calibration map u","cbCainnio1RH5ICW","https://ap.wps.com/l/cbCainnio1RH5ICW","pdf",1979726,1,23,"English","en",105,"# Introduction\n# Background\n## Probabilistic calibration\n## Model setup and notation\n# Main contributions\n# Empirical study and methods","[{\"question\":\"Why is probabilistic calibration important for neural network regression?\",\"answer\":\"It ensures that predicted probability intervals or quantiles match the true frequencies of outcomes, improving decision making under uncertainty.\"},{\"question\":\"Which methods are compared to improve probabilistic calibration?\",\"answer\":\"The study evaluates post-hoc recalibration, conformal prediction, and regularization methods, including newly proposed differentiable recalibration and regularization objectives.\"},{\"question\":\"What key findings explain why post-hoc methods can outperform others?\",\"answer\":\"Post-hoc approaches yield superior probabilistic calibration, attributed to conformal prediction’s finite-sample coverage guarantee.\"}]","A Large-Scale Study of Probabilistic Calibration in Neural Network Regression - Proceedings | PDF",1785808091,58,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"a-large-scale-study-of-probabilistic-calibration-in-neural-network-regression-proceedings","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/a-large-scale-study-of-probabilistic-calibration-in-neural-network-regression-proceedings/121972/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05","2026-08-04",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"Why is probabilistic calibration important for neural network regression?","Question",{"text":76,"@type":77},"It ensures that predicted probability intervals or quantiles match the true frequencies of outcomes, improving decision making under uncertainty.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"Which methods are compared to improve probabilistic calibration?",{"text":81,"@type":77},"The study evaluates post-hoc recalibration, conformal prediction, and regularization methods, including newly proposed differentiable recalibration and regularization objectives.",{"name":83,"@type":74,"acceptedAnswer":84},"What key findings explain why post-hoc methods can outperform others?",{"text":85,"@type":77},"Post-hoc approaches yield superior probabilistic calibration, attributed to conformal prediction’s finite-sample coverage guarantee.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":46,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":46,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]