[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117601-en":3,"doc-seo-117601-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117601,962075114765,"Quinn","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Probabilistic Calibration by Design for Neural Network Regression","Generating calibrated and sharp predictive distributions for regression is crucial for reliable decisions in real-world applications. Neural networks often suffer from probabilistic miscalibration, motivating post-hoc correction and training-time regularization strategies. Post-hoc approaches typically improve calibration but remain independent of model training. This work proposes Quantile Recalibration Training, an end-to-end procedure that integrates post-hoc calibration into training without extra parameters. Experiments on 57 tabular datasets show improved predictive accuracy while preserving calibration, supported by ablation and hyperparameter analyses.","Probabilistic Calibration by Design for Neural Network Regression  \nVictor Dheur Souhaib Ben Taieb  \nDepartment of Computer Science,  \nUniversity of Mons, Belgium  \nAbstract  \nGenerating calibrated and sharp neural network predictive distributions for regression problems is essential for optimal decisionmaking in many real-world applications. To address the miscalibration issue of neural networks, various methods have been proposed to improve calibration, including post-hoc methods that adjust predictions after training and regularization methods that act during training. While post-hoc methods have shown better improvement in calibration compared to regularization methods, the post-hoc step is completely independent of model training. We introduce a novel end-to-end model training procedure called Quantile Recalibration Training, integrating post-hoc calibration directly into the training process without additional parameters. We also present a unified algorithm that includes our method and other post-hoc and regularization methods, as particular cases. We demonstrate the performance of our method in a large-scale experiment involving 57 tabular regression datasets, showcasing improved predictive accuracy while maintaining calibration. We also conduct an ablation study to evaluate the significance of different components within our proposed method, as well as an in-depth analysis of the impact of the base model and different hyperparameters on predictive accuracy.  \nProceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS) 2024, Valencia, Spain. PMLR: Volume 238 . Copyright 2024 by the author(s) .  \n1 INTRODUCTION  \nCritical decisions depend on the predictions made by neural networks in many applications such as medical diagnostics and autonomous driving (Begoli et al., 2019; Michelmore et al., 2018) . To make decisions effectively, it is often crucial to quantify predictive uncertainty accurately (Gawlikowski et al., 2021; Abdar et al., 2021) . Yet, neural networks might exhibit miscalibration (Guo et al., 2017) .  \nWe focus on regression models that output a predictive distribution. Central to our study, probabilistic calibration 1 (Gneiting et al. , 2007) is an important property that states that all quantiles must be calibrated. This implies that the predicted 90% quantiles should exceed 90% of the corresponding realizations.  \nSeveral methods have been proposed to improve probabilistic calibration and they can be divided into two main categories. Post-hoc methods such as Quantile Recalibration (Kuleshov et al., 2018) act after training a base model and transform the predictions based on a separate calibration dataset. Regularization methods act during training and add a regularization term that penalizes calibration (Chung et al., 2021) . Empirical evidence suggests that post-hoc methods outperform regularization methods in terms of calibration within the context of regression (Dheur and Ben Taieb, 2023) . This superiority has been attributed to the finite-sample guarantee from which post-hoc methods benefit.  \nThis paper introduces a novel method called Quantile Recalibration Training that seamlessly integrates post-hoc calibration into the training process, resulting in an end-to-end approach. Our method leverages the concept of minimizing the sharpness of predictions while ensuring calibration (Gneiting et al., 2007) . By minimizing the negative log-likelihood (NLL), our approach achieves the desired sharpness, while simultaneously ensuring calibration at each training step using a  \n1 In this paper, we refer to probabilistic calibration as calibration to simplify terminology.  \ndedicated calibration dataset. Recalibration Training stands apart from other regularization methods by offering improvements in both the NLL and calibration of the final model. Our approach aligns with the recommendation made by Wang et al. (2021) to view model training and post-hoc calibration as an","cbCaiiAHfVZhB4yX","https://ap.wps.com/l/cbCaiiAHfVZhB4yX","pdf",3230881,1,33,"English","en",105,"# Introduction\n## Problem of miscalibration in neural regression\n## Probabilistic calibration and quantile calibration\n# Background on probabilistic calibration\n## Univariate regression setting\n## Predictive distributions and calibration definition","[{\"question\":\"What problem does the paper address in neural network regression?\",\"answer\":\"Neural networks can produce miscalibrated predictive distributions in regression. The paper focuses on probabilistic miscalibration, where predicted quantiles do not match the empirical frequencies.\"},{\"question\":\"How does Quantile Recalibration Training differ from existing post-hoc calibration?\",\"answer\":\"It integrates calibration directly into the training process. Unlike post-hoc methods that adjust predictions after training using an external calibration step, the proposed training procedure incorporates calibration during optimization without adding extra parameters.\"},{\"question\":\"What evidence is provided to validate the method?\",\"answer\":\"A large-scale experiment evaluates the approach on 57 tabular regression datasets, showing improved predictive accuracy while maintaining calibration. The study also includes ablation experiments and analysis of base model and hyperparameter impacts.\"}]","Probabilistic Calibration by Design for Neural Network Regression | PDF",1785677208,83,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"probabilistic-calibration-by-design-for-neural-network-regression","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/probabilistic-calibration-by-design-for-neural-network-regression/117601/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the paper address in neural network regression?","Question",{"text":75,"@type":76},"Neural networks can produce miscalibrated predictive distributions in regression. The paper focuses on probabilistic miscalibration, where predicted quantiles do not match the empirical frequencies.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does Quantile Recalibration Training differ from existing post-hoc calibration?",{"text":80,"@type":76},"It integrates calibration directly into the training process. Unlike post-hoc methods that adjust predictions after training using an external calibration step, the proposed training procedure incorporates calibration during optimization without adding extra parameters.",{"name":82,"@type":73,"acceptedAnswer":83},"What evidence is provided to validate the method?",{"text":84,"@type":76},"A large-scale experiment evaluates the approach on 57 tabular regression datasets, showing improved predictive accuracy while maintaining calibration. The study also includes ablation experiments and analysis of base model and hyperparameter impacts.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]