[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117378-en":3,"doc-seo-117378-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117378,7971461740886,"Theodore","https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2",8,"Research & Report","Beyond the Norms - Detecting Prediction Errors in Regression Models","This paper tackles the challenge of detecting unreliable behavior in regression algorithms, driven by intrinsic variability such as aleatoric uncertainty or by modeling errors such as model uncertainty. It introduces a formal notion of unreliability defined via a specified discrepancy between prediction and expected behavior. Probabilistic modeling is used to estimate discrepancy density, and statistical diversity is quantified with a proposed dissimilarity metric. The resulting data-driven score expresses regression outcome uncertainty, improving error detection across multiple regression tasks and advancing uncertainty quantification and safe machine learning.","Beyond the Norms: Detecting Prediction Errors in Regression Models  \nAndres Altieri 1 Marco Romanelli 2 Georg Pichler 3 Florence Alberge 4 Pablo Piantanida 5  \nAbstract  \nThis paper tackles the challenge of detecting unreliable behavior in regression algorithms, which may arise from intrinsic variability (e.g., aleatoric uncertainty) or modeling errors (e.g., model uncertainty) . First, we formally introduce the notion of unreliability in regression, i.e., when the output of the regressor exceeds a specified discrepancy (or error) . Then, using powerful tools for probabilistic modeling, we estimate the discrepancy density, and we measure its statistical diversity using our proposed metric for statistical dissimilarity. In turn, this allows us to derive a data-driven score that expresses the uncertainty of the regression outcome. We show empirical improvements in error detection for multiple regression tasks, consistently outperforming popular baseline approaches, and contributing to the broader field of uncertainty quantification and safe machine learning systems. Our code is available at [https:](https:)// [zenodo.org/records/11281964](zenodo.org/records/11281964) .  \n1. Introduction  \nIn recent years, machine learning has gained ground in fields such as automatic processing, and autonomous decisionmaking, generating a crucial need for operational safety and reliability, aiming to prevent catastrophic errors (Amodeiet al., 2016 ; Antun et al., 2020 ; Li et al., 2023) . The need for trustworthy models has generated a lot of research in areas concerned with the identification of anomalous patterns that  \n1Laboratoire des signaux et systmes (L2S), Universit ParisSaclay CNRS CentraleSuplec, Gif-sur-Yvette, France 2New York University, New York, NY, USA 3Institute of Telecommunications, TU Wien, Vienna, Austria 4 Systmes et applications des technologies de l’information et de l’nergie (SATIE), CNRS Universit Paris-Saclay, Gif-sur-Yvette, France 5International Laboratory on Learning Systems (ILLS) and Quebec AI Institute (Mila), McGill ETS CNRS Universit Paris-Saclay CentraleSuplec, Montreal (QC), Canada. Correspondence to: Andres Altieri \u003C[andres.altieri@centralesupelec.fr](andres.altieri@centralesupelec.fr) >, Marco Romanelli \u003C[mr6852@nyu.edu](mr6852@nyu.edu) >.  \nProceedings of the 41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024 . Copyright 2024 by the author(s) .  \nmay trigger undesired, and potentially dangerous, predictions such as out-of-distribution detection (Dadalto et al., 2022 ; Ming et al., 2023), selective classification (Corbire et al., 2019 ; Huang et al., 2020), misclassification detection (Granese et al., 2021), and adversarial attacks (Goodfellow et al., 2015 ; Picot et al., 2023), among others.  \nAnomalies may stem from intrinsic variability in the data, i.e., aleatoric uncertainty (Kiureghian & Ditlevsen, 2009) . For instance, certain inputs may exhibit a larger dispersion of the dependent variable than expected, i.e., heteroscedastic uncertainty (Kendall & Gal, 2017) . Additional sources of unreliable behavior may be associated with approximation errors or model uncertainty (Kiureghian & Ditlevsen, 2009), occurring when the regressor is not statistically close to potential observations. This discrepancy may arise from various factors such as sub-optimal model selection, inadequate model fitting, and insufficient training, among others.  \nThe challenge of identifying anomalous predictions in machine learning models is closely connected to the task of uncertainty quantification, which aims to assess the uncertainty inherent in model predictions (Kotelevskii et al., 2022 ; Liu et al., 2020) . In classification tasks, the process typically involves estimating a probability distribution across labels, introducing a clear distinction between correct and incorrect decisions and a straightforward understanding of uncertainty. In stark contrast, regression problems often present a mo","cbCaiqlVVWbjo0rc","https://ap.wps.com/l/cbCaiqlVVWbjo0rc","pdf",1085518,1,36,"English","en",105,"# Introduction\n## Contributions\n## Related Work","[{\"question\":\"What problem does the paper address in regression models?\",\"answer\":\"It addresses detecting unreliable behavior in regression predictions, which can arise from intrinsic data variability (aleatoric uncertainty) or from modeling errors (model uncertainty).\"},{\"question\":\"How is “unreliability” defined for regression outputs?\",\"answer\":\"Unreliability is defined by whether the regressor output exceeds a specified discrepancy (error) threshold relative to typical observations.\"},{\"question\":\"What does the proposed method produce and how is it used?\",\"answer\":\"It estimates discrepancy density using probabilistic modeling, quantifies statistical diversity with a dissimilarity metric, and derives a data-driven score that expresses uncertainty and flags potentially anomalous prediction outcomes.\"}]","Beyond the Norms - Detecting Prediction Errors in Regression Models | PDF",1785675473,91,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"beyond-the-norms-detecting-prediction-errors-in-regression-models","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/beyond-the-norms-detecting-prediction-errors-in-regression-models/117378/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the paper address in regression models?","Question",{"text":75,"@type":76},"It addresses detecting unreliable behavior in regression predictions, which can arise from intrinsic data variability (aleatoric uncertainty) or from modeling errors (model uncertainty).","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How is “unreliability” defined for regression outputs?",{"text":80,"@type":76},"Unreliability is defined by whether the regressor output exceeds a specified discrepancy (error) threshold relative to typical observations.",{"name":82,"@type":73,"acceptedAnswer":83},"What does the proposed method produce and how is it used?",{"text":84,"@type":76},"It estimates discrepancy density using probabilistic modeling, quantifies statistical diversity with a dissimilarity metric, and derives a data-driven score that expresses uncertainty and flags potentially anomalous prediction outcomes.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]