[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124559-en":3,"doc-seo-124559-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},124559,1099513958607,"Jiven","https://ap-avatar.wpscdn.com/avatar/100002390cf8733938c?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778829742770036399",8,"Research & Report","QUANTIFYING ALEATORIC AND EPISTEMIC UNCERTAINTY IN MACHINE LEARNING - ARE CONDITIONAL ENTROPY AND MUTUAL INFORMATION APPROPRIATE MEASURES?","This short note provides a critical discussion on measuring aleatoric and epistemic uncertainty using conditional entropy and mutual information, which has become common in machine learning. It challenges the widely used notion that total uncertainty can be additively decomposed into distinct aleatoric and epistemic parts. The argument is framed in terms of predictive uncertainty in supervised learning and the role of second-order (level-2) uncertainty representations.","arXiv :2209 .03302v 1 [ cs .LG] 7 Sep 2022  \nQUANTIFYING ALEATORIC AND EPISTEMIC UNCERTAINTY IN MACHINE LEARNING: ARE CONDITIONAL ENTROPY AND MUTUAL INFORMATION APPROPRIATE MEASURES?  \nA PREPRINT  \nEyke Hüllermeier  \nInstitute of Informatics  \nUniversity of Munich (LMU)  \n[eyke@lmu.de](eyke@lmu.de)  \nSeptember 8, 2022  \nABSTRACT  \nThis short note is a critical discussion of the quantification of aleatoric and epistemic uncertainty in terms of conditional entropy and mutual information, respectively, which has recently been proposed in machine learning and has become quite common since then. More generally, we question the idea of an additive decomposition of total uncertainty into its aleatoric and epistemic constituents.  \n1 Introduction  \nThe distinction between different types of uncertainty and their quantification has been of major interest in the recent machine learning literature [Senge et al., 2014, Kendall and Gal, 2017], and various methods for quantifying so-called aleatoric and epistemic uncertainty have been proposed [Hüllermeier and Waegeman, 2021] . In the context of supervised learning, the focus is typically on predictive uncertainty, i.e., the learner's uncertainty in the outcome y 2 Y given a query instance x (an element of an underlying instance space X ) for which a prediction is sought. The aleatoric part of this uncertainty is due to the supposedly stochastic nature of the dependence between instances and outcomes, whence the “ground-truth” is a conditional probability distribution p(􀀁 j x) on Y. Thus, each outcome y has a certain probability to occur, so that even complete knowledge about the underlying data-generating process does not allow for predicting the outcome with full certainty.  \nObviously, the learner does not even know p(􀀁 j x) . Instead, it produces a “guess” p^(􀀁 j x) on the basis of the sample data D provided for training. Broadly speaking, epistemic uncertainty refers to uncertainty about the true probability and hence the discrepancy between p and p^. This (second-order) uncertainty should be captured and represented by an“uncertainty-aware” learner in one way or the other. One approach is Bayesian inference, where the learner translates uncertainty about the (Bayes) predictor into uncertainty about a prediction expressed in terms of the posterior predictive distribution. Another idea is to let the learner more directly predict, not only the target variable, but also its own uncertainty about the prediction. For example, instead of predicting a probability distribution p^(􀀁 j x), the learner may predict a second-order distribution in the form of a distribution of distributions [Malinin and Gales, 2018, 2019, Malininet al., 2020, Charpentier et al., 2020, Huseljic et al., 2020, Kopetzki et al., 2021] .  \nEither way, what is eventually produced by the learner is a second-order or level-2 predictor  \nH : X 􀀀! 􀀁 (2)K ; (1)  \nwhere 􀀁 (2)K = P (P(Y)) denotes the set of second-order distributions, i.e., probability distributions on probability distributions on Y. For the sake of simplicity, we subsequently omit the conditioning on the query instance x for which  \nFigure 1: Uncertain prediction Q of a Bernoulli distribution, which corresponds to the probability 􀀒 of the positive class in binary classification. In Bayesian inference, Q is given by the posterior predictive distribution (epistemic level), anda point prediction ^􀀒 (aleatoric level) is obtained by model averaging.  \na prediction is sought. Moreover, we assume the target variable to be categorical (although the arguments put forward also apply to the case of regression), i.e., Y = fy1 ; : : : ; yKg, so that P (Y) can be identified with the K-simplex  \n􀀁 K .. = n􀀒 = (􀀒1 ; : : : ; 􀀒K ) 2 [0; 1]K j k􀀒k1 = 1 o : (2)  \nEach probability vector 􀀒 2 􀀁 K identifies a categorical distribution Cat(􀀒) on Y such that 􀀒k = p (yk j 􀀒) is the probability of outcome yk , and one of these distributions, 􀀒 􀀃 , corresponds to the (unknown) ground-truth. If ","cbCair2JtWznSAwC","https://ap.wps.com/l/cbCair2JtWznSAwC","pdf",400430,1,7,"English","en",105,"# Introduction\n# Uncertainty quantification","[{\"question\":\"What is the main purpose of this note?\",\"answer\":\"It critically evaluates whether conditional entropy and mutual information are appropriate measures for aleatoric and epistemic uncertainty in machine learning.\"},{\"question\":\"Why does the note question additive decomposition of uncertainty?\",\"answer\":\"It challenges the idea that total uncertainty should be split additively into aleatoric and epistemic components, as commonly proposed in the literature.\"},{\"question\":\"How is uncertainty described in supervised learning in the document?\",\"answer\":\"It distinguishes aleatoric uncertainty (stochastic dependence captured by a conditional probability distribution) from epistemic uncertainty (uncertainty about the true probability distribution), often represented via second-order predictors.\"}]","QUANTIFYING ALEATORIC AND EPISTEMIC UNCERTAINTY IN MACHINE LEARNING - ARE CONDITIONAL ENTROPY AND MUTUAL INFORMATION APPROPRIATE MEASURES? | PDF",1785892990,18,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"quantifying-aleatoric-and-epistemic-uncertainty-in-machine-learning-are-conditional-entropy-and-mutual-information-appropriate-measures","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/quantifying-aleatoric-and-epistemic-uncertainty-in-machine-learning-are-conditional-entropy-and-mutual-information-appropriate-measures/124559/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main purpose of this note?","Question",{"text":75,"@type":76},"It critically evaluates whether conditional entropy and mutual information are appropriate measures for aleatoric and epistemic uncertainty in machine learning.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Why does the note question additive decomposition of uncertainty?",{"text":80,"@type":76},"It challenges the idea that total uncertainty should be split additively into aleatoric and epistemic components, as commonly proposed in the literature.",{"name":82,"@type":73,"acceptedAnswer":83},"How is uncertainty described in supervised learning in the document?",{"text":84,"@type":76},"It distinguishes aleatoric uncertainty (stochastic dependence captured by a conditional probability distribution) from epistemic uncertainty (uncertainty about the true probability distribution), often represented via second-order predictors.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]