[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-120621-en":3,"doc-seo-120621-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},120621,13056703020460,"Valentina","https://ap-avatar.wpscdn.com/avatar/be000253dac470eee5d?_k=1778207105932848923",8,"Research & Report","Overparametrization, Regularization, Identi􀀌ability and Uncertainty in Machine Learning","Workshop report focusing on how theoretical sub-communities in machine learning handle the inherently ill-posed inference problem that maps finite data into higher- or infinite-dimensional answers. Discussions connect overparametrization and regularization with identifiability, uncertainty quantification, approximate posterior methods, and probabilistic numerics. It further examines regret minimization in sequential decision-making, including continuous and multi-agent settings, and challenges of non-stationary data with out-of-distribution and out-of-variables generalization perspectives.","Mathematisches Forschungsinstitut Oberwolfach  \nReport No. 4/2025  \nDOI: 10.4171/OWR/2025/4  \nOverparametrization, Regularization, Identi􀀌ability and Uncertainty in Machine Learning  \nOrganized by Nicol􀀒o Cesa-Bianchi, Milano Philipp Hennig, T¨ubingen  \nAndreas Krause, Z¨urich  \nUlrike von Luxburg, T¨ubingen  \n26 January – 31 January 2025  \nAbstract. In machine learning, a 􀀌eld addressing the extraction of information and structure from 􀀌nite data with the means of computer science and mathematics, maps from 􀀌nite-dimensional spaces of data or computations into spaces of higher, or in􀀌nite dimensionality are a central theme.  \nThe workshop brought together researchers with diverse viewpoints to discuss how di􀀋erent theoretical sub-communities within the 􀀌eld treat the resulting ill-posed operations, and what kind of features of algorithms and models can  \nemerge as a result.  \nMathematics Subject Classi􀀌cation (2020): 62-XX, 68-XX.  \nLicense: Unless otherwise noted, the content of this report is licensed under CC BY SA 4.0 .  \nIntroduction by the Organizers  \n1. Topical Overview  \nWhile machine learning (ML) currently enjoys outsized interest by the public as well as by multiple scienti􀀌c domains, it is still a relatively young 􀀌eld, drawing methods, concepts, and formalisms from older domains. Since its inception, the ML community has struck a productive balance between empirical and applied work on the one hand, and a desire for rigorous theoretical analysis on the other. This has allowed 􀀌eld to identify new, and often disruptive ideas quickly, but to then also develop them into general, e􀀎cient, well-understood frameworks. In the early years of this decade, the empirical side of the 􀀌eld has once again taken  \n176 Oberwolfach Report 4/2025  \nthe lead, making ground-breaking advances in particular in the area of generative modelling from unsupervised data using deep neural architectures (parametrizeddi􀀋erentiable functions) to address extremely high-dimensional structured probability distributions. These models, which were elevated with two Nobel prizes in 2024, pose many theoretical questions about their behaviour, limitations, ande􀀎cient algorithmic means to train and control them.  \nOberwolfach Workshop 4/2025 sought to bring together members of multiple theoretical sub-communities of machine learning, mostly from Europe, to discuss recent advances in their work. We made a deliberate e􀀋ort to connect people with di􀀋erent perspectives to draw connections across the 􀀌eld, even if this meant that the title of the workshop became a bit unwieldy. For, as the 􀀌eld continues to grow, similar observations are made repeatedly, from di􀀋erent viewpoints and using di􀀋erent technical vocabularies. The central operation of machine learning is that of inference: Mapping from a 􀀌nite-dimensional space of data, prompts, inputs into a typically much higher – or in􀀌nite-dimensional space of answers, hypotheses, or otherwise latent variables. This is principally an ill-posed process, and di􀀋erent sub-communities address the resulting challenges and features di􀀋erently. A few examples of the resulting questions are as follows:  \n• From the perspective of probability theory, identi􀀌ability is associated with uncertainty quanti􀀌cation. But most contemporary deep architectures do not quantify epistemic uncertainty. Several of the talks at the workshop discussed approximate techniques to endow advanced, contemporary deep models with such posterior probability measures, including for models with nonparametric, function-valued outputs. Numerical methods commonly used in machine learning (optimization, least-squares, simulation methods, etc.) also have internal discretization leading to misspeci􀀌cation and uncertainty. This error interacts non-trivially with the empirical estimation error caused by 􀀌nite data. By modelling computational error with probability measures Probabilistic numerics envisions a uni􀀌ed notion of uncertainty / error estimatio","cbCaigXQbq2PXtLc","https://ap.wps.com/l/cbCaigXQbq2PXtLc","pdf",304584,1,34,"English","en",105,"# Introduction by the Organizers\n## Topical Overview","[{\"question\":\"Why is inference in machine learning considered ill-posed?\",\"answer\":\"It involves mapping from finite-dimensional data spaces into much higher- or infinite-dimensional spaces of hypotheses or latent variables, which makes the operation inherently ill-posed.\"},{\"question\":\"How does identifiability relate to uncertainty quantification in the workshop discussions?\",\"answer\":\"Identifiability is linked to uncertainty quantification, and talks covered approximate techniques to equip deep models with posterior probability measures, as well as how numerical discretization and probability modeling can unify uncertainty/error estimation.\"},{\"question\":\"What settings were considered for regret minimization in sequential decision-making?\",\"answer\":\"The workshop discussed how minimax regret depends on properties like the decision space and feedback, and included implications in continuous decision spaces and multi-agent environments such as online market problems.\"}]","Overparametrization, Regularization, Identi􀀌ability and Uncertainty in Machine Learning | PDF",1785730936,86,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"overparametrization-regularization-identifiability-and-uncertainty-in-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/overparametrization-regularization-identifiability-and-uncertainty-in-machine-learning/120621/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is inference in machine learning considered ill-posed?","Question",{"text":75,"@type":76},"It involves mapping from finite-dimensional data spaces into much higher- or infinite-dimensional spaces of hypotheses or latent variables, which makes the operation inherently ill-posed.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does identifiability relate to uncertainty quantification in the workshop discussions?",{"text":80,"@type":76},"Identifiability is linked to uncertainty quantification, and talks covered approximate techniques to equip deep models with posterior probability measures, as well as how numerical discretization and probability modeling can unify uncertainty/error estimation.",{"name":82,"@type":73,"acceptedAnswer":83},"What settings were considered for regret minimization in sequential decision-making?",{"text":84,"@type":76},"The workshop discussed how minimax regret depends on properties like the decision space and feedback, and included implications in continuous decision spaces and multi-agent environments such as online market problems.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]