[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-121599-en":3,"doc-seo-121599-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},121599,7971461741311,"Ophelia","https://ap-avatar.wpscdn.com/avatar/74000253aff267980c6?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779345379180704826",8,"Research & Report","Probabilistic Perspectives in Neural Network-Based Machine Learning - Mini-Workshop","Artificial neural networks provide powerful tools in modern machine learning, while their mathematical foundations remain only partially understood. A central issue is the stochastic character of ANN training: optimization in high-dimensional parameter spaces with complex loss landscapes is shaped by stochastic initialization and noisy gradient updates. The mini-workshop surveyed recent progress on stochastic training dynamics using probabilistic methods and asymptotic limit theorems, enabling cross-fertilization between probability, optimization, and deep learning theory to advance new perspectives on neural network training.","Mathematisches Forschungsinstitut Oberwolfach  \nReport No. 50/2025  \nDOI: 10.4171/OWR/2025/50  \nMini-Workshop: Probabilistic Perspectives in Neural Network-Based Machine Learning  \nOrganized by  \nSteffen Dereich, M¨unster,  \nAymeric Dieuleveut, Palaiseau  \nSebastian Kassing, Berlin  \nSophie Langer, Bochum  \n26 October – 31 October 2025  \nAbstract. Artificial neural networks (ANNs) have emerged as a powerful tool in modern machine learning, yet their mathematical foundations remain only partially understood. A key challenge is the inherently stochastic nature of ANN training: optimization occurs in high-dimensional parameter spaces with complex loss landscapes, influenced by stochastic initialization and noisy gradient updates. Understanding these dynamics requires probabilistic methods and asymptotic frameworks. This workshop explored recent advances in stochastic training dynamics, emphasizing probabilistic techniques and limit theorems. By bringing together researchers from probability, optimization, and deep learning theory, this workshop laid the groundwork for new directions in understanding neural network training from a stochastic perspective.  \nMathematics Subject Classification (2020): 62M45, 60Gxx, 62G20, 90C30 .  \nLicense: Unless otherwise noted, the content of this report is licensed under CC BY SA 4.0 .  \nIntroduction by the Organizers  \nThe workshop Probabilistic Perspectives in Neural Network-Based Machine Learning, organized by Steffen Dereich (Universit¨at M¨unster), Aymeric Dieuleveut (´Ecole Polytechnique, Palaiseau), Sebastian Kassing (Bergische Universit¨at Wuppertal), and Sophie Langer (Ruhr Universit¨at Bochum), was attended by 16 participants from Germany, Switzerland, France, Italy, and the Netherlands. The program included 20 talks (45 minutes each) and two open problem sessions (90 minutes), providing ample time for discussions. Interactions during talks, breaks,  \n2674 Oberwolfach Report 50/2025  \nand problem sessions fostered new collaborations, strengthened existing connections, and enabled participants to exchange ideas across disciplines. The workshop brought together researchers working on probability theory, stochastic processes, interacting particle systems, and optimization. Several talks highlighted recent advances in limit theorems, stochastic dynamics, random structures, and applications to statistical physics and algorithmics. Substantial interaction between communities allowed the blending of analytical, probabilistic, and geometric techniques.  \nThe scientific highlights can be summarized as follows:  \nAnalysis and Design of Optimization Algorithms: Several talks of this workshop focused on providing rigorous theoretical guarantees, such as limit theorems, and developing systematic methods for fundamental optimization algorithms used in training machine learning models. S. Dereich established a Central Limit Theorem (CLT) for the averaged Adam algorithm, while J. Schmidt-Hieber provided a quenched CLT and convergence rates under noisy SGD (such as with dropout) .  \nS. Weissmann provided almost sure convergence guarantees for the last iterate of SGD under a gradient domination condition. S. Kassing analyzed convergence of SGD schemes for Lojasiewicz-landscapes (a weak non-convex condition), providing bounds on step-sizes and perturbation size to guarantee almost sure convergence of the iterates. A. Dieuleveut used the PEPIT framework to provably show that the classical Heavy Ball method does not achieve accelerated convergence on smooth, strongly convex functions, resolving a long-standing question. A. Taylor overviewed systematic methods for algorithm analysis, including the primaland dual Performance Estimation Problems (PEP), which facilitate constructive worst-case analysis and the design of optimal first-order algorithms.  \nContinuum and Fluctuation Models of Training Dynamics Several talks analyzed discrete training dynamics, such as SGD, via continuous limits (SDEs/SPDEs) to study noise","cbCaijB5z3qqLPXN","https://ap.wps.com/l/cbCaijB5z3qqLPXN","pdf",266999,1,28,"English","en",105,"# Abstract\n# Introduction by the Organizers\n## Participation and Program\n# Scientific Highlights\n## Analysis and Design of Optimization Algorithms\n## Continuum and Fluctuation Models of Training Dynamics\n## Architecture Theory and Probabilistic Effects\n## Generative Models and High-Dimensional Problems","[{\"question\":\"What is the main focus of the mini-workshop?\",\"answer\":\"The workshop focuses on probabilistic perspectives on neural network training, especially stochastic training dynamics analyzed with probabilistic techniques and asymptotic frameworks.\"},{\"question\":\"Why is stochasticity central to understanding ANN training?\",\"answer\":\"Training involves optimization over high-dimensional parameter spaces with complex loss landscapes, where stochastic initialization and noisy gradient updates strongly influence the dynamics.\"},{\"question\":\"What research themes were highlighted in the program?\",\"answer\":\"Key themes included theoretical analysis and guarantees for optimization algorithms, continuum/fluctuation models via stochastic differential or partial differential equations, and architectural effects on training trajectories and model properties.\"}]","Probabilistic Perspectives in Neural Network-Based Machine Learning - Mini-Workshop | PDF",1785736421,71,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"probabilistic-perspectives-in-neural-network-based-machine-learning-mini-workshop","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/probabilistic-perspectives-in-neural-network-based-machine-learning-mini-workshop/121599/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main focus of the mini-workshop?","Question",{"text":75,"@type":76},"The workshop focuses on probabilistic perspectives on neural network training, especially stochastic training dynamics analyzed with probabilistic techniques and asymptotic frameworks.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Why is stochasticity central to understanding ANN training?",{"text":80,"@type":76},"Training involves optimization over high-dimensional parameter spaces with complex loss landscapes, where stochastic initialization and noisy gradient updates strongly influence the dynamics.",{"name":82,"@type":73,"acceptedAnswer":83},"What research themes were highlighted in the program?",{"text":84,"@type":76},"Key themes included theoretical analysis and guarantees for optimization algorithms, continuum/fluctuation models via stochastic differential or partial differential equations, and architectural effects on training trajectories and model properties.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]