[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-125511-en":3,"doc-seo-125511-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},125511,7971461741311,"Ophelia","https://ap-avatar.wpscdn.com/avatar/74000253aff267980c6?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779345379180704826",8,"Research & Report","Noisy gradient flows - with applications in machine learning","This thesis studies the behavior of machine-learning models with gradient-flow structure, emphasizing two key applications: stochastic gradient descent and a toy model of transformers. In the overparameterized regime, the thesis analyzes limiting dynamics of noisy gradient descent near the large set of global minimizers, showing how noise structure changes both the limiting process and its time scale. It applies results to Dropout, label noise, and minibatch SGD noise, then develops aggregation-type PDE models for self-attention. ","Noisy gradient flows  \nCitation for published version (APA):  \nShalova, A. (2025) . Noisy gradient flows: with applications in machine learning. [Phd Thesis 1 (Research TU/e / Graduation TU/e), Mathematics and Computer Science] . Eindhoven University of Technology.  \nDocument status and date:  \nPublished: 12/09/2025  \nDocument Version:  \nPublisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)  \nPlease check the document version of this publication:  \n• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.  \n• The final author version and the galley proof are versions of the publication after peer review.  \n• The final published version features the final layout of the paper including the volume, issue and page numbers.  \nLink to publication  \nGeneral rights  \nCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.  \n• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.  \n• You may not further distribute the material or use it for any profit-making activity or commercial gain  \n• You may freely distribute the URL identifying the publication in the public portal.  \nIf the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:  \n[www.tue.nl/taverne](www.tue.nl/taverne)  \nTake down policy  \nIf you believe that this document breaches copyright please contact us at:  \n[openaccess@tue.nl](openaccess@tue.nl)  \nproviding details and we will investigate your claim.  \nDownload date: 28. Apr. 2026  \nNoisy gradient flows with applications in machine learning  \nAnna Shalova  \nThe work is a part of the research program ‘Unraveling Neural Networks with Structure-Preserving Computing’(file number OCENW.GROOT.2019.044), which is financed by the Dutch Research Council (Nederlandse Organisatie voor Wetenschappelijk Onderzoek—NWO) .  \nA catalogue record is available from the Eindhoven University of Technology Library.  \nISBN: 978-90-386-6435-4  \nPrinted by Ipskamp Printing  \nCopyright ©2025 by Anna Shalova. All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without prior permission of the author.  \nNOISY GRADIENT FLOWS  \nwith applications in machine learning  \nPROEFSCHRIFT  \nter verkrijging van de graad van doctor aan de Technische Universiteit Eindhoven, op gezag van de rector magnificus prof.dr. S. K. Lenaerts, voor een commissie aangewezen door het College voor Promoties, in het openbaar te verdedigen op vrijdag 12 September 2025 om 11:00 uur  \ndoor  \nAnna Shalova  \ngeboren te Taganrog, Rusland  \nDit proefschrift is goedgekeurd door de promotoren en de samenstelling van de promotiecommissie is als volgt:  \nvoorzitter: prof.dr. E.R. van den Heuvel 1e promotor: prof.dr. M.A. Peletier  \n2e promotor: prof.dr. A. Schlichting (Universität Ulm)  \nleden: prof.dr. C. Maes (Katholieke Universiteit Leuven)  \nprof.dr. A.J. Schmidt-Hieber (Universiteit Twente) dr. O.T.C. Tse  \ndr. J. Sanders  \nprof.dr. A. Agazzi (Universität Bern)  \nHet onderzoek dat in dit proefschrift wordt beschreven is uitgevoerd in overeenstemming met de TU/e Gedragscode Wetenschapsbeoefening.  \nMathematics is the language with which God has written the universe.  \nGalileo Galilei","cbCaiskpNDR1k6sj","https://ap.wps.com/l/cbCaiskpNDR1k6sj","pdf",3516502,1,246,"English","en",105,"# Abstract\n## Noisy gradient descent in the overparameterized regime\n## Aggregation PDE models for transformer self-attention\n## McKean–Vlasov equation, critical points, bifurcations and phase transitions","[{\"question\":\"What are the main machine-learning applications studied in the thesis?\",\"answer\":\"The thesis focuses on stochastic gradient descent and a toy model of transformers, formulated through gradient-flow and aggregation-type structures.\"},{\"question\":\"How does the thesis analyze noisy gradient descent dynamics?\",\"answer\":\"It characterizes limiting dynamics for noisy gradient descent systems in the overparameterized regime, especially near the zero-loss set, and studies the small step-size and small noise limit.\"},{\"question\":\"What additional mathematical models appear in the study of transformers?\",\"answer\":\"The work studies aggregation partial differential equations as simplified models of self-attention, including the McKean–Vlasov equation on compact manifolds and the high-dimensional sphere, analyzing minimizers and phase transitions.\"}]","Noisy gradient flows - with applications in machine learning | PDF",1785899517,620,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"noisy-gradient-flows-with-applications-in-machine-learning","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/noisy-gradient-flows-with-applications-in-machine-learning/125511/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What are the main machine-learning applications studied in the thesis?","Question",{"text":75,"@type":76},"The thesis focuses on stochastic gradient descent and a toy model of transformers, formulated through gradient-flow and aggregation-type structures.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the thesis analyze noisy gradient descent dynamics?",{"text":80,"@type":76},"It characterizes limiting dynamics for noisy gradient descent systems in the overparameterized regime, especially near the zero-loss set, and studies the small step-size and small noise limit.",{"name":82,"@type":73,"acceptedAnswer":83},"What additional mathematical models appear in the study of transformers?",{"text":84,"@type":76},"The work studies aggregation partial differential equations as simplified models of self-attention, including the McKean–Vlasov equation on compact manifolds and the high-dimensional sphere, analyzing minimizers and phase transitions.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]