[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117055-en":3,"doc-seo-117055-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117055,3848291630094,"Emma Wilson","https://eur-avatar.wpscdn.com/davatar_085a072bc5b1113ac321206ff7593b45",8,"Research & Report","On Machine Learning Loss Landscapes","Loss functions are central to training machine learning models, and evaluating them across large parameter ranges yields a loss landscape. This thesis explores geometric properties of loss landscapes for neural networks and Gaussian processes, leveraging theoretical chemistry energy-landscape concepts to study model performance, robustness, and interpretability. Cross-disciplinary analogues—such as heat capacity, monotonic sequence basins, and catastrophe theory—enable new tools for analysis. Applications include ensemble generation, loss-function selection via appAUC versus conventional losses, and identifying input feature relevance from conserved-weight structures. For Gaussian processes, the work analyzes loss-function hyperparameter space across kernel choices, tracing fold catastrophes as the Matérn smoothness parameter ν varies and assessing common half-integer parameterizations.","On Machine Learning Loss Landscapes  \nMaximilian Paul Niroomand  \nDepartment of Chemistry  \nUniversity of Cambridge  \nThis thesis is submitted for the degree of Doctor of Philosophy  \nDowning College November 2023  \nI would like to dedicate this thesis to my family and friends, the people who keep me motivated every day and who allow me to be my happiest self. To my parents, who have been there for me literally since day one, and who have helped me to make many good, and avoid some bad, decisions. To Katha, who has the dubious fortune of spending about every waking hour with me, bringing happiness to my life and making every day the best possible-to many more adventures! To my siblings Leopold and Fine, whose value in my life grows with each passing day. To everyone else in the family: Anna-Flo, Daniele, Eva, Julian, Oma(s) & Opa, Pia-Marie, Thomas, Utawho I have spent countless wonderful moments with. To all my friends, across Cambridge and the world-I’m grateful for having you all in my life and hope that our friendships will continue and deepen far into the future. To Julius, Jan, Fynn, Enoch, Trenton, Will, Marc, Sven, Zach, Elena, Simon, Lukas, Jörn, Patrick, Felix, Laura, Aadi, Niklas, Aadya, Hishan, Felix, Elisa, and everyone else whom I’ve met on this  \nexciting journey. Ad astra!  \nDeclaration  \nThis thesis is the result of my own work and includes nothing which is the outcome of work done in collaboration except as declared in the preface and specified in the text. It is not substantially the same as any work that has already been submitted, or, is being concurrently submitted, for any degree, diploma or other qualification atthe University of Cambridge or any other University or similar institution except as declared in the preface and specified in the text. It does not exceed the prescribed word limit for the relevant Degree Committee.  \nAcknowledgements  \nFirst and foremost I would like to express my thanks to my supervisor, Prof. David Wales for giving me all the freedom to explore my own ideas, but also for all the guidance, support and wisdom he has provided me with on my way. Always following the ideas that are currently most exciting, as David so often says, will stick with me for the rest of my life. I would also like to acknowledge all collaborators from IBM, most importantly Dr Edward Pyzer-Knapp and Dr Luke Dicks. Furthermore the whole Wales group for a great time at AVT, social events or in Porquerolles. Lastly, I would like to gratefully acknowledge Downing College and the Voellm-Hruska fellowship for PhD funding and countless wonderful moments in college.  \nAbstract  \nLoss functions are pivotal to the training of every machine learning model. When evaluating the loss function over a large range of parameters, a loss landscape is obtained. In this thesis, loss landscapes for various classes of machine learning models are explored. Geometric properties of the loss landscape can provide deep insights into machine learning models and their decision making process. Today, the most critical questions in machine learning are matters of model performance, robustness and interpretability. Here, tools from the energy landscapes field in theoretical chemistry are used to study these questions for neural networks and Gaussian processes. This cross-disciplinary approach is facilitated by a collection of analogues between both fields, such as the concept of heat capacity, monotonic sequence basins, or catastrophe theory, as developed in this work.  \nThe energy landscape perspective of loss landscape proves to be a helpful approach to examine critical questions in the machine learning field. Energy landscapes tools are applied to neural network ensemble generation and reveal that different minima of the loss landscape specialise on different sections of the input data, improving classification accuracy when combined. A second application is the evaluation and selection of loss functions, guiding hyperparameter choice","cbCailmUouz2WPbU","https://ap.wps.com/l/cbCailmUouz2WPbU","pdf",16502466,1,156,"English","en",105,"# Abstract\n## Loss landscapes and energy-landscape tools\n## Applications: ensembles and loss-function selection\n## Monotonic sequence basins and feature relevance\n## Gaussian processes: kernel choices and Matérn parameter ν\n## Model comparisons and future work","[{\"question\":\"Why are loss functions important for machine learning in this thesis?\",\"answer\":\"Loss functions are pivotal for training machine learning models. By evaluating a loss function over many parameters, the thesis forms and studies corresponding loss landscapes.\"},{\"question\":\"How does the thesis connect loss landscapes to theoretical chemistry?\",\"answer\":\"It uses tools and analogues from the energy landscapes field in theoretical chemistry. Examples include heat capacity, monotonic sequence basins, and catastrophe theory.\"},{\"question\":\"What findings are reported for loss-function choice using appAUC?\",\"answer\":\"The thesis compares appAUC to conventional alternatives and finds appAUC is more accurate but less robust, making it inferior for real-life problems.\"}]","On Machine Learning Loss Landscapes | PDF",1785673471,393,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"on-machine-learning-loss-landscapes","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/on-machine-learning-loss-landscapes/117055/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why are loss functions important for machine learning in this thesis?","Question",{"text":75,"@type":76},"Loss functions are pivotal for training machine learning models. By evaluating a loss function over many parameters, the thesis forms and studies corresponding loss landscapes.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the thesis connect loss landscapes to theoretical chemistry?",{"text":80,"@type":76},"It uses tools and analogues from the energy landscapes field in theoretical chemistry. Examples include heat capacity, monotonic sequence basins, and catastrophe theory.",{"name":82,"@type":73,"acceptedAnswer":83},"What findings are reported for loss-function choice using appAUC?",{"text":84,"@type":76},"The thesis compares appAUC to conventional alternatives and finds appAUC is more accurate but less robust, making it inferior for real-life problems.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]