[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-125636-en":3,"doc-seo-125636-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},125636,962075006959,"Anda","https://ap-avatar.wpscdn.com/avatar/e0002397efbe92a78e?_k=1776741047341049297",8,"Research & Report","Asymptotic Analysis of Machine Learning Models - Comparison Theorems and Universality","The thesis investigates machine learning models in asymptotic regimes, where data points and model parameters grow to infinity while maintaining constant ratios. It analyzes problems including LASSO and random-features regression, deriving asymptotic behavior such as learning-curve characterization and the predicted training and generalization error as a function of overparameterization. The work applies Gaussian comparison theorems as the main methodological framework: the convex Gaussian min-max theorem enables study of complex optimization by relating them to simpler alternative models. It further develops universality results showing that many ML statistics are determined by lower-order moments, enabling surrogate Gaussian models. ","Thesis for The Degree of Licentiate of Engineering  \nAsymptotic Analysis of Machine Learning Models  \nComparison Theorems and Universality  \nDavid Bosch  \nDepartment of Computer Science and Engineering Chalmers University of Technology | University of Gothenburg  \nGothenburg, Sweden, 2023  \nAsymptotic Analysis of Machine Learning Models  \nComparison Theorems and Universality  \nDavid Bosch  \n© David Bosch, 2023  \nexcept where otherwise stated.  \nAll rights reserved.  \nISSN 1652-876X  \nDepartment of Computer Science and Engineering Division of Data Science and AI  \nChalmers University of Technology | University of Gothenburg SE-412 96 G¨oteborg,  \nSweden  \nPhone: +46(0)31 772 1000  \nPrinted by Chalmers Digitaltryck, Gothenburg, Sweden 2023 .  \nTo my family.  \ni  \nAsymptotic Analysis of Machine Learning Models  \nComparison Theorems and Universality David Bosch  \nDepartment of Computer Science and Engineering  \nChalmers University of Technology | University of Gothenburg  \nAbstract  \nThe study of Machine Learning models in asymptotic regimes, has provided insight into many of the properties of ML models, but seemingly contradicts classical statistical wisdom. To solve this mystery, this thesis focuses on the analysis of models such as the LASSO and Random features regression, when the data points and model parameters grow infinite at constant ratios. It provides analysis for the asymptotic behavior of these problems, including characterization of the learning curves; the predicted training and generalization error as a function of the degree of overparameterization.  \nThe papers in this thesis particularly focus on the usage of Gaussian comparison theorems as a methodological tool for the analysis of these problems. In particular, the convex Gaussian min max theorem allows us to study more complex ML optimization problems, by considering alternative models that are simpler to analyze, but asymptotically hold similar properties.  \nSecondarily, this thesis considers universality, which within the asymptotic context demonstrates that many statistics of ML models are fully determined by lower order statistical moments. This allows us to study surrogate Gaussian models, matching these moments. These surrogate Gaussian models can subsequently be analyzed by means of the Gaussian comparison theorems.  \nKeywords  \nAsymptotic Analysis, Learning Curves, Convex Gaussian Min-max Theorem, CGMT, Universality  \nList of Publications  \nAppended publications  \nThis thesis is based on the following publications:  \n[Paper I] David Bosch, Ashkan Panahi, Ayca ¨Ozcelikkale, Double Descent in Feature Selection: Revisiting LASSO and Basis Pursuit  \nICML 2021 Workshop on Overparameterization: Pitfalls & Opportunities.  \n[Paper II] David Bosch, Ashkan Panahi, Ayca ¨Ozcelikkale, Devdatt Dubhashi, Random Features Model with General Convex Regularization:  \nA Fine Grained Analysis with Precise Asymptotic Learning Curves AISTATS 2023.  \nOther publications  \nThe following publications were published during my PhD studies, or are currently in submission/under revision. However, they are not appended to this thesis, due to contents overlapping that of appended publications or contents not related to the thesis.  \n[a] Firooz Shahriari-Mehr, David Bosch, Ashkan Panahi, Decentralized Constrained Optimization: Double Averaging and Gradient Projection 2021 60th IEEE Conference on Decision and Control.  \n[b] David Bosch, Ashkan Panahi, Babak Hassibi, Precise Asymptotic Analysis of Deep Random Feature Models  \nSubmitted to COLT2023.  \nAcknowledgment  \nI would like to express my gratitude to my PhD supervisor, Ashkan Panahi, for his continued advice and support with my research. Without your guidance the work within would not have been possible. I would also like to thank my co-supervisor Devdatt Dubhashi and my examiner Dag Wedelin, for their support, feedback, and insight.  \nI would also like to express thanks to the people of the DSAI division. Among them my fellow PhD","cbCaibVMIjdDPqqa","https://ap.wps.com/l/cbCaibVMIjdDPqqa","pdf",440149,1,36,"English","en",105,"# Abstract\n# Keywords\n# List of Publications\n## Appended publications\n## Other publications\n# Acknowledgment","[{\"question\":\"Which machine learning models and regimes are analyzed in this thesis?\",\"answer\":\"The thesis analyzes models such as LASSO and random-features regression in asymptotic regimes where both data points and model parameters grow to infinity at constant ratios.\"},{\"question\":\"What is the role of Gaussian comparison theorems in the analysis?\",\"answer\":\"Gaussian comparison theorems, especially the convex Gaussian min-max theorem, provide a methodological tool to study complex ML optimization problems by comparing them to simpler models with asymptotically similar properties.\"},{\"question\":\"How does universality contribute to the overall results?\",\"answer\":\"In the asymptotic context, universality shows that many statistics of ML models depend only on lower-order statistical moments. This enables the use of surrogate Gaussian models matched to those moments for subsequent analysis via comparison theorems.\"}]","Asymptotic Analysis of Machine Learning Models - Comparison Theorems and Universality | PDF",1785900347,91,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"asymptotic-analysis-of-machine-learning-models-comparison-theorems-and-universality","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/asymptotic-analysis-of-machine-learning-models-comparison-theorems-and-universality/125636/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Which machine learning models and regimes are analyzed in this thesis?","Question",{"text":75,"@type":76},"The thesis analyzes models such as LASSO and random-features regression in asymptotic regimes where both data points and model parameters grow to infinity at constant ratios.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is the role of Gaussian comparison theorems in the analysis?",{"text":80,"@type":76},"Gaussian comparison theorems, especially the convex Gaussian min-max theorem, provide a methodological tool to study complex ML optimization problems by comparing them to simpler models with asymptotically similar properties.",{"name":82,"@type":73,"acceptedAnswer":83},"How does universality contribute to the overall results?",{"text":84,"@type":76},"In the asymptotic context, universality shows that many statistics of ML models depend only on lower-order statistical moments. This enables the use of surrogate Gaussian models matched to those moments for subsequent analysis via comparison theorems.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]