[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118305-en":3,"doc-seo-118305-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118305,7971461740886,"Theodore","https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2",8,"Research & Report","Automated Machine Learning in the Era of Large Foundation Models","Automated Machine Learning advances AI by turning machine-learning pipeline construction into a search problem that can automatically choose architectures, optimizers, hyperparameters, and reasoning paths from a large space. This dissertation examines how AutoML interacts with large foundation models, focusing on recent generative breakthroughs such as large language models and diffusion models. It highlights emergent behaviors from scaling and develops AutoML methods to automate training and inference design, supporting more capable and robust model development toward Artificial General Intelligence.","UCLA  \nUCLA Electronic Theses and Dissertations  \nTitle  \nAutomated Machine Learning in the Era of Large Foundation Models  \nPermalink  \n[https://escholarship.org/uc/item/1vc4421f](https://escholarship.org/uc/item/1vc4421f)  \nAuthor  \nWang, Ruochen  \nPublication Date  \n2024  \nPeer reviewed|Thesis/dissertation  \n[eScholarship.org](eScholarship.org) Powered by the California Digital Library  \nUniversity of California  \nUniversity of California  \nLos Angeles  \nAutomated Machine Learning in the Era of Large Foundation Models  \nA dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Computer Science  \nby  \nRuochen Wang  \n2024  \n© Copyright by Ruochen Wang 2024  \nAbstract of the DISSERTATION  \nAutomated Machine Learning in the Era of Large Foundation Models  \nby  \nRuochen Wang  \nDoctor of Philosophy in Computer Science University of California, Los Angeles, 2024 Professor Cho-Jui Hsieh, Co-Chair  \nProfessor Wei Wang, Co-Chair  \nIntelligence, one of the most profound phenomena on Earth, has evolved over 600 million years, transforming from simple neural systems into human cognition capable of unraveling universal mysteries and creating silicon-based intelligence. This evolutionary process, with its inherent drive towards increasing complexity, seemingly defies thermodynamic principles, suggesting the existence of self-evolving mechanisms in life. Recreating such mechanisms within artificial systems is a crucial milestone on the path toward Artificial General Intelligence (AGI) .  \nAutomated Machine Learning (AutoML) represents a significant step in this direction. By enabling AI systems to optimize their own design processes, AutoML reformulates machine learning pipeline construction as a search problem, automating the selection of architectures, optimizers, hyperparameters, and even reasoning paths from an expansive search space. Despite its current limitations, AutoML has already demonstrated remarkable success in advancing machine  \nlearning across diverse applications.  \nThis thesis delves into the synergistic interplay between AutoML and large foundation models, particularly the recent breakthroughs in large-scale generative models like large language models (LLMs) and diffusion models. These models exhibit emergent behaviors, highlighting machine learning systems as complex entities where scaling produces unpredictable and potentially transformative capabilities. We emphasize the application of AutoML in automating the design of training and inference processes, crucial for the continued advancement of these models.  \nUltimately, we aspire that this research contributes a meaningful step towards the ambitious pursuit of Artificial General Intelligence (AGI) .  \nThe dissertation of Ruochen Wang is approved.  \nAditya Grover Baharan Mirzasoleiman Wei Wang, Committee Co-Chair Cho-Jui Hsieh, Committee Co-Chair  \nUniversity of California, Los Angeles 2024  \nTo my parents  \nv  \nTable of Contents  \n1 Introduction ................................ 1  \nI Training: Search for Architectures and Optimizers 3  \n2 Efficient and Robust Search Algorithm for Architectures ... 4  \n2.1 Problem Statement .......................... 4  \n2.2 Understanding Differentiable NAS .................. 6  \n2.3 Differentiable Architecture Search Framework ........... 6  \n2.4 Failure Mode Analysis of DARTS .................. 7  \n2.5 The pitfall of magnitude-based architecture selection in DARTS . 8  \n2.5.1 α may not represent the operation strength ........ 8  \n2.5.2 A case study: skip connection ................ 10  \n2.6 Improving the Effectiveness and Robustness of Differentiable NAS 13  \n2.7 Perturbation-based architecture selection .............. 14  \n2.7.1 Evaluating the strength of each operation ......... 14  \n2.7.2 The complete architecture selection process ......... 15  \n2.7.3 Experimental Evaluation ................... 15  \n2.8 Aligning architecture parameter with operation strength via distribution le","cbCaivu1ggCw3ApK","https://ap.wps.com/l/cbCaivu1ggCw3ApK","pdf",22667735,1,130,"English","en",105,"# Introduction\n## Training: Search for Architectures and Optimizers\n# Efficient and Robust Search Algorithm for Architectures\n## Problem Statement\n## Understanding Differentiable NAS\n## Differentiable Architecture Search Framework\n## Failure Mode Analysis of DARTS\n## The pitfall of magnitude-based architecture selection in DARTS\n## Improving the Effectiveness and Robustness of Differentiable NAS\n## Perturbation-based architecture selection\n## Aligning architecture parameter with operation strength via distribution learning\n## Beyond Differentiable NAS-Predictor-based Architecture Search\n## Predictor-based NAS\n## Limitations of differentiable architecture search\n# Generic Search Space\n## Problem settings of optimizer search\n## Efficient, scalable and generalizable framework for optimizer search\n## Empirical evaluations on a diverse set of tasks\n## Conclusion\n# Inference: From Model Selection to Prompt Optimization\n## When Model Selection Becomes Prompt Optimization\n## Prompt Optimization for (Multimodal) Large Language Models","[{\"question\":\"What does Automated Machine Learning (AutoML) automate in this thesis?\",\"answer\":\"AutoML reframes machine learning pipeline construction as a search problem, automating selection of architectures, optimizers, hyperparameters, and even reasoning paths from a large search space.\"},{\"question\":\"How does the thesis connect AutoML with large foundation models?\",\"answer\":\"It studies the synergistic interplay between AutoML and large generative foundation models, emphasizing how scaling leads to emergent behaviors and how AutoML can automate training and inference design.\"},{\"question\":\"What are the main research goals regarding Artificial General Intelligence (AGI)?\",\"answer\":\"The work aims to contribute meaningful progress toward the pursuit of AGI by improving how models are designed and optimized automatically for large foundation model settings.\"}]","Automated Machine Learning in the Era of Large Foundation Models | PDF",1785682934,328,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"automated-machine-learning-in-the-era-of-large-foundation-models","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/automated-machine-learning-in-the-era-of-large-foundation-models/118305/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What does Automated Machine Learning (AutoML) automate in this thesis?","Question",{"text":75,"@type":76},"AutoML reframes machine learning pipeline construction as a search problem, automating selection of architectures, optimizers, hyperparameters, and even reasoning paths from a large search space.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the thesis connect AutoML with large foundation models?",{"text":80,"@type":76},"It studies the synergistic interplay between AutoML and large generative foundation models, emphasizing how scaling leads to emergent behaviors and how AutoML can automate training and inference design.",{"name":82,"@type":73,"acceptedAnswer":83},"What are the main research goals regarding Artificial General Intelligence (AGI)?",{"text":84,"@type":76},"The work aims to contribute meaningful progress toward the pursuit of AGI by improving how models are designed and optimized automatically for large foundation model settings.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]