[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119396-en":3,"doc-seo-119396-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119396,687197207057,"Sage","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","Efficient Algorithms for Modern Machine Learning Optimization - Dissertation Abstract","Machine learning model training is framed as optimization, yet the optimization landscape introduces new and complex difficulties. This dissertation tackles two related optimization problems: decentralized optimization arising in distributed training, and sum-of-minimum optimization appearing in mixed-model training. For decentralized optimization, a communication-optimal exact consensus scheme is constructed and integrated into decentralized stochastic gradient descent, yielding scalability with constant communication overhead and state-of-the-art transient iteration complexity. For sum-of-minimum optimization, a new connection to generalized clustering problems enables a two-phase method with strong initialization error control and proven convergence rates. Extensive numerical experiments validate empirical performance.","UCLA  \nUCLA Electronic Theses and Dissertations  \nTitle  \nEfficient Algorithms for Modern Machine Learning Optimization  \nPermalink  \n[https://escholarship.org/uc/item/9pt7h7xk](https://escholarship.org/uc/item/9pt7h7xk)  \nAuthor  \nDing, Lisang  \nPublication Date  \n2025  \nPeer reviewed|Thesis/dissertation  \n[eScholarship.org](eScholarship.org) Powered by the California Digital Library  \nUniversity of California  \nUNIVERSITY OF CALIFORNIA Los Angeles  \nEfficient Algorithms for Modern Machine Learning Optimization  \nA dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Mathematics  \nby  \nLisang Ding  \n2025  \n© Copyright by Lisang Ding 2025  \nABSTRACT OF THE DISSERTATION  \nEfficient Algorithms for Modern Machine Learning Optimization  \nby  \nLisang Ding  \nDoctor of Philosophy in Mathematics  \nUniversity of California, Los Angeles, 2025  \nProfessor Stanley J. Osher, Chair  \nThe training of machine learning models is a central topic in modern computational research and is often formulated as an optimization problem. However, the optimization landscape in machine learning presents new and complex challenges. This dissertation addresses two related optimization problems: decentralized optimization, which arises in distributed training settings, and sum-of-minimum optimization, which emerges in mixed-model training.  \nFor the decentralized optimization problem, we revisit and construct a communicationoptimal exact consensus scheme. This scheme is then judiciously integrated into the decentralized stochastic gradient descent algorithm. The proposed decentralized algorithm is scalable to any number of computing nodes and achieves state-of-the-art performance, which requires a transient iteration complexity of ˜O(n3 ) and a communication overhead of one.  \nFor the sum-of-minimum optimization problem, we identify a novel connection to generalized clustering problems. Leveraging this insight, we develop a two-phase algorithm. In the initialization phase, we generalize the k-means++ clustering method; in the iteration phase, we apply a variant of the Lloyd algorithm. Theoretically, a tight initialization error bound and a convergence rate are provided.  \nFor both problems, extensive numerical experiments are provided to illustrate the empirical performance of the proposed algorithms.  \nThe dissertation of Lisang Ding is approved.  \nGuido Francisco Mont´ufar Cuartas Ernest K. Ryu Hayden Kyler Schaeffer Wotao Yin  \nStanley J. Osher, Committee Chair  \nUniversity of California, Los Angeles 2025  \nTABLE OF CONTENTS  \n1 Introduction ...................................... 1  \n1.1 Background .................................... 1  \n1.2 Distributed and Decentralized Optimization .................. 4  \n1.3 Mixed-Model Optimization and Sum-of-Minimum Optimization ....... 5  \n1.4 Notations and Preliminaries ........................... 7  \n1.4.1 Notations ................................. 7  \n1.4.2 Convex analysis tools ........................... 7  \n2 Decentralized Optimization and Consensus Algorithm ........... 9  \n2.1 Introduction .................................... 10  \n2.2 Preliminaries and Related Work ......................... 15  \n2.2.1 Notations ................................. 15  \n2.2.2 Preliminary ................................ 15  \n2.2.3 Related work ............................... 18  \n2.3 Communication-Optimal Exact Consensus ................... 20  \n2.3.1 2-port optimal exact consensus ..................... 20  \n2.3.2 1-port optimal exact consensus ..................... 21  \n2.4 DSGD-CECA Algorithm ............................. 24  \n2.4.1 Algorithm development .......................... 24  \n2.4.2 Convergence analysis ........................... 26  \n2.5 Numerical Experiments .............................. 39  \n2.6 Chapter Conclusion ................................ 46  \n2.A Supplementary Materials on CECA ....................... 47  \n2.A.1 CECA for the 2-port message passing sys","cbCailaIFC46zRIT","https://ap.wps.com/l/cbCailaIFC46zRIT","pdf",1978337,1,135,"English","en",105,"# Introduction\n## Background\n## Distributed and Decentralized Optimization\n## Mixed-Model Optimization and Sum-of-Minimum Optimization\n## Notations and Preliminaries\n# Decentralized Optimization and Consensus Algorithm\n## Communication-Optimal Exact Consensus\n## DSGD-CECA Algorithm\n## Numerical Experiments\n## Chapter Conclusion\n# Sum-of-Minimum Optimization\n## Algorithms\n## Theoretical Analysis\n## Numerical Experiments\n## Chapter Conclusion\n# Conclusion\n## References","[{\"question\":\"What two optimization problems does the dissertation focus on?\",\"answer\":\"It focuses on decentralized optimization for distributed training and sum-of-minimum optimization that emerges in mixed-model training.\"},{\"question\":\"How is decentralized optimization addressed in the proposed method?\",\"answer\":\"The work constructs a communication-optimal exact consensus scheme and integrates it into a decentralized stochastic gradient descent algorithm, improving scalability and achieving state-of-the-art transient iteration complexity.\"},{\"question\":\"What is the key idea behind solving sum-of-minimum optimization?\",\"answer\":\"The dissertation links sum-of-minimum optimization to generalized clustering problems, then develops a two-phase algorithm that generalizes k-means++ for initialization and uses a variant of the Lloyd algorithm for iteration.\"}]","Efficient Algorithms for Modern Machine Learning Optimization - Dissertation Abstract | PDF",1785724087,340,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"efficient-algorithms-for-modern-machine-learning-optimization-dissertation-abstract","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/efficient-algorithms-for-modern-machine-learning-optimization-dissertation-abstract/119396/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What two optimization problems does the dissertation focus on?","Question",{"text":75,"@type":76},"It focuses on decentralized optimization for distributed training and sum-of-minimum optimization that emerges in mixed-model training.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How is decentralized optimization addressed in the proposed method?",{"text":80,"@type":76},"The work constructs a communication-optimal exact consensus scheme and integrates it into a decentralized stochastic gradient descent algorithm, improving scalability and achieving state-of-the-art transient iteration complexity.",{"name":82,"@type":73,"acceptedAnswer":83},"What is the key idea behind solving sum-of-minimum optimization?",{"text":84,"@type":76},"The dissertation links sum-of-minimum optimization to generalized clustering problems, then develops a two-phase algorithm that generalizes k-means++ for initialization and uses a variant of the Lloyd algorithm for iteration.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]