[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118939-en":3,"doc-seo-118939-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118939,3848291630094,"Emma Wilson","https://eur-avatar.wpscdn.com/davatar_085a072bc5b1113ac321206ff7593b45",8,"Research & Report","Structure-Driven Algorithm Design in Optimization and Machine Learning - Thesis/Dissertation","Structure-driven algorithm design studies how exploiting problem-specific structure can yield fast, practical optimization methods with strong performance guarantees. It addresses the limitations of generic methods such as simplex and gradient descent, which often converge slowly or return suboptimal solutions under broad regularity. The dissertation demonstrates how random perturbations can improve nonconvex gradient descent, and how special-structure solvers accelerate large-scale linear-program approaches for MDP and entropic optimal transport. It further develops algorithms for reliable minimax optimization, multi-agent learning, gradient-free nonsmooth nonconvex optimization, and adaptive game learning.","UC Berkeley  \nUC Berkeley Electronic Theses and Dissertations  \nTitle  \nStructure-Driven Algorithm Design in Optimization and Machine Learning  \nPermalink  \n[https://escholarship.org/uc/item/3196933p](https://escholarship.org/uc/item/3196933p)  \nAuthor  \nLin, Tianyi  \nPublication Date  \n2023  \nPeer reviewed|Thesis/dissertation  \n[eScholarship.org](eScholarship.org) Powered by the California Digital Library  \nUniversity of California  \nStructure-Driven Algorithm Design in Optimization and Machine Learning  \nBy  \nTianyi Lin  \nA dissertation submitted in partial satisfaction of the requirements for the degree of  \nDoctor of Philosophy  \nin  \nEngineering-Electrical Engineering and Computer Sciences  \nin the  \nGraduate Division  \nof the  \nUniversity of California, Berkeley  \nCommittee in charge:  \nProfessor Michael I. Jordan, Chair  \nProfessor Peter L. Bartlett  \nAssociate Professor Aditya Guntuboyina  \nSpring 2023  \nStructure-Driven Algorithm Design in Optimization and Machine Learning  \nCopyright 2023  \nby  \nTianyi Lin  \n1  \nAbstract  \nStructure-Driven Algorithm Design in Optimization and Machine Learning  \nBy  \nTianyi Lin  \nDoctor of Philosophy in Engineering-Electrical Engineering and Computer Sciences  \nUniversity of California, Berkeley  \nProfessor Michael I. Jordan, Chair  \nA textbook property of optimization algorithms is their ability to solve the problems under generic regularity conditions. Two examples are simplex method and gradient descent (GD) method. However, the performance of these fundamental and general-purpose optimization algorithms is often unsatisfactory; they often run slowly and perhaps return the suboptimal solutions in generic settings. In my view, this is the price of their generality; indeed, the generic algorithms are an achievement, but for many problems, the gains from leveraging special structure can be huge. A basic question then arises: how can we harness problem-speci􀀌c structure within our algorithms to obtain fast, practical algorithms with strong performance guarantees? As more structured data-driven decision-making models emerge, this question has become increasingly pressing and relevant to practitioners.  \nFor example, the GD is known to get stuck at a suboptimal saddle points in nonconvex optimization. Nonetheless, a line of recent works have shown that random initialization or perturbation changes the dynamics of GD and makes it provably converge to a global optimal solution. In addition, both Markov decision process (MDP) and discrete optimal transport (OT) problems can be solved using large-scale linear programs. Rather than using generic LP algorithms, the policy iteration and the Sinkhorn iteration exploit special structures in MDP and OT and thus perform better in practice. Adapting algorithms to problem-speci􀀌c structure is generally referred to as structure-driven algorithm design.  \nAlthough this line of research { which has been studied extensively for over 70 years { has enjoyed widespread success, the machine-learning success stories have introduced new formulations ripe for deep theoretical analysis and remarkable practical impact. My research pushes this frontier by identifying special structure of reliable machine learning (minimax optimization) and multi-agent machine learning (high-order optimization and beyond ) and design optimal algorithms for computing the appropriately de􀀌ned optimal solutions; and other structured problems, such as e􀀎cient entropic regularized optimal transport, gradientfree nonsmooth nonconvex optimization, and adaptive and doubly optimal learning in games.  \ni  \nTo my family  \nii  \nContents  \nContents ii  \nList of Figures v  \nList of Tables vii  \n1 Introduction 1  \n1.1 Motivation ..................................... 1  \n1.2 Overview of Our Results ............................. 6  \n1.3 Organization ................................... 9  \nI Minimax Optimization 11  \n2 Two-Timescale Gradient Descent Ascent 12  \n2.1 Introduction .....................","cbCaisQUYtogvvBM","https://ap.wps.com/l/cbCaisQUYtogvvBM","pdf",11042230,1,418,"English","en",105,"# Introduction\n## Motivation\n## Overview of Our Results\n## Organization\n# Minimax Optimization\n## Two-Timescale Gradient Descent Ascent\n## Related Works\n## Preliminaries\n## Main Results\n# Near-Optimal Gradient-Based Algorithm\n## Algorithm Components\n## Accelerating Convex-Concave Optimization\n## Accelerating Nonconvex-Concave Optimization\n# Riemannian Gradient-Based Algorithm\n## Motivating Examples\n## Riemannian Corrected Extragradient Method\n## Experiments\n## Conclusion","[{\"question\":\"Why do generic optimization algorithms like gradient descent often underperform?\",\"answer\":\"They are designed for broad regularity and generality, which can lead to slow runtime and suboptimal solutions in settings where stronger structure exists.\"},{\"question\":\"How does the dissertation connect structure-driven design to machine learning success?\",\"answer\":\"New structured formulations in machine learning enable deeper theory and large practical impact, motivating algorithms that exploit reliable machine learning and multi-agent learning structure for faster computation with guarantees.\"},{\"question\":\"What examples of structured problem settings are used to illustrate the approach?\",\"answer\":\"The work highlights perturbation-improved dynamics for nonconvex optimization, and structure-exploiting iterations for solving MDP and entropic regularized optimal transport more efficiently than generic linear-program solvers.\"}]","Structure-Driven Algorithm Design in Optimization and Machine Learning - Thesis/Dissertation | PDF",1785721110,1053,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"structure-driven-algorithm-design-in-optimization-and-machine-learning-thesisdissertation","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/structure-driven-algorithm-design-in-optimization-and-machine-learning-thesisdissertation/118939/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do generic optimization algorithms like gradient descent often underperform?","Question",{"text":75,"@type":76},"They are designed for broad regularity and generality, which can lead to slow runtime and suboptimal solutions in settings where stronger structure exists.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the dissertation connect structure-driven design to machine learning success?",{"text":80,"@type":76},"New structured formulations in machine learning enable deeper theory and large practical impact, motivating algorithms that exploit reliable machine learning and multi-agent learning structure for faster computation with guarantees.",{"name":82,"@type":73,"acceptedAnswer":83},"What examples of structured problem settings are used to illustrate the approach?",{"text":84,"@type":76},"The work highlights perturbation-improved dynamics for nonconvex optimization, and structure-exploiting iterations for solving MDP and entropic regularized optimal transport more efficiently than generic linear-program solvers.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]