[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124776-en":3,"doc-seo-124776-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},124776,3848291630094,"Emma Wilson","https://eur-avatar.wpscdn.com/davatar_085a072bc5b1113ac321206ff7593b45",8,"Research & Report","Constrained Machine Learning - Algorithms and Models - Doctoral Thesis","Constrained machine learning focuses on designing efficient methods that incorporate known structure into machine learning models. The structure can come from problem formulation such as physical and aggregation constraints, or from desired model properties including energy efficiency, sparsity, and robustness. The thesis enforces such prior knowledge exactly to improve safety guarantees and to reduce training data and computation. It combines continuous constrained optimization and differentiable statistical modeling to develop convex constrained Frank-Wolfe variants and deep learning models with hard constraints via bi-level and differentiable optimization.","UC Berkeley  \nUC Berkeley Electronic Theses and Dissertations  \nTitle  \nConstrained machine learning: algorithms and models  \nPermalink  \n[https://escholarship.org/uc/item/92d86234](https://escholarship.org/uc/item/92d86234)  \nAuthor  \nNegiar, Geoffrey  \nPublication Date  \n2023  \nPeer reviewed|Thesis/dissertation  \n[eScholarship.org](eScholarship.org) Powered by the California Digital Library  \nUniversity of California  \nConstrained Machine Learning: Algorithms and Models  \nBy  \nGeoffrey Négiar  \nA dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in  \nEngineering – Electrical Engineering and Computer Science  \nin the  \nGraduate Division  \nof the  \nUniversity of California, Berkeley  \nCommittee in charge:  \nProfessor Laurent El Ghaoui, Co-chair Professor Michael Mahoney, Co-chair Professor Somayeh Sojoudi Assistant Professor Aditi Krishnapriyan  \nSummer 2023  \nConstrained Machine Learning: Algorithms and Models  \nCopyright 2023  \nby Geoffrey Négiar  \n1  \nAbstract  \nConstrained Machine Learning: Algorithms and Models  \nby  \nGeoffrey Négiar  \nDoctor of Philosophy in Engineering – Electrical Engineering and Computer Science  \nUniversity of California, Berkeley  \nProfessor Laurent El Ghaoui, Co-chair  \nProfessor Michael Mahoney, Co-chair  \nThis thesis is concerned with designing efficient methods to incorporate known structure in machine learning models. Structure arises either from problem formulation (e.g. physical constraints, aggregation constraints), or desirable model properties (energy efficiency, sparsity, robustness) . In many cases, the modeler has a certain knowledge about the system that they are modeling, which must be enforced in an exact manner. This can be necessary for providing adequate safety guarantees, or for improving the system’s efficiency: training a system with less data, or less computation costs. This thesis provides methods to do so ina variety of settings, by building on the two foundational fields of continuous, constrained optimization and of differentiable statistical modeling (also known as deep learning) .  \nThe first part of the thesis is centered on designing and analyzing efficient algorithms for optimization problems with convex constraints. In particular, it focuses on two variants of the Frank-Wolfe algorithm: the first variant proposes a fast backtracking-line search algorithm to adaptively set the step size in the full-gradient setting; the second variant proposes a fast stochastic Frank-Wolfe algorithm for constrained finite-sum problems. I also describe contributions to open-source constrained optimization software. The second part of this thesis is concerned with designing deep learning models which enforce certain constraints exactly: constraints based on physics, and aggregation constraints for probabilistic forecasting models. This part leverages bi-level optimization models, and differentiable optimization to constrain the output of a complex neural network. We demonstrate that complex non-linear constraints can be enforced on complex non-convex models, including probabilistic models.  \nThese examples showcase the power of hybrid models which couple data-driven learning and leverage complex nonlinear models such as deep neural networks, and well-studied optimization problems allowing for efficient algorithms. These hybrid models help highly flexible models to pick up structural patterns, achieving strong performance with sometimes no data access at all.  \ni  \nTo my grandparents, Jeannine, Charles, Louise and James.  \nTo my parents, Carol and Xavier.  \nii  \nContents  \nContents ii  \nList of Figures v  \nList of Tables x  \n1 Overview 1  \nI Algorithms 4  \n2 Linearly convergent Frank-Wolfe with backtracking line-search 5  \n2.1 Introduction .................................... 5  \n2.2 Methods ...................................... 7  \n2.3 Analysis ...................................... 11  \n2.4 Benchmarks ............................","cbCaid7zCz8vlhFd","https://ap.wps.com/l/cbCaid7zCz8vlhFd","pdf",3611560,1,152,"English","en",105,"# Overview\n# Linearly convergent Frank-Wolfe with backtracking line-search\n## Introduction\n## Methods\n## Analysis\n## Benchmarks\n## Conclusion and Future Work\n# Stochastic Frank-Wolfe for constrained finite-sum minimization\n## Introduction\n## Methods\n## Analysis\n## Stopping Criterion\n## Discussion\n## Implementation Details\n## Experiments\n## Conclusion and Future Work\n# Constrained Optimization Software\n## Introduction\n## Project Vision\n## Methods Currently Implemented\n## Underlying Technologies\n## Examples\n# Learning differentiable solvers for systems with hard constraints\n## Introduction\n## Background and Related work\n## Methods\n## Experimental results and implementation\n## Conclusions\n# Probabilistic forecasting\n## Introduction\n## Background and Related Work\n## Our Main Method\n## Empirical Evaluation\n## Conclusion","[{\"question\":\"What is the central goal of this thesis on constrained machine learning?\",\"answer\":\"To design efficient methods that incorporate known structure into machine learning models exactly, such as physical and aggregation constraints, improving safety guarantees and efficiency.\"},{\"question\":\"How does the thesis handle constraints in optimization-based parts?\",\"answer\":\"It develops and analyzes efficient algorithms for convex constrained optimization, focusing on Frank-Wolfe variants including a backtracking-line search approach and a stochastic Frank-Wolfe method for finite-sum problems.\"},{\"question\":\"How are hard constraints enforced in the deep learning model part?\",\"answer\":\"The thesis proposes deep learning models that enforce constraints exactly using bi-level optimization and differentiable optimization to constrain neural network outputs, including for probabilistic models.\"}]","Constrained Machine Learning - Algorithms and Models - Doctoral Thesis | PDF",1785894526,383,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"constrained-machine-learning-algorithms-and-models-doctoral-thesis","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/constrained-machine-learning-algorithms-and-models-doctoral-thesis/124776/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the central goal of this thesis on constrained machine learning?","Question",{"text":75,"@type":76},"To design efficient methods that incorporate known structure into machine learning models exactly, such as physical and aggregation constraints, improving safety guarantees and efficiency.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the thesis handle constraints in optimization-based parts?",{"text":80,"@type":76},"It develops and analyzes efficient algorithms for convex constrained optimization, focusing on Frank-Wolfe variants including a backtracking-line search approach and a stochastic Frank-Wolfe method for finite-sum problems.",{"name":82,"@type":73,"acceptedAnswer":83},"How are hard constraints enforced in the deep learning model part?",{"text":84,"@type":76},"The thesis proposes deep learning models that enforce constraints exactly using bi-level optimization and differentiable optimization to constrain neural network outputs, including for probabilistic models.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]