[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-128294-en":3,"doc-seo-128294-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},128294,962085570644,"Evangeline","https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d",8,"Research & Report","Scaling Mixed-Integer Optimization for Machine Learning Problems - Doctoral Dissertation","Imposing priors on machine learning models—such as interpretability, sparsity, or fairness—often converts otherwise tractable regression tasks into NP-hard optimization problems. This dissertation investigates how to build and exploit strong convex relaxations for NP-hard problems arising from discrete substructures in regression. The resulting relaxations enable scalable algorithms with high-quality solutions. Computational experiments show substantial improvements in runtime and empirical performance in sparse and fair regression, and the approach is extended to mixed-integer non-linear optimization problems with indicator variables.","UC Berkeley  \nUC Berkeley Electronic Theses and Dissertations  \nTitle  \nScaling Mixed-Integer Optimization for Machine Learning Problems  \nPermalink  \n[https://escholarship.org/uc/item/4rm0z4rt](https://escholarship.org/uc/item/4rm0z4rt)  \nISBN  \n9798293893607  \nAuthor  \nDeza, Anna  \nPublication Date  \n2025-08-01  \nPeer reviewed|Thesis/dissertation  \n[eScholarship.org](eScholarship.org) Powered by the California Digital Library  \nUniversity of California  \nScaling Mixed-Integer Optimization for Machine Learning Problems  \nBy  \nAnna Deza  \nA dissertation submitted in partial satisfaction of the requirements for the degree of  \nDoctor of Philosophy  \nin  \nIndustrial Engineering & Operations Research in the  \nGraduate Division  \nof the  \nUniversity of California, Berkeley  \nCommittee in charge:  \nProfessor Alper Atamt¨urk, Chair  \nProfessor Dorit Hochbaum  \nAssociate Professor Paul Grigas Associate Professor Peng Ding  \nSummer 2025  \nScaling Mixed-Integer Optimization for Machine Learning Problems  \nCopyright 2025  \nby  \nAnna Deza  \n1  \nAbstract  \nScaling Mixed-Integer Optimization for Machine Learning Problems  \nby  \nAnna Deza  \nDoctor of Philosophy in Industrial Engineering & Operations Research University of California, Berkeley  \nProfessor Alper Atamt¨urk, Chair  \nImposing priors or desirable properties on machine learning models, such as interpretability, sparsity, or fairness, can significantly enhance their practical utility. However, incorporating such requirements into the training procedure often transforms otherwise tractable problems, such as least squares regression, into NP-hard ones. Traditional approaches tackle this computational challenge by using convex proxy models which allow for efficient optimization but can lead to sub-optimal performance. Alternatively, exact formulations can be modeled with the addition of binary indicator variables, potentially leading to models with superior performance but typically suffer from poor scalability if approached naively.  \nIn this dissertation, we study how to develop and leverage strong convex relaxations for NPhard problems that arise in regression models with discrete substructures. These relaxations serve as a basis to develop scalable algorithms that provide high-quality solutions. Through computational experiments, we demonstrate significant improvements in runtime and empirical performance in sparse and fair regression problems. We also extend our studies to broader classes of mixed-integer non-linear optimization problems with indicator variables.  \nChapters 2 and 3 address scaling the exact solution times for ℓ0 − ℓ2 sparse regression models. We leverage strong convex relaxations to provide screening rules for logistic regression. We further generalize screening rules to logic rules which efficiently provide valid clique inequalities, improving branch-and-bound performance.  \nIn Chapter 4, we take a mixed-integer optimization perspective to solve fair linear regression. We propose an exact formulation for fair regression training and develop strong convex relaxations and algorithms that provide high-quality solutions efficiently.  \nFinally, in Chapter 5 we generalize and deepen preprocessing techniques for broader classes of mixed-integer nonlinear optimization problems with indicator variables. Our main contribution is a computational framework that extracts optimality conditions from strong convex  \n2  \nnonlinear relaxations, bridging their advantages with modern branch-and-bound solvers incapable of directly handling such relaxations at scale.  \ni  \nThis thesis is dedicated to papa, ima, and my four brothers.  \nii  \nContents  \nContents ii  \nList of Figures iv  \nList of Tables vi  \n1 Introduction 1  \n1.1 Machine Learning Problems with Discrete Structure .............. 1  \n1.2 Relevant Mixed-Integer Nonlinear Optimization Background ......... 3  \n1.3 Thesis Organization ................................ 7  \n2 Safe Screening for Logistic Regression with ℓ2–ℓ0 Regu","cbCaiqYYV3O36231","https://ap.wps.com/l/cbCaiqYYV3O36231","pdf",2512411,2,1,112,"English","en",105,"# 1 Introduction\n## 1.1 Machine Learning Problems with Discrete Structure\n## 1.2 Relevant Mixed-Integer Nonlinear Optimization Background\n## 1.3 Thesis Organization\n# 2 Safe Screening for Logistic Regression with ℓ2–ℓ0 Regularization\n## 2.1 Introduction\n## 2.2 Conic Reformulations\n## 2.3 Safe Screening Rules\n## 2.4 Computational Results\n## 2.5 Conclusion\n# 3 Logic Rules and Chordal Graphs for Sparse Learning\n## 3.1 Introduction\n## 3.2 Preliminaries\n## 3.3 Logic Rules\n## 3.4 Clique Inequalities\n## 3.5 Numerical Results\n## 3.6 Conclusion\n# 4 Fair and Accurate Regression: Strong Formulations and Algorithms\n## 4.1 Introduction\n## 4.2 MIO formulations for fair regression\n## 4.3 Coordinate descent algorithm\n## 4.4 Algorithms, comparisons and extensions\n## 4.5 Computational experiments\n## 4.6 Conclusion\n# 5 Two-Phase Framework for Nonlinear Mixed Integer Programs with Indicators\n## 5.1 Introduction\n## 5.2 Proposed Framework\n## 5.3 Application 1: Mixed-Integer Quadratic with Indicators\n## 5.4 Application 2: Sparse PCA\n## 5.5 Numerical Results\n## 5.6 Conclusion and Next Steps\n# 6 Conclusion\n# Bibliography","[{\"question\":\"Why do priors like sparsity or fairness make machine learning training harder?\",\"answer\":\"Incorporating such requirements into training often turns problems that are tractable in standard regression settings into NP-hard optimization problems.\"},{\"question\":\"What is the dissertation’s main methodological contribution?\",\"answer\":\"It studies how to develop and leverage strong convex relaxations for NP-hard regression problems with discrete substructures to build scalable algorithms that yield high-quality solutions.\"},{\"question\":\"What improvements are shown in the experimental results?\",\"answer\":\"Computational experiments demonstrate significant gains in runtime and empirical performance for sparse and fair regression settings, and the ideas extend to broader mixed-integer non-linear problems with indicator variables.\"}]","Scaling Mixed-Integer Optimization for Machine Learning Problems - Doctoral Dissertation | PDF",1785946663,282,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"scaling-mixed-integer-optimization-for-machine-learning-problems-doctoral-dissertation","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,48,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":20},"https://docshare.wps.com/document/","Document",{"item":49,"name":12,"@type":44,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/scaling-mixed-integer-optimization-for-machine-learning-problems-doctoral-dissertation/128294/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-30","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"Why do priors like sparsity or fairness make machine learning training harder?","Question",{"text":76,"@type":77},"Incorporating such requirements into training often turns problems that are tractable in standard regression settings into NP-hard optimization problems.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What is the dissertation’s main methodological contribution?",{"text":81,"@type":77},"It studies how to develop and leverage strong convex relaxations for NP-hard regression problems with discrete substructures to build scalable algorithms that yield high-quality solutions.",{"name":83,"@type":74,"acceptedAnswer":84},"What improvements are shown in the experimental results?",{"text":85,"@type":77},"Computational experiments demonstrate significant gains in runtime and empirical performance for sparse and fair regression settings, and the ideas extend to broader mixed-integer non-linear problems with indicator variables.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]