[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-118916-en":3,"doc-seo-118916-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},118916,1649267921044,"Ava Thompson","https://us-avatar.wpscdn.com/avatar/1800007509477c92dfb?_k=1782875107921204101",8,"Research & Report","Multi-Objective Hyperparameter Optimization in Machine Learning - An Overview","Multi-objective hyperparameter optimization addresses the need to tune machine learning pipelines beyond predictive accuracy by incorporating additional metrics and real-world constraints. The work introduces the fundamentals of multi-objective hyperparameter optimization, explains why it is often overlooked due to limited knowledge and software support, and motivates its value for applied ML. It surveys optimization strategies, spanning evolutionary algorithms and Bayesian optimization, and demonstrates utility across ML applications with objectives such as operating conditions, prediction time, sparsity, fairness, interpretability, and robustness.","arXiv :2206 .07438v 3 [ cs .LG] 6 Jun 2024  \nMulti-Objective Hyperparameter Optimization in Machine Learning – An Overview  \nFLORIAN KARL∗ , Fraunhofer Institut für integrierte Schaltungen, Germany TOBIAS PIELOK∗ , Ludwig-Maximilians-Universität München, Germany JULIA MOOSBAUER, Ludwig-Maximilians-Universität München, Germany FLORIAN PFISTERER, Ludwig-Maximilians-Universität München, Germany STEFAN COORS, Ludwig-Maximilians-Universität München, Germany MARTIN BINDER, Ludwig-Maximilians-Universität München, Germany LENNART SCHNEIDER, Ludwig-Maximilians-Universität München, Germany JANEK THOMAS, Ludwig-Maximilians-Universität München, Germany JAKOB RICHTER, Ludwig-Maximilians-Universität München, Germany MICHEL LANG, Technische Universität Dortmund, Germany  \nEDUARDO C. GARRIDO-MERCHÁN, Universidad Pontificia Comillas, Spain JUERGEN BRANKE, Warwick Business School, UK  \nBERND BISCHL, Ludwig-Maximilians-Universität München, Germany  \nHyperparameter optimization constitutes a large part of typical modern machine learning workflows. This arises from the fact that machine learning methods and corresponding preprocessing steps often only yield optimal performance when hyperparameters are properly tuned. But in many applications, we are not only interested in optimizing ML pipelines solely for predictive accuracy; additional metrics or constraints must be considered when determining an optimal configuration, resulting in a multi-objective optimization problem. This is often neglected in practice, due to a lack of knowledge and readily available software implementations for multi-objective hyperparameter optimization. In this work, we introduce the reader to the basics of multiobjective hyperparameter optimization and motivate its usefulness in applied ML. Furthermore, we provide an extensive survey of existing optimization strategies, both from the domain of evolutionary algorithmsand Bayesian optimization. We illustrate the utility of MOO in several specific ML applications, considering objectives such as operating conditions, prediction time, sparseness, fairness, interpretability and robustness.  \nCCS Concepts: • Computing methodologies → Supervised learning; • Theory of computation → Evolutionary algorithms; • Applied computing → Multi-criterion optimization and decision-making.  \nAdditional KeyWords and Phrases: Multi-Objective Hyperparameter Optimization, Neural Architecture Search, Bayesian Optimization  \n1 INTRODUCTION  \nWith the immense popularity of machine learning (ML) and data-driven solutions for many domains [227], the demand for automating the creation of suitable ML pipelines has strongly increased [129]. Automated machine learning (AutoML) and hyperparameter optimization (HPO) promise to simplify the ML process by enabling less experienced practitioners to optimally configure ML models for a variety of tasks-reducing manual effort and improving performance at the same  \n∗ Both authors contributed equally to this research.  \nAuthors’ addresses: Florian Karl, [florian.karl@iis.fraunhofer.de](florian.karl@iis.fraunhofer.de), Fraunhofer Institut für integrierte Schaltungen, Germany; Tobias Pielok, [tobias.pielok@stat.uni-muenchen.de](tobias.pielok@stat.uni-muenchen.de), Ludwig-Maximilians-Universität München, Germany; Julia Moosbauer, Ludwig-Maximilians-Universität München, Germany; Florian Pfisterer, Ludwig-Maximilians-Universität München, Germany; Stefan Coors, Ludwig-Maximilians-Universität München, Germany; Martin Binder, Ludwig-Maximilians-Universität München, Germany; Lennart Schneider, Ludwig-Maximilians-Universität München, Germany; Janek Thomas, LudwigMaximilians-Universität München, Germany; Jakob Richter, Ludwig-Maximilians-Universität München, Germany; Michel Lang, Technische Universität Dortmund, Germany; Eduardo C. Garrido-Merchán, Universidad Pontificia Comillas, Spain; Juergen Branke, Warwick Business School, UK; Bernd Bischl, Ludwig-Maximilians-Universität München, Germany.  \n2 Karl and Pielok, et al.  \n","cbCaiaBMmm6AZydj","https://ap.wps.com/l/cbCaiaBMmm6AZydj","pdf",1837284,1,50,"English","en",105,"# Introduction\n## Multi-objective perspective on hyperparameter optimization\n## Relation to automated machine learning and black-box/noisy optimization\n## Challenges of conflicting metrics and stakeholder requirements\n## Examples of objective trade-offs in practice","[{\"question\":\"Why does hyperparameter optimization often require a multi-objective formulation?\",\"answer\":\"Modern ML pipelines only achieve optimal performance when hyperparameters are properly tuned, but real deployments rarely rely on predictive accuracy alone. Multiple constraints and additional metrics must be considered, which leads to a multi-objective optimization problem.\"},{\"question\":\"What optimization approaches are surveyed in the work?\",\"answer\":\"The survey covers strategies from evolutionary algorithms and Bayesian optimization, addressing how these methods can handle multi-objective hyperparameter optimization.\"},{\"question\":\"What kinds of objectives are used in the presented ML applications?\",\"answer\":\"The work illustrates multi-objective optimization using objectives such as operating conditions, prediction time, sparseness, fairness, interpretability, and robustness.\"}]","Multi-Objective Hyperparameter Optimization in Machine Learning - An Overview | PDF",1785720937,126,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"multi-objective-hyperparameter-optimization-in-machine-learning-an-overview","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/multi-objective-hyperparameter-optimization-in-machine-learning-an-overview/118916/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why does hyperparameter optimization often require a multi-objective formulation?","Question",{"text":75,"@type":76},"Modern ML pipelines only achieve optimal performance when hyperparameters are properly tuned, but real deployments rarely rely on predictive accuracy alone. Multiple constraints and additional metrics must be considered, which leads to a multi-objective optimization problem.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What optimization approaches are surveyed in the work?",{"text":80,"@type":76},"The survey covers strategies from evolutionary algorithms and Bayesian optimization, addressing how these methods can handle multi-objective hyperparameter optimization.",{"name":82,"@type":73,"acceptedAnswer":83},"What kinds of objectives are used in the presented ML applications?",{"text":84,"@type":76},"The work illustrates multi-objective optimization using objectives such as operating conditions, prediction time, sparseness, fairness, interpretability, and robustness.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":21,"slug":113},6,"Technology","technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]