[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-81546-en":3,"doc-seo-81546-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},81546,687197100911,"Himbo","https://ap-avatar.wpscdn.com/avatar/a000239b6f1da00475?x-image-process=image/resize,m_fixed,w_180,h_180&k=1782698725881665579",8,"Research & Report","Information-Theoretic Bayesian Optimization for Bilevel Optimization Problems","Bilevel optimization models two nested problems where the lower-level optimum induces a constraint for the upper level. The study develops Bayesian optimization for the setting in which both levels are expensive black-box functions, addressing the limited prior focus versus other Bayesian optimization extensions. An information-theoretic bilevel information gain is introduced by jointly assessing information from upper- and lower-optimal solutions and their values. Practical lower-bound evaluation is provided, and empirical results on benchmark datasets validate effectiveness.","arXiv :2509 .2 1725v 3 [ cs .LG] 9 Jul 2026  \nInformation-Theoretic Bayesian Optimization for Bilevel Optimization Problems  \nTakuya Kanayama 1 , Yuki Ito 1 , Tomoyuki Tamura 1 , and Masayuki Karasuyama∗1  \n1 Nagoya Institute of Technology  \nAbstract  \nA bilevel optimization problem consists of two optimization problems nested as an upperand a lower-level problem, in which the optimality of the lower-level problem defines a constraint for the upper-level problem. This paper considers Bayesian optimization (BO) for the case that both the upper-and lower-levels involve expensive black-box functions. Because of its nested structure, bilevel optimization has a complex problem definition, by which bilevel BO has not been widely studied compared with other standard extensions of BO such as multi-objective or constraint problems. We propose an information-theoretic approach that considers the information gain of both the upper-and lower-optimal solutions and values. This enables us to define a unified criterion that measures the benefit for both level problems, simultaneously. Further, we also show a practical lower bound based approach to evaluating the information gain. We empirically demonstrate the effectiveness of our proposed method through several benchmark datasets.  \n1 Introduction  \nThe bilevel optimization is a standard formulation for a decision making problem that has a hierarchical structure. It consists of two optimization problems nested as an upper-and a lower-level problem, in which the optimality of the lower-level problem defines a constraint for the upper-level problem. Bilevel optimization techniques is applicable to hierarchical decision makings in a variety of contexts such as inverse optimal control (Suryan et al., 2016), chemical reaction optimization (Abbassi et al., 2021), and shape optimization (Herskovits et al., 2000) .  \nWe particularly focus on the case that both level problems are defined by expensive black-box functions, while most of existing studies assume that the lower level problem is not expensive to observe. For example, consider the case that the upper-and lower-objective functions are defined through simulators of a subject of interest, which can occur in a variety of scientific, engineering, and industrial fields. If these simulators consist of expensive computations (such as quantummechanical calculations), both level problems are expensive to observe. An example is that the simulator-based optimization of a physical property of inorganic crystals (computational materials design) under the stability constraint (energy minimization) can be seen as an instance of this class of problems.  \n∗ [karasuyama@nitech.ac.jp](karasuyama@nitech.ac.jp)  \nMost of existing BO studies for bilevel optimization applies BO only to the upper-level problem (e.g., Kieffer et al., 2017; Dogan and Prestwich, 2023) as pointed out by (Chew et al., 2025) . Typically, for a selected query to the upper-level problem, repeated queries to the lower-level problem is required, and/or further, the gradient of the lower-level problem is often assumed (e.g., Fu et al. , 2024) . A few studies (Islam et al., 2018; Wang et al., 2021) consider BO in both levels, but repeated queries on the lower-level is still required. These approaches are not fully suitable when both level problems are expensive black-boxes in which the gradient is not available. It is well-known that hyper-parameter optimization (HPO) of a machine learning model can be seen as a bilevel problem. However, the lower objective is usually inexpensive white-box function, and therefore, HPO is different from our focus, while there exist specific studies for HPO bilevel problems (e.g., Lorraine et al., 2020) . Further, some recent studies assume that the objectives are defined as an average (expectation) of some base functions and the stochastic samples of it are available, which can occur in problems such as HPO, federated learning, and adversarial attacks (e.g. , ","cbCaidy6b7H3jUIF","https://ap.wps.com/l/cbCaidy6b7H3jUIF","pdf",3030057,4,1,37,"English","en",105,"# Introduction\n## Problem formulation and bilevel structure\n## Expensive black-box both-level setting\n## Related work and limitations","[{\"question\":\"What is a bilevel optimization problem, and how do its two levels relate?\",\"answer\":\"A bilevel optimization problem nests an upper-level and a lower-level optimization. The optimality of the lower-level defines a constraint for the upper-level.\"},{\"question\":\"What challenge does the paper target compared with existing Bayesian optimization for bilevel problems?\",\"answer\":\"The focus is the case where both levels are expensive black-box functions and gradients are not available, unlike much prior work that assumes the lower level is easier or uses assumptions requiring lower-level access or gradients.\"},{\"question\":\"How does the proposed method measure improvement for both levels during Bayesian optimization?\",\"answer\":\"It introduces an information-theoretic criterion, called bilevel information gain, that jointly accounts for information from the upper- and lower-optimal solutions and the values, enabling a unified evaluation of benefit for both problems simultaneously.\"}]",1784174215,93,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"information-theoretic-bayesian-optimization-for-bilevel-optimization-problems","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":20},"https://docshare.wps.com/document/information-theoretic-bayesian-optimization-for-bilevel-optimization-problems/81546/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-25","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is a bilevel optimization problem, and how do its two levels relate?","Question",{"text":75,"@type":76},"A bilevel optimization problem nests an upper-level and a lower-level optimization. The optimality of the lower-level defines a constraint for the upper-level.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What challenge does the paper target compared with existing Bayesian optimization for bilevel problems?",{"text":80,"@type":76},"The focus is the case where both levels are expensive black-box functions and gradients are not available, unlike much prior work that assumes the lower level is easier or uses assumptions requiring lower-level access or gradients.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the proposed method measure improvement for both levels during Bayesian optimization?",{"text":84,"@type":76},"It introduces an information-theoretic criterion, called bilevel information gain, that jointly accounts for information from the upper- and lower-optimal solutions and the values, enabling a unified evaluation of benefit for both problems simultaneously.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]