[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-287051-105":3,"detail-sidebar-cat-1-en-105":80,"doc-detail-287051-en":126},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":73,"head_meta":75,"extra_data":77,"updated_unix":79},105,"en","scaling-multi-task-bayesian-optimization-with-large-language-models-paper-conference","SCALING MULTI-TASK BAYESIAN OPTIMIZATION WITH LARGE LANGUAGE MODELS - Paper Conference","","Multi-task Bayesian optimization transfers experience from previously optimized tasks to make optimizing new tasks more efficient. Traditional shared-surrogate approaches often saturate after only tens of tasks, limiting gains when scaling to hundreds or thousands. BOLT introduces an initialization-only transfer strategy: distill prior BO runs into a periodically fine-tuned LLM that proposes candidates for new tasks, while keeping test-time surrogate single-task. Evaluations on database query optimization and antimicrobial peptide design show steady improvement and faster, oracle-efficient BO performance.",{"@graph":14,"@context":72},[15,34,55],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/template/","Template",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/template/general/","General",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/template/scaling-multi-task-bayesian-optimization-with-large-language-models-paper-conference/287051/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/scaling-multi-task-bayesian-optimization-with-large-language-models-paper-conference/287051.png","ImageObject",442,249,{"name":42,"@type":43},"\tJames","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-09-20","2026-09-17",true,{"@type":52,"interactionType":53,"userInteractionCount":30},"InteractionCounter",{"@type":54},"ViewAction",{"@type":56,"mainEntity":57},"FAQPage",[58,64,68],{"name":59,"@type":60,"acceptedAnswer":61},"What problem does multi-task Bayesian optimization aim to solve?","Question",{"text":62,"@type":63},"It uses knowledge from optimizing existing related tasks to accelerate optimization of new, unseen tasks.","Answer",{"name":65,"@type":60,"acceptedAnswer":66},"How does BOLT differ from shared-surrogate multi-task BO methods?",{"text":67,"@type":63},"BOLT distills prior BO experience into an LLM for initialization only, while the test-time surrogate remains single-task and a standard single-task BO refines the candidates.",{"name":69,"@type":60,"acceptedAnswer":70},"On which domains is BOLT evaluated, and what is the outcome?",{"text":71,"@type":63},"BOLT is evaluated on database query optimization and antimicrobial peptide design, where LLM-generated initializations improve and accelerate BO, often matching or surpassing from-scratch BO with far fewer oracle calls after sufficient fine-tuning.","https://schema.org",{"og:url":32,"og:type":74,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":76,"canonical":32},"index,follow",{"doc_id":78,"site_id":7},287051,1789631836,{"code":4,"msg":81,"data":82},"success",[83,88,93,98,103,108,113,118,123],{"id":84,"doc_module":22,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},11,"Presentations",90,"presentations",{"id":89,"doc_module":22,"doc_module_name":25,"category_name":90,"show_sort_weight":91,"slug":92},12,"Resumes",80,"resumes",{"id":94,"doc_module":22,"doc_module_name":25,"category_name":95,"show_sort_weight":96,"slug":97},14,"Invoices",70,"invoices",{"id":99,"doc_module":22,"doc_module_name":25,"category_name":100,"show_sort_weight":101,"slug":102},15,"Posters",60,"posters",{"id":104,"doc_module":22,"doc_module_name":25,"category_name":105,"show_sort_weight":106,"slug":107},16,"Social Media",50,"social-media",{"id":109,"doc_module":22,"doc_module_name":25,"category_name":110,"show_sort_weight":111,"slug":112},17,"Forms",40,"forms",{"id":114,"doc_module":22,"doc_module_name":25,"category_name":115,"show_sort_weight":116,"slug":117},18,"Letters",30,"letters",{"id":119,"doc_module":22,"doc_module_name":25,"category_name":120,"show_sort_weight":121,"slug":122},21,"Paper Templates",5,"papers-templates",{"id":124,"doc_module":22,"doc_module_name":25,"category_name":29,"show_sort_weight":4,"slug":125},158,"general-158",{"code":4,"msg":81,"data":127},{"doc_id":78,"user_id":128,"nickname":42,"user_avatar":129,"doc_module":22,"category_id":124,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":130,"file_id":131,"file_url":132,"file_type":133,"file_size":134,"view_count":30,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":135,"language":136,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":137,"faqs":138,"seo_title":139,"seo_description":12,"update_tm":79,"read_time":140},2336474466412,"https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d","SCALING MULTI-TASK BAYESIAN OPTIMIZATION WITH LARGE LANGUAGE MODELS  \nYimeng Zeng 1 , Natalie Maus 1 , Haydn Thomas Jones 1 , Jeffrey Tao 1 , Fangping Wan2 , Marcelo Der Torossian Torres2 ,  \nCesar de la Fuente-Nunez2 , Ryan Marcus 1 , Osbert Bastani 1 , Jacob R. Gardner 1  \n1 Computer and Information Science, University of Pennsylvania  \n2Perelman School of Medicine, University of Pennsylvania [yimengz@seas.upenn.edu](yimengz@seas.upenn.edu)  \nABSTRACT  \nIn multi-task Bayesian optimization, the goal is to leverage experience from optimizing existing tasks to improve the efficiency of optimizing new ones. While approaches using multi-task Gaussian processes or deep kernel transfer exist, the performance improvement is marginal when scaling beyond a moderate number of tasks. We introduce BOLT, an initialization-only transfer strategy that distills prior BO runs into an LLM which proposes candidates for new tasks, while the surrogate at test time remains single-task. The LLM is periodically fine-tuned ontop solutions from completed runs, creating a closed loop where better BO outputs yield better initializations over time. This decoupled design scales to roughly 1500 tasks without the saturation observed for shared-surrogate MTBO and adds only a small, amortized overhead relative to the BO inner loops. We evaluate on two domains: database query optimization and antimicrobial peptide design. We demonstrate that LLM-generated initializations steadily improve and accelerate BO, and with sufficient fine-tuning, a few LLM samples often match or surpass full “from scratch” BO with far fewer oracle calls.  \n1 INTRODUCTION  \nMulti-task optimization seeks to use related, previously observed tasks to accelerate the optimization of new ones. Multi-task optimization appears naturally in a variety of domains where similar problems are encountered repeatedly, such as hyperparameter optimization, material science, database query optimization, and drug design. Formally, suppose we have tasks {1, 2 ,..., T}, each associated with its own objective function ft (x) . For each task t ∈ {1, 2 ,..., T}, we seek to find some x∗t such that  \nx∗t = arg minft (x) . (1)  \nx∈X  \nWe focus on the setting where, for each task, we have collected a dataset Dt of observations, and we wish to leverage this data when optimizing unseen test tasks.  \nMulti-task Bayesian optimization (BO) has traditionally learned across tasks by building a shared surrogate, typically via multi-output GPs and/or shared-weight neural feature extractors (Swersky et al., 2013; Perrone et al., 2018; Feurer, 2018; Patacchiola et al., 2020; Hakhamaneshi et al., 2022) . A standard approach involves placing a multi-output GP over the input-task space, decomposing the kernel as an input kernel k (x, x′) and a task kernel k (t, t′) . Despite their effectiveness, many of these methods—with the notable exception of recent work such as Wang et al. (2024b)—tend to saturate in performance after tens of training tasks and do not extract additional performance improvement on new tasks when given hundreds or thousands of related tasks.  \nWe propose Bayesian Optimization with LLM Transfer (BOLT), a straightforward approach to multitask BO that departs from the framework of building related task information into the BO surrogate model. Instead, as BO completes optimization for training tasks, we fine-tune a large language model (LLM) to, given a task description or context C[ft], generate solutions for that optimization problem that we can use as strong initialization for BO.  \nThis approach creates a self-reinforcing feedback loop: BO generates high-quality solutions that we can leverage to fine-tune the LLM; the fine-tuned LLM, in turn, produces better initializations that improve BO performance. Over time, the LLM learns to directly generate solutions that are highly competitive, enabling top-k-samples from the LLM (requiring just a few oracle calls) to outperform full “from scratch” BO runs (requirin","cbCaiumEwCo12cIT","https://ap.wps.com/l/cbCaiumEwCo12cIT","pdf",3898242,26,"English","# Abstract\n# Introduction\n## Contributions","[{\"question\":\"What problem does multi-task Bayesian optimization aim to solve?\",\"answer\":\"It uses knowledge from optimizing existing related tasks to accelerate optimization of new, unseen tasks.\"},{\"question\":\"How does BOLT differ from shared-surrogate multi-task BO methods?\",\"answer\":\"BOLT distills prior BO experience into an LLM for initialization only, while the test-time surrogate remains single-task and a standard single-task BO refines the candidates.\"},{\"question\":\"On which domains is BOLT evaluated, and what is the outcome?\",\"answer\":\"BOLT is evaluated on database query optimization and antimicrobial peptide design, where LLM-generated initializations improve and accelerate BO, often matching or surpassing from-scratch BO with far fewer oracle calls after sufficient fine-tuning.\"}]","SCALING MULTI-TASK BAYESIAN OPTIMIZATION WITH LARGE LANGUAGE MODELS - Paper Conference | PDF",9]