[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-1-en-105":3,"doc-seo-281411-105":53,"doc-detail-281411-en":126},{"code":4,"msg":5,"data":6},0,"success",[7,14,19,24,29,34,39,44,49],{"id":8,"doc_module":9,"doc_module_name":10,"category_name":11,"show_sort_weight":12,"slug":13},11,1,"Template","Presentations",90,"presentations",{"id":15,"doc_module":9,"doc_module_name":10,"category_name":16,"show_sort_weight":17,"slug":18},12,"Resumes",80,"resumes",{"id":20,"doc_module":9,"doc_module_name":10,"category_name":21,"show_sort_weight":22,"slug":23},14,"Invoices",70,"invoices",{"id":25,"doc_module":9,"doc_module_name":10,"category_name":26,"show_sort_weight":27,"slug":28},15,"Posters",60,"posters",{"id":30,"doc_module":9,"doc_module_name":10,"category_name":31,"show_sort_weight":32,"slug":33},16,"Social Media",50,"social-media",{"id":35,"doc_module":9,"doc_module_name":10,"category_name":36,"show_sort_weight":37,"slug":38},17,"Forms",40,"forms",{"id":40,"doc_module":9,"doc_module_name":10,"category_name":41,"show_sort_weight":42,"slug":43},18,"Letters",30,"letters",{"id":45,"doc_module":9,"doc_module_name":10,"category_name":46,"show_sort_weight":47,"slug":48},21,"Paper Templates",5,"papers-templates",{"id":50,"doc_module":9,"doc_module_name":10,"category_name":51,"show_sort_weight":4,"slug":52},158,"General","general-158",{"code":4,"msg":54,"data":55},"ok",{"site_id":56,"language":57,"slug":58,"title":59,"keywords":60,"description":61,"schema_data":62,"social_meta":119,"head_meta":121,"extra_data":123,"updated_unix":125},105,"en","large-scale-multi-task-bayesian-optimization-with-large-language-models-preprint","Large Scale Multi-Task Bayesian Optimization with Large Language Models - Preprint","","Multi-task Bayesian optimization aims to transfer experience from previously optimized tasks to accelerate finding solutions for new ones. Many Gaussian-process and deep-kernel transfer methods saturate once the number of tasks reaches tens, limiting gains from hundreds or thousands of related tasks. This work introduces BOLT, fine-tuning a large language model on high-quality Bayesian optimization trajectories and using it to propose initialization points in a closed feedback loop. Experiments on database query optimization and antimicrobial peptide design show progressively improving initializations that reduce oracle calls while achieving better few-shot performance than from-scratch BO.",{"@graph":63,"@context":118},[64,80,101],{"@type":65,"itemListElement":66},"BreadcrumbList",[67,71,74,77],{"item":68,"name":69,"@type":70,"position":9},"https://docshare.wps.com","Home","ListItem",{"item":72,"name":10,"@type":70,"position":73},"https://docshare.wps.com/template/",2,{"item":75,"name":11,"@type":70,"position":76},"https://docshare.wps.com/template/presentations/",3,{"item":78,"name":59,"@type":70,"position":79},"https://docshare.wps.com/template/large-scale-multi-task-bayesian-optimization-with-large-language-models-preprint/281411/",4,{"url":78,"name":59,"@type":81,"image":82,"author":87,"headline":59,"publisher":90,"fileFormat":93,"inLanguage":57,"description":61,"dateModified":94,"datePublished":95,"encodingFormat":93,"isAccessibleForFree":96,"interactionStatistic":97},"DigitalDocument",{"url":83,"@type":84,"width":85,"height":86},"https://docshare.wps.com/thumbnails/large-scale-multi-task-bayesian-optimization-with-large-language-models-preprint/281411.png","ImageObject",442,249,{"name":88,"@type":89},"Margaret","Person",{"url":68,"name":91,"@type":92},"DocShare","Organization","application/pdf","2026-11-04","2026-09-16",true,{"@type":98,"interactionType":99,"userInteractionCount":45},"InteractionCounter",{"@type":100},"ViewAction",{"@type":102,"mainEntity":103},"FAQPage",[104,110,114],{"name":105,"@type":106,"acceptedAnswer":107},"What problem does multi-task Bayesian optimization address?","Question",{"text":108,"@type":109},"It seeks to leverage knowledge from previously optimized related tasks to speed up and improve optimization on new tasks with limited data.","Answer",{"name":111,"@type":106,"acceptedAnswer":112},"How does BOLT use large language models within Bayesian optimization?",{"text":113,"@type":109},"After BO finds high-quality solutions for training tasks, BOLT fine-tunes an LLM on those solutions and uses the tuned LLM to generate initialization points for BO on new tasks.",{"name":115,"@type":106,"acceptedAnswer":116},"What are the evaluation domains and key results?",{"text":117,"@type":109},"BOLT is evaluated on database query plan optimization and antimicrobial peptide design. The results show a positive feedback loop where LLM-generated initializations improve over time and can outperform from-scratch BO while requiring fewer oracle calls.","https://schema.org",{"og:url":78,"og:type":120,"og:title":59,"og:site_name":91,"og:description":61},"article",{"robots":122,"canonical":78},"index,follow",{"doc_id":124,"site_id":56},281411,1791448012,{"code":4,"msg":5,"data":127},{"doc_id":124,"user_id":128,"nickname":88,"user_avatar":129,"doc_module":9,"category_id":8,"category_name":11,"doc_title":59,"doc_description":61,"doc_content":130,"file_id":131,"file_url":132,"file_type":133,"file_size":134,"view_count":45,"is_deleted":4,"is_public":9,"is_downloadable":9,"audit_status":9,"page_count":40,"language":135,"language_code":57,"site_id":56,"html_lang":57,"table_of_contents":136,"faqs":137,"seo_title":138,"seo_description":61,"update_tm":139,"read_time":140},137451207643,"https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2","arXiv :2503 .08131v2 [ cs .LG] 12 Jun 2025  \nLarge Scale Multi-Task Bayesian Optimization with Large Language Models  \nYimeng Zeng 1 Natalie Maus 1 Haydn Thomas Jones 1 Jeffrey Tao 1 Fangping Wan2 Marcelo Der Torossian Torres2  \nCesar de la Fuente-Nunez2 Ryan Marcus 1 Osbert Bastani 1 Jacob R. Gardner 1  \n1 Computer and Information Science, UPenn 2Perelman School of Medicine, UPenn  \n[yimengz@seas.upenn.edu](yimengz@seas.upenn.edu)  \nAbstract  \nIn multi-task Bayesian optimization, the goal is to leverage experience from optimizing existing tasks to improve the efficiency of optimizing new ones. While approaches using multi-task Gaussian processes or deep kernel transfer exist, the performance improvement is marginal when scaling beyond a moderate number of tasks. We introduce a novel approach leveraging large language models (LLMs) to learn from, and improve upon, previous optimization trajectories, scaling to approximately 1500 distinct tasks. Specifically, we propose a feedback loop in which an LLM is fine-tuned on the high quality solutions to specific tasks found by Bayesian optimization (BO) . This LLM is then used to generate initialization points for future BO searches for new tasks. The trajectories of these new searches provide additional training data for fine-tuning the LLM, completing the loop. We evaluate our method on two distinct domains: database query optimization and antimicrobial peptide design. Results demonstrate that our approach creates a positive feedback loop, where the LLM’s generated initializations gradually improve, leading to better optimization performance. As this feedback loop continues, we find that the LLM is eventually able to generate solutions to new tasks in just a few shots that are better than the solutions produced by “from scratch” by Bayesian optimization while simultaneously requiring significantly fewer oracle calls.  \n1 Introduction  \nMulti-task optimization seeks to use related, previously solved tasks to accelerate the optimization of new ones. Multi-task optimization appears naturally in a variety of domains where similar problems are encountered repeatedly, such as hyperparameter optimization, material science, database query optimization, and drug design. Formally, suppose we have tasks {1, 2 ,..., T}, each associated with its own objective function ft (x) . For each task t ∈ {1, 2 ,..., T}, we seek to find some x∗t such that  \nx∗t = arg minft (x) . (1)  \nx∈X  \nWe focus on the setting where, for each task, we have collected a dataset Dt of observations, and we wish to leverage this data when optimizing unseen test tasks.  \nMulti-task Bayesian optimization (BO) methods have predominantly used Gaussian processes (GPs) and/or a variety of shared weight neural network feature extractors to jointly learn correlations across tasks [40, 31, 30, 10] . A standard approach involves placing a multi-output GP over the input-task space, decomposing the kernel as an input kernel k(x, x′) and a task kernel k(t, t′) . Despite their effectiveness, many of these methods—with the notable exception of recent work such as Wanget al. [47]—tend to saturate in performance after tens of training tasks and do not extract additional performance improvement on new tasks when given hundreds or thousands of related tasks.  \nPreprint. Under review.  \nWe propose Bayesian Optimization with LLM Transfer (BOLT), a straightforward approach to multitask BO that departs from the framework of building related task information into the BO surrogate model. Instead, as BO completes optimization for training tasks, we fine-tune a large language model (LLM) to, given a task description or context C[ft], generate solutions for that optimization problem that we can use as strong initialization for BO.  \nThis approach creates a self-reinforcing feedback loop: BO generates high-quality solutions that we can leverage to fine-tune the LLM; the fine-tuned LLM, in turn, produces better initializations that improve BO performan","cbCaiu3RJIlHRmYe","https://ap.wps.com/l/cbCaiu3RJIlHRmYe","pdf",3982975,"English","# Abstract\n# 1 Introduction\n## Application 1: Database query plan optimization\n## Application 2: Antimicrobial peptide (AMP) design","[{\"question\":\"What problem does multi-task Bayesian optimization address?\",\"answer\":\"It seeks to leverage knowledge from previously optimized related tasks to speed up and improve optimization on new tasks with limited data.\"},{\"question\":\"How does BOLT use large language models within Bayesian optimization?\",\"answer\":\"After BO finds high-quality solutions for training tasks, BOLT fine-tunes an LLM on those solutions and uses the tuned LLM to generate initialization points for BO on new tasks.\"},{\"question\":\"What are the evaluation domains and key results?\",\"answer\":\"BOLT is evaluated on database query plan optimization and antimicrobial peptide design. The results show a positive feedback loop where LLM-generated initializations improve over time and can outperform from-scratch BO while requiring fewer oracle calls.\"}]","Large Scale Multi-Task Bayesian Optimization with Large Language Models - Preprint | PDF",1789566038,6]