[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82179-en":3,"doc-seo-82179-105":29,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82179,687197207057,"Sage","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","Pitfalls and Remedies for Multi-Task Bayesian Optimization","Bayesian optimization commonly warm-starts target experiments using data from related source tasks, with multi-task Gaussian processes as the standard surrogate. This work revisits that default under controlled affine task structure and shows the textbook multi-task GP misestimates cross-task correlation, even when transfer should succeed. Two independent structural mechanisms drive the failure: per-task standardization propagates finite-sample alignment error, and marginal-likelihood correlation identification is diluted by non-overlapping designs. Three conservative remedies—learning per-task means/scales, constraining task covariance to non-negative correlations, and co-locating source/target queries—recover target-only performance on simple instances, while harder cases still show negative transfer patterns across variants.","Pitfalls and Remedies for Multi-Task Bayesian  \nOptimization  \nCarl Hvarfner  \nMeta [hvarfner@meta.com](hvarfner@meta.com)  \nSam Daulton  \nMeta [sdaulton@meta.com](sdaulton@meta.com)  \nMax Balandat  \nMeta [balandat@meta.com](balandat@meta.com)  \narXiv :2607 .09073v 1 [ cs .LG] 10 Jul 2026  \nEytan Bakshy  \nMeta  \n[ebakshy@meta.com](ebakshy@meta.com)  \nAbstract  \nBayesian optimization routinely warm-starts a target experiment with data from related source tasks, and the multi-task Gaussian process is the textbook surrogate for the job. We revisit this default in a controlled setting and find that it misestimates the cross-task correlation even in the simplest non-trivial case, affinely related source and target tasks, where a working transfer-learning method should obviously succeed. We trace the failure to two independent structural mechanisms. Per-task standardization, the textbook fix for the affine slice ambiguity, propagates a finite-sample alignment error into the recovered correlation. The marginal likelihood itself identifies the correlation only at a per-sample rate that a Gaussian process at non-overlapping designs further dilutes. We propose three conservative remedies that follow from the analysis: promoting per-task means and scales to model parameters, restricting the task covariance to non-negative correlations, and co-locating part of the source and target designs. Across synthetic multi-task problems and surrogate-based hyperparameter-tuning transfer, these remedies recover the target-only baseline on the simple instances, while the broader failure persists on harder instances and across most rank-based and latent-context variants.  \n1 Introduction  \nBayesian optimization (BO) [10, 12, 27, 28] is a workhorse for sample-efficient experimentation, and in production settings, target experiments rarely arrive in isolation [9, 13, 21] . BO transfer learning (BOTL) is the default story whenever a target experiment has a related predecessor: gather the source data, fit a multi-task Gaussian process (MTGP), and expect fewer target evaluations to find the optimum. Reports on BOTL tend to emphasize positive results on curated suites [4, 9, 26], and library defaults [3, 11] present the Intrinsic Coregionalization Model (ICM)-based MTGP as the go-to surrogate. Controlled comparisons against target-only BO [8, 22], and direct audits of whether the standard model recovers correct task correlations, remain comparatively rare.  \nYet on two affinely related tasks drawn from a standard benchmark function, the textbook MTGP misestimates the cross-task correlation: it attenuates the recovered correlation and, once more than one source is present, can even flip its sign. Affinely related tasks are the textbook example a working transfer-learning method must handle: the source is a perfect linear image of the target, so every standard MTGP variant should recover near-perfect  \nPreprint. Under review.  \ncorrelations and transfer should obviously help. We find the opposite: similar pathologies persist not only for the textbook ICM but for nearly every multi-task and rank-based variant in common use, and across our multi-task BO grid the textbook MTGP loses to a singletask Gaussian process (GP) on most base functions – the textbook signature of negative transfer [19, 33] . A method failing here is failing structurally, not because the task is hard; prior BOTL benchmarks report ICM-based MTGPs on suites where the underlying transferability is itself uncertain, so a clean failure on affinely related tasks is diagnostic. We trace the failure to a structural identifiability defect in the standard parameterization, not a fitting accident.  \nThe failure has two structurally distinct, independent causes: a finite-sample per-task standardization error that rides the affine reparameterization symmetry into the recovered correlation, and an information-theoretic floor on correlation inference that a GP at nonoverlapping designs further dilutes. B","cbCaikP8zVnMDomd","https://ap.wps.com/l/cbCaikP8zVnMDomd","pdf",978399,1,19,"English","en",105,"# Introduction\n# Background","[{\"question\":\"What remedies does the paper propose to reduce negative transfer on simple affine instances?\",\"answer\":\"It proposes three conservative fixes: model per-task means and scales explicitly, restrict task covariance to non-negative correlations, and co-locate parts of the source and target query designs to reduce inference variance.\"}]",1784178615,48,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":27},"pitfalls-and-remedies-for-multi-task-bayesian-optimization","",{"@graph":35,"@context":77},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/pitfalls-and-remedies-for-multi-task-bayesian-optimization/82179/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"What remedies does the paper propose to reduce negative transfer on simple affine instances?","Question",{"text":75,"@type":76},"It proposes three conservative fixes: model per-task means and scales explicitly, restrict task covariance to non-negative correlations, and co-locate parts of the source and target query designs to reduce inference variance.","Answer","https://schema.org",{"og:url":51,"og:type":79,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":81,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,102,107,112,115,120,123,127],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":45,"category_name":99,"show_sort_weight":100,"slug":101},5,"Comic",60,"comic",{"id":103,"doc_module":4,"doc_module_name":45,"category_name":104,"show_sort_weight":105,"slug":106},6,"Technology",50,"technology",{"id":108,"doc_module":4,"doc_module_name":45,"category_name":109,"show_sort_weight":110,"slug":111},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":113,"slug":114},30,"research-report",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":45,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":98,"slug":129},"General","general"]