[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82322-en":3,"doc-seo-82322-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82322,1374391974564,"Clementine","https://ap-avatar.wpscdn.com/avatar/14000253aa45c000a9e?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779874745381141002",8,"Research & Report","Spectrally Deconfounded Gradient Boosting","Flexible machine learning methods can exploit associations induced by hidden confounders rather than stable signal, harming causal validity and robustness under distribution shift. Spectral deconfounding mitigates this by shrinking high-variance covariance directions that concentrate latent confounder information in dense confounding. This work develops a nonlinear framework for gradient boosting via a spectral loss, where deconfounding emerges from its interaction with regularization and early stopping, supported by mixed-model and empirical-Bayes interpretations and extensions to general likelihoods and kernel random effects.","arXiv :2607 .09371v1 [ stat .ML] 10 Jul 2026  \nSpectrally Deconfounded Gradient Boosting Andrea Nava∗†‡ Peter B¨uhlmann∗ Fabio Sigrist∗  \nAbstract  \nFlexible machine-learning methods can be sensitive to hidden confounding: they may learn associations induced by unobserved confounders rather than stable signals. Spectral deconfounding mitigates this problem by shrinking high-variance directions of the covariate matrix that, under dense confounding, carry latent confounder information. Existing work has largely focused on linear models. We develop a nonlinear spectral deconfounding framework for gradient boosting. Our approach replaces the ordinary squared-error loss by a spectral loss, which alters the boosting dynamics by slowing down learning in confounding-aligned directions. We show that deconfounding is not achieved by the spectral loss alone, but by the interaction between spectral shrinkage and regularization, especially in terms of early stopping. Moreover, we provide a mixed-model interpretation that connects LAVA-type shrinkage to random-effects adjustment and yields an empirical-Bayes procedure for tuning the spectral loss. We also extend the method to general likelihoods and nonlinear confounding using Laplace approximations and kernel random effects. Across synthetic and real-world experiments, spectrally deconfounded boosting improves estimation of the target function under hidden confounding and is substantially more scalable than existing nonlinear spectral deconfounding baselines.  \nKeywords: boosting, causal inference, confounding, random effects, mixed models  \n1 Introduction  \nCausal inference from observational data is fundamentally challenging in the presence of unobserved confounding. When latent variables influence both covariates and an outcome, naive estimation of the effect of covariates on the outcome is generally biased. Beyond causal estimation, hidden confounding can also harm predictive stability: flexible learners may exploit spurious associations that are predictive on the observed distribution but fail to generalize to new environments, thereby degrading robustness under distribution shift [B¨uhlmann, 2020] . Gradient boosting [Friedman et al., 2000, Friedman, 2001] is one of the most accurate methods for prediction on tabular data [Grinsztajn et al., 2022], but precisely because of its flexibility it is vulnerable to hidden confounding: when confounding-driven patterns are predictive in sample, boosting can fit them and thereby lose robustness under changes in the confounding structure. In this paper, we study how to improve the robustness of gradient boosting to hidden confounding. Since latent confounding cannot be resolved from observational data without additional structure, this requires explicit assumptions on how confounding enters the data-generating process.  \nWe consider the dense-confounding regime. The key idea is that when latent confounders affect many observed covariates simultaneously, their influence leaves a detectable imprint in the covariance geometry of the design matrix, typically concentrating in the directions corresponding to the largest singular values of the design matrix. This intuition already appears in genome-wide association studies (GWAS) through principal component adjustment, where a small number of top principal components are included to control for population structure [Price et al., 2006, Novembre et al., 2008], and in linear mixed models, which introduce random effects with covariance matrices derived from the design matrix and thereby adjust for confounding without explicitly selecting the number of principal components [Zhang et al., 2010, Sul et al., 2018] .  \n∗ Seminar for Statistics, ETH Z¨urich, Z¨urich, Switzerland  \n†IFZ, Lucerne University of Applied Sciences and Arts, Rotkreuz, Switzerland ‡Corresponding author: [andrea.nava@stat.math.ethz.ch](andrea.nava@stat.math.ethz.ch)  \nSpectral deconfounding [´Cevid et al. , 2020] takes this route. Rath","cbCaihjeWt7ovdJj","https://ap.wps.com/l/cbCaihjeWt7ovdJj","pdf",2758048,1,41,"English","en",105,"# Introduction\n## Dense-confounding motivation\n## Spectral deconfounding background\n## Contribution and approach","[{\"question\":\"Why can gradient boosting fail under hidden confounding?\",\"answer\":\"When latent confounders influence both covariates and outcomes, boosting can fit confounding-driven patterns that are predictive in-sample but do not generalize as confounding structure changes, reducing robustness under distribution shift.\"},{\"question\":\"How does spectral deconfounding work in this framework?\",\"answer\":\"Spectral deconfounding applies a spectral transformation that shrinks covariance directions aligned with high variance, which under dense confounding concentrate latent confounder information, and it is implemented in boosting through a spectral loss.\"},{\"question\":\"Is the spectral loss alone sufficient to achieve deconfounding?\",\"answer\":\"No. The document states that confounding is not removed by the spectral loss alone; robustness comes from the interaction between spectral shrinkage and regularization, especially through early stopping.\"}]",1784179612,103,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"spectrally-deconfounded-gradient-boosting","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/spectrally-deconfounded-gradient-boosting/82322/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why can gradient boosting fail under hidden confounding?","Question",{"text":75,"@type":76},"When latent confounders influence both covariates and outcomes, boosting can fit confounding-driven patterns that are predictive in-sample but do not generalize as confounding structure changes, reducing robustness under distribution shift.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does spectral deconfounding work in this framework?",{"text":80,"@type":76},"Spectral deconfounding applies a spectral transformation that shrinks covariance directions aligned with high variance, which under dense confounding concentrate latent confounder information, and it is implemented in boosting through a spectral loss.",{"name":82,"@type":73,"acceptedAnswer":83},"Is the spectral loss alone sufficient to achieve deconfounding?",{"text":84,"@type":76},"No. The document states that confounding is not removed by the spectral loss alone; robustness comes from the interaction between spectral shrinkage and regularization, especially through early stopping.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]