[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-128855-105":59,"doc-detail-128855-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","gaussian-covariance-and-scalable-variational-inference","Gaussian Covariance and Scalable Variational Inference","","The paper studies computational aspects of variational approximate inference for sparse linear models where scalable large-scale use requires efficient treatment of Gaussian covariance structure. It argues that exact covariance approximation is computationally difficult and that harmful independence factorizations can be avoided through data-dependent low-rank approximations. The work provides theoretical and empirical analysis of how low-rank covariance approximation errors affect decision outcomes in nonlinear sequential Bayesian experimental design.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/gaussian-covariance-and-scalable-variational-inference/128855/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/gaussian-covariance-and-scalable-variational-inference/128855.png","ImageObject",300,407,{"name":92,"@type":93},"Aria","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-09-20","2026-08-06",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",11,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"Why is scalability difficult for variational approximate inference in sparse linear models?","Question",{"text":113,"@type":114},"Because the posterior contains very many nonlocal dependencies, which in the large-scale continuous-variable setting are captured by Gaussian covariances. Approximating these covariances efficiently is computationally hard.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"How does the paper avoid problematic factorization assumptions?",{"text":118,"@type":114},"It uses data-dependent low-rank covariance approximations, tracking limited principal covariance directions instead of forcing dependencies into a predetermined factorized form.",{"name":120,"@type":111,"acceptedAnswer":121},"What effects do PCA-based low-rank covariance approximations have on inference outcomes?",{"text":122,"@type":114},"The paper shows that convergence is maintained when covariances are approximated by PCA, and that PCA approximation errors systematically strengthen sparsity regularization in the sparse linear model inference setting.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},128855,1786003927,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":39,"language":140,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":67,"update_tm":130,"read_time":46},2336474459895,"https://ap-avatar.wpscdn.com/avatar/22000baeef7a5ed0655?x-image-process=image/resize,m_fixed,w_180,h_180&k=1786071322749376916","View metadata, citation and similar [papers at ](papers at core.ac.uk)[core.ac.uk](papers at core.ac.uk) brought to you by CORE  \nprovided by Infoscience- École polytechnique fédérale de Lausanne  \nGaussian Covariance and Scalable Variational Inference  \nMatthias W. Seeger [mseeger@mmci.uni-saarland.de](mseeger@mmci.uni-saarland.de)  \n[Saarland University and Max Planck Institute for Informatics](Saarland University and Max Planck Institute for Informatics), [Campus E1.7](Campus E1.7) , [66123 Saarbruecken](66123 Saarbruecken), [Germany](Germany)  \nAbstract  \nWe analyze computational aspects of variational approximate inference techniques for sparse linear models, which have to be understood to allow for large scale applications. Gaussian covariances play a key role, whose approximation is computationally hard. While most previous methods gainscalability by not even representing most posterior dependencies, harmful factorization assumptions can be avoided by employing datadependent low-rank approximations instead.  \nWe provide theoretical and empirical insights into algorithmic and statistical consequences of low-rank covariance approximation errorson decision outcomes in nonlinear sequential Bayesian experimental design.  \n1. Introduction  \nSparse linear models (SLMs) enjoy enormous popularity in high-dimensional statistics, signal and image processing, and machine learning. A large part of this success story is due to regard for computational details: maximum a posteriori (MAP) estimation is formulated in terms of convex problems, which are reduced to standard primitives of numerical mathematics and digital signal processing. In such point estimation techniques, the Bayesian posterior is used as a criterion to be maximized rather than a distribution to be approximated and queried. Many applications require posterior information beyond its mode's location. Decision theory and Bayesian experimental design can be used to optimize sampling patterns (Seeger et al. , 2009) or data acquisition, and sparse bilinear model reconstruction is greatly improved by Bayesian averaging (Levin et al. , 2009) . However, today's approximate inference technology lags far behind MAP estimation in terms of scalability, robustness, and the-  \nAppearing in Proceedings of the 27th International Conference on Machine Learning, Haifa, Israel, 2010 . Copyright 2010 by the author(s)/owner(s) .  \noretical understanding.  \nIn this paper, we focus on computational aspects of scalable variational inference for large SLMs. Bayesian inference is hard and useful for the same underlying reason: the emergence of very many nonlocal dependencies in the posterior distribution. In the large scale continuous-variable context, these are approximated by Gaussian covariances of restricted structure and dimensionality. The choice of these restrictions not only impacts the 􀀌nal best 􀀌t to the posterior, but also the optimization process leading there. By far most methods to date attain scalability through factorization assumptions, whereby all dependencies are forced into a predetermined form, and most of them are ruled out up front. In contrast, Seeger et al. (2009) show how to avoid factorizations entirely, using low-rank covariance approximations such as PCA or the Lanczos algorithm (Schneider & Willsky, 2001) instead. The latter concept of tracking a limited number of principal covariance directions alongside the variational optimization has advantages in practice, since most Bayesian decision making or experimental design applications are driven by dominating modes of posterior dependencies.  \nWe point out the fundamental role of Gaussian (co)variance computations for large scale variational inference and experimental design in Section 2, and review approximation methods in Section 3. Our main contribution is an analysis of how low-rank Gaussian covariance approximations a􀀋ect inference outcomes in the framework of Seeger et al. (2009) . First, we prove that if covari","cbCaibQhIOVhT5ZJ","https://ap.wps.com/l/cbCaibQhIOVhT5ZJ","pdf",1908341,"English","# Introduction\n## Variational Inference for Sparse Linear Models\n### Posterior distribution for sparse inference","[{\"question\":\"Why is scalability difficult for variational approximate inference in sparse linear models?\",\"answer\":\"Because the posterior contains very many nonlocal dependencies, which in the large-scale continuous-variable setting are captured by Gaussian covariances. Approximating these covariances efficiently is computationally hard.\"},{\"question\":\"How does the paper avoid problematic factorization assumptions?\",\"answer\":\"It uses data-dependent low-rank covariance approximations, tracking limited principal covariance directions instead of forcing dependencies into a predetermined factorized form.\"},{\"question\":\"What effects do PCA-based low-rank covariance approximations have on inference outcomes?\",\"answer\":\"The paper shows that convergence is maintained when covariances are approximated by PCA, and that PCA approximation errors systematically strengthen sparsity regularization in the sparse linear model inference setting.\"}]","Gaussian Covariance and Scalable Variational Inference | PDF"]