Amortized Low-Rank Approximation for Hyperparameter Marginalization in PDE-Governed Bayesian Inverse Problems

Jul 16, 2026

Upload by Eliana

This paper addresses efficient solution of hierarchical Bayesian inverse problems with a high- or infinite-dimensional parameter field and a moderate number of hyperparameters. The parameter-to-observable map is a linear PDE, making the conditional problem Gaussian for fixed hyperparameters. Hyperparameter marginalization requires repeated marginal-density evaluations, dominated by large-scale log-determinant ratio computations and MAP estimations. A scalable framework using generalized low-rank approximations of prior-to-posterior precision updates is extended via two amortized variants, analyzed theoretically and compared computationally. Experiments on advection–diffusion inverse problems in 2D and 3D show 30–45x speedups for 100 marginal density evaluations in 3D.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages23

Reading Time58 min

Introduction

Problem statement

Related literature

arXiv :2607 .03355v1 [math .NA] 3 Jul 2026 AMORTIZED LOW-RANK APPROXIMATION FOR HYPERPARAMETER MARGINALIZATION IN PDE-GOVERNED BAYESIAN INVERSE PROBLEMS ∗ SONIA REILLY† AND GEORG STADLER† Abstract. This paper addresses the efficient solution of hierarchical Bayesian inverse problems with a high-or infinite-dimensional parameter field and a moderate number of hyperparameters. We focus on a class of problems in which the parameter-to-observable mapping is a linear PDE, so that, for fixed hyperparameters, the problem becomes conditionally Gaussian. Marginalizing the hyperparameters entails repeated evaluation of their marginal density, which in turn entails repeated large-scale log-determinant ratio computations and maximum a posteriori (MAP) estimations. To address these computational challenges, we introduce a scalable framework that relies on generalized low-rank approximations of the update from the prior to the posterior precision matrix. Though such a framework is state-of-the-art in non-hierarchical settings, in the case of nonlinear prior hyperparameters such ascovariance length scales, a direct application to hierarchical problems is inefficient. We propose two amortized variants of this framework, comparing their theoretical properties and computational complexity to the direct method. In numerical experiments, we evaluate their performance on an advection–diffusion initial condition inverse problem in two and three spatial dimensions, with hyperparameters in both the prior and the noise covariance. We find that our amortized methods achieve a factor of 30–45 speedup for 100 marginal density evaluations relative to the direct method in the 3D problem. Key words. hierarchical Bayesian inference, PDE-governed inverse problems, hyperparameter marginalization, low-rank approximation, discretization-invariant. AMS subject classifications. 65M32, 62F15, 35R30, 35Q62, 65F30 . 1. Introduction. In Bayesian inverse problems where the forward operator is costly to compute, often because it requires solving a PDE, one usually distinguishes between linear Gaussian settings and nonlinear, potentially non-Gaussian ones. In the linear Gaussian case, the posterior distribution is available in closed form and can be accurately approximated even when the parameter dimension is very large or infinite. The nonlinear case typically necessitates the use of MCMCor importance sampling, often combined with sophisticated proposal or dimension reduction techniques. An important class of problems involving hyperparameters lies between these two extremes. While considering the full set of parameters leads to a nonlinear problem, there is frequently an underlying linear structure: for fixed hyperparameters, the conditional distributions have a linear Gaussian form. If the hyperparameter dimension is moderate, this structure can be used to devise approximation methods that do not require sampling, or only require sampling in the space of hyperparameters. We now formalize the class of problems we consider. 1.1. Problem statement. We define a map A : X → Rq with X a separable Hilbert space and q ≥ 1. We consider the problem of inferring the parameter m ∈ X and a vector of hyperparameters θ ∈ Rk from observations y ∈ Rq using the linear relation (1.1) y = Am + ε , ∗ Version from July 7, 2026 . Funding: Partially supported by the Multidisciplinary University Research Initiatives (MURI) Program Office of Naval Research (ONR) grant \#N00014-19-1-242, by ONR \#N00014-26-1-2101, by the US National Science Foundation (NSF) under \#2411229, and by the U.S. Department of Energy, Office of Science, Office of ASCR, DOE Computational Science Graduate Fellowship under \#DE-SC0022158 . †Courant Institute School of Mathematics, Computing and Data Science, New York University, New York, USA, [sonia.reilly@nyu.edu](sonia.reilly@nyu.edu), [stadler@cims.nyu.edu](stadler@cims.nyu.edu) 2 SONIA REILLY, GEORG STADLER where ε is noise that corrupts th

FAQs

What class of Bayesian inverse problems does the paper focus on?

It studies hierarchical Bayesian inverse problems where the parameter-to-observable mapping is a linear PDE, so the conditional posterior is Gaussian for fixed hyperparameters.

Why is hyperparameter marginalization computationally expensive here?

Marginalizing the hyperparameters requires repeatedly evaluating the hyperparameter marginal density, which repeatedly triggers large-scale log-determinant ratio computations and MAP estimations.

What solution does the paper propose to reduce computational cost?

It introduces amortized variants of a scalable framework based on generalized low-rank approximations of the update from prior to posterior precision matrices.

Jul 16, 2026

Upload by Eliana