Structure-Guided Gauss-Newton Method for Linear Advection-Reaction Equation

Jul 16, 2026

Least-squares neural network (LSNN) techniques for linear advection–reaction equations can approximate discontinuous solutions without prior knowledge of interface location, but they produce a non-convex and computationally intensive training problem. This paper introduces a structure-guided Gauss–Newton (SgGN) approach that alternates between linear (output) parameters solved via a linear solver and nonlinear (hidden-layer) parameters updated by a modified Gauss–Newton method that removes GN matrix singularities explicitly. Numerical results on the benchmark problems from prior work show superiority over Adam in both accuracy and cost.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages20

Reading Time50 min

Introduction

Problem setting and least-squares formulation

LSNN background and optimization challenge

Gauss–Newton and structure-guided strategy

arXiv :2607 .07506v2 [math .NA] 9 Jul 2026 STRUCTURE-GUIDED GAUSS-NEWTON METHOD: LINEAR ADVECTION-REACTION EQUATION ZHIQIANG CAI∗ AND C´ESAR HERRERA∗ Abstract. The least-squares neural network (LSNN) method introduced in [5] for linear advection-reaction equations is capable of accurately approximating discontinuous solutions without a priori knowledge of the interface location. However, the resulting discretization is a non-convex optimization problem that is computationally intensive and complex. In this paper, we propose a structure-guided Gauss-Newton (SgGN) method that alternates between the linear (output) and the nonlinear (hidden layer) parameters. At each outer iteration, the linear parameters are computed by a linear solver, and the nonlinear parameters are updated by a modified Gauss-Newton (GN) method that explicitly removes the singularities of the GN matrix. Numerical experiments for all test problems presented in [5] show that the SgGN method is superior to the Adam optimizer [13], the commonly used first-order optimization algorithm, not only in computational cost but, more importantly, in accuracy. Key words. Advection-reaction equation, Least-squares method, ReLU neural network, Gauss-Newton method 1. Introduction. Let Ω be a bounded open domain in R2 , and let β(x) = (β1 ,β2 )T ∈ C1 (¯Ω)2 and γ ∈ C (¯Ω) be the given advective velocity field and reaction coefficient, respectively. Denote by Γ − = {x ∈ Γ : β (x)·n(x) < 0} the inflow part of the boundary Γ = ∂Ω, where n (x) is the unit outward vector normal to Γ atx ∈ Γ . Without loss of generality, assume that the magnitude of β(x) is one in Ω, i.e. , |β(x)| = 1 for x ∈ Ω . Consider the following linear advection-reaction equation (1.1) 􀀚 uβ + γuu fg inon ΩΓ,− , where f ∈ L2 (Ω) and g ∈ L2 (Γ − ) are given scalar-valued functions, and uβ is the directional derivative of u along the direction β defined by u (x) − u 􀀀x − τβ(x)􀀁 (1.2) uβ (x) = Dβ u (x) = lim ~~ ~~ τ→0 τ . Note that (1.1) holds for discontinuous solutions on discontinuity interfaces, while β · ∇u is invalid (see [15] for details) . Denote the solution space by Vβ = 􀀈v ∈ L2 (Ω) : vβ ∈ L2 (Ω) 􀀉 , equipped with the norm ∥v∥β = 􀀀∥v∥20 ,Ω + ∥vβ ∥20 ,Ω 􀀁 1/2 . Denote the weighted L2 (Γ − ) norm on the inflow boundary by ∥v∥0 ,β ,Γ − = ZΓ − |β·n|v2 ds! 1/2 . Introducing the following least-squares functional (1.3) L (v;f) = ∥vβ + γ v − f∥20 ,Ω + ∥v − g∥20 ,β ,Γ − , ∀ v ∈ Vβ ∗ Department of Mathematics, Purdue University, West Lafayette, IN ([caiz@purdue.edu](caiz@purdue.edu), [herre125@purdue.edu](herre125@purdue.edu)). with f = (f, g), then the least-squares formulation of problem (1.1) is to seek u ∈ Vβ such that (1.4) L (u;f) = min L (v;f) . v∈Vβ When the solution is smooth, this formulation was studied in [2, 10 , 3], and the coercivity of the homogeneous least-squares functional was established under the assumption that there exists a positive constant γ0 > 0 such that 1 γ (x) − 2 ∇ · β ≥ γ0 , ∀ x ∈ Ω . The least-squares ReLU neural network (LSNN) method was introduced in [5] for solving the linear advection-reaction equation in (1.1) with discontinuous solutions. The LSNN is one of the physics-preserving neural network (P2 NN) methods [16] and is specially designed for problems without natural minimization principle. Based on the L2-norm least-squares formulation in (1 .3), this method employs ReLU neural networks (NN) as approximating functions and physics-preserving numerical differentiation operator. Without using a priori knowledge of the discontinuity interface location, the LSNN is capable of approximating discontinuous solutions without oscillation andovershooting. The LSNN method was also developed for scalar nonlinear hyperbolic conservation laws (see [4]) . Despite the impressive approximation capabilities of NNs, the resulting discretization leads to a non-convex optimization problem in the NN parameters. The methods of gradi

FAQs

What limitation does LSNN have when applied to linear advection–reaction equations?

The LSNN discretization leads to a non-convex optimization problem in neural network parameters, making training computationally intensive and complex.

How does the structure-guided Gauss–Newton (SgGN) method update network parameters?

It alternates: at each outer iteration it computes linear (output) parameters with a linear solver, and updates nonlinear (hidden-layer) parameters using a modified Gauss–Newton method.

Why is the Gauss–Newton matrix singular, and how does SgGN address it?

The symmetric Gauss–Newton matrix is often singular. SgGN uses the algebraic structure of the shallow ReLU neural network to remove all singularities of the GN matrix at each iteration step.

Jul 16, 2026

Upload by Evangeline