Preserving Conservation Laws in the Time-Evolving Natural Gradient Method via Relaxation and Projection Techniques Journal: Elsevier

Zihao Shi, Dongling Wang · 2026

Read Preserving Conservation Laws in the Time-Evolving Natural Gradient Method via Relaxation and Projection Techniques Journal: Elsevier by Zihao Shi, Dongling Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20816.txt.

Introduction

Shi and Wang add conservation laws to the time evolving natural gradient method for neural PDE solvers. Local training over time suits evolving equations well. But plain updates can drift away from true invariants like energy or mass. Their first tool relaxes the target solution so it keeps the invariants. The second projects updated parameters back onto the invariant manifold. Tests on Burgers, KdV, and acoustic wave equations followed. Conservation improved a lot while accuracy stayed high.

Catalog record

CREATORZihao Shi, Dongling Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35867
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.20816

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Preserving Conservation Laws in the Time-Evolving Natural Gradient Method via Relaxation and Projection Techniques Journal: Elsevier Zihao Shi, Dongling Wang Abstract Neural networks have demonstrated significant potential in solving partial differential equations (PDEs). While global approaches such as Physics-Informed Neural Networks (PINNs) offer promising capabilities, they often lack inherent temporal causality, which can limit their accuracy and stability for time-dependent problems. In contrast, local training frameworks that progressively update network parameters over time are naturally suited for evolving PDEs. However, a critical challenge remains: many physical systems possess intrinsic invariants—such as energy or mass—that must be preserved to ensure physically meaningful solutions. This paper addresses this challenge by enhancing the Time-Evolving Natural Gradient (TENG) method, a recently proposed local training framework. We introduce two complementary techniques: (i) a relaxation algorithm that ensures the target solution u target u_{\text{target}} preserves both quadratic and general nonlinear invariants of the original system, providing a structure-preserving learning target; and (ii) a projection technique that maps the updated network parameters θ ⁡ ( t ) \theta(t) back onto the invariant manifold, ensuring the final neural network solution strictly adheres to the conservation laws. Numerical experiments on the inviscid Burgers equation, Korteweg-de Vries equation, and acoustic wave equation demonstrate that our proposed approach significantly improves conservation properties while maintaining high accuracy. 1 Introduction 1.1 Motivation and Background Neural networks have emerged as a prominent mesh-free alternative for solving partial differential equations (PDEs) r1 , du2021evolutional , han2018solving , li2020fourier . Their principal advantage lies in the use of automatic differentiation to compute differential operators without spatial grids, offering a viable pathway for simulating high-dimensional PDEs. Raissi et al. proposed Physics-Informed Neural Networks (PINNs) r1 , which embed governing equations directly into the loss function within a static optimization framework. However, a notable limitation arises for time-dependent problems: PINNs lack an explicit mechanism to enforce temporal causality, as they process temporal and spatial coordinates concurrently jung2024ceens . This may yield solutions that violate the temporal consistency required by physical processes. Existing strategies to address this limitation fall into two categories jung2024ceens , r11 , r10 : soft constraints assign time-varying weights to encourage fitting earlier times first r10 , while hard constraints adopt classical time-stepping concepts r11 . Both approaches, however, increase computational overhead and may slow the solving process. Recently, a novel time-local training approach known as the Neural Galerkin method was introduced bruna2024neural , du2021evolutional . This method updates network parameters sequentially in time, evolving from static \theta to time-dependent \theta(t) . Grounded in the Dirac-Frenkel time-dependent variational principle lubich2008quantum , the right-hand side of the governing equation is projected onto the tangent space of the parametric solution manifold, yielding a dynamical system for \theta(t) . In this local training framework, learning becomes the time integration of a parameter dynamics system driven by the underlying physics. Building upon this foundation, Chen et al. chen2024teng proposed the Time-Evolving Natural Gradient (TENG) method, which combines the Neural Galerkin scheme with optimization-based time integration. TENG projects solution evolution in physical space onto network parameter space, guiding parameter updates via natural gradients rather than blind search. Many dynamical systems possess intrinsic invariants—such as energy, mass, or momentum—that reflect fundamental conservation laws. A key insight from classical numerical analysis is that strict preservation of these invariants is paramount for stability and reliability, sometimes even more crucial than precisely satisfying the original equations HairerWannerLubich2006 , boesen2025neural . For neural-network-based solvers, ensuring that parameterized approximations respect these physical laws is essential for long-term accuracy and physical fidelity. When applying TENG to time-dependent PDEs with conservative structures, two fundamental difficulties arise: (i) Target solution lacks conservation. The target solution u_{\text{target}} , typically generated by explicit Runge-Kutta methods, often violates quadratic and nonlinear conserved quantities such as energy. This causes the training data itself to deviate from true physical invariants, compromising the learning target from the outset. (ii) Parameter updates deviate from the invariant manifold. The tangent-space approximation process in TENG does not incorporate conservation constraints into parameter optimization. Consequently, trained network parameters \theta are not confined to the theoretical conservation manifold \mathcal{M}_{I} , fundamentally limiting the model’s ability to predict long-term system evolution accurately. To address these shortcomings, we introduce a dual-strategy enhancement to the TENG framework: 1. Relaxation algorithm for target solution: Following the relaxation approach proposed by Ranocha et al. ranocha2020relaxation , we incorporate a relaxation parameter \gamma that scales the time step to ensure u_{\text{target}} preserves desired invariants while maintaining the accuracy order of the original time discretization. 2. Projection algorithm for parameter updates: Drawing on geometric numerical integration principles HairerWannerLubich2006 , schwerdtner2024nonlinear , we apply a projection technique that maps updated network parameters back onto the conservation manifold \mathcal{M}_{I} , ensuring the final neural network solution strictly adheres to conservation laws. This combined approach—which we denote as RP-TENG (Relaxation-Projection TENG)—provides a structure-preserving learning target and enforces conservation throughout parameter evolution. Numerical experiments on three benchmark problems demonstrate significant improvements in conservation properties while maintaining computational efficiency and accuracy. The remainder of this paper is structured as follows. Section 2 reviews the mathematical formulation of structure-preserving PDEs and the Neural Galerkin framework. Section 3 introduces the TENG method and presents our relaxation and projection enhancements. Section 4 provides comprehensive numerical experiments on the inviscid Burgers equation, Korteweg-de Vries equation, and acoustic wave equation. Section 5 concludes and discusses future research directions. 2 Preliminaries This section reviews the mathematical modeling of time-dependent PDEs with conservation laws and the foundational concepts of the Neural Galerkin method. 2.1 Time-Dependent PDEs with Conservation Laws Consider a partial differential equation on a bounded domain \mathcal{X}\subseteq\mathbb{R}^{d} . Let \mathcal{U} be a suitable function space such that solutions u(\cdot,t) over the time interval belong to this space. A PDE with physical structure can be written as \displaystyle\partial_{t}u(x,t)=f\bigl(x,u(\cdot,t)\bigr),\qquad\forall(x,t)\in\mathcal{X}\times[0,\infty), (1) \displaystyle u(x,0)=u_{0}(x),\qquad\forall x\in\mathcal{X}, where f contains spatial differential operators such as the Laplacian \Delta . Let I be a physical quantity associated with the equation. For a conservative system, I satisfies \frac{d}{dt}I(u(t))=0,\quad\forall t\geq 0, (2) for all solutions of the equation. More generally, dissipative systems satisfy dI/dt\leq 0 . This paper focuses on conservative systems where equality holds identically. For numerical approximations, it is crucial that discrete solutions preserve such invariants, i.e., I(u_{n})=I(u_{0}) for all n\geq 1 . To study conservation laws systematically, we consider invariants of the form I_{i}(u)=\int_{\mathcal{X}}k_{i}(u(x,t))\,d\mu,\quad i=1,\ldots,m, (3) where k_{i} may be linear, quadratic, or nonlinear functions of u , corresponding to linear, quadratic, or nonlinear invariants, respectively. 2.2 The Neural Galerkin Framework We assume that at any time t , the solution u(\cdot,t) admits a parameterized representation \hat{u}(\cdot;\theta(t)) , where the parameters \theta(t)\in\Theta\subseteq\mathbb{R}^{n_{\theta}} are time-dependent: u(t,x)\approx\hat{u}(\theta(t),x),\qquad\forall(t,x)\in[0,\infty)\times\mathcal{X}. (4) Denote the parameterized solution manifold as \mathcal{M}=\{\hat{u}(\eta,\cdot)\mid\eta\in\Theta\}\subseteq\mathcal{U} , with tangent space T_{\hat{u}(\theta(t),\cdot)}\mathcal{M}=\operatorname{span}\{\partial_{\theta_{1}}\hat{u},\ldots,\partial_{\theta_{n}}\hat{u}\}. (5) Substituting the ansatz ( 4 ) into ( 1 ) yields the residual r_{t}(x,\theta(t),\dot{\theta}(t))=\partial_{t}\hat{u}-f(x,\hat{u})=\nabla_{\theta}\hat{u}(\theta(t),x)\cdot\dot{\theta}(t)-f(x,\hat{u}). (6) The Dirac-Frenkel variational principle requires the residual to be orthogonal to the tangent space: \langle\partial_{\theta_{i}}\hat{u}(\theta(t),\cdot),r_{t}(\theta(t),\dot{\theta}(t))\rangle_{\mu}=0,\quad i=1,\ldots,n_{\theta}, (7) where \langle x,y\rangle_{\mu}=\int_{\mathcal{X}}x\cdot y\,d\mu ,where \mu denotes a probability measure. This yields the dynamical system for network parameters: M(\theta(t))\dot{\theta}(t)=F(\theta(t)), (8) with \displaystyle M(\theta(t))=\int_{\mathcal{X}}\nabla_{\theta}\hat{u}^{\top}\nabla_{\theta}\hat{u}\,d\mu,\quad F(\theta(t))=\int_{\mathcal{X}}\nabla_{\theta}\hat{u}^{\top}f(\hat{u})\,d\mu. (9) Equation ( 8 ) resembles a classical ODE system, with numerical stability determined by the invertibility of the mass matrix M . Due to inherent symmetries in

Read the complete text