Convergence of the generalization error for Deep Gradient Flow Methods for PDEs

Chenguang Liu, Antonis Papapantoleon, Jasper Rou · 2025

Read Convergence of the generalization error for Deep Gradient Flow Methods for PDEs by Chenguang Liu, Antonis Papapantoleon, Jasper Rou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-25017.txt.

Introduction

Chenguang Liu, Antonis Papapantoleon, and Jasper Rou — with co-authors — put deep gradient flow methods for partial differential equations on firm mathematical ground. Their analysis splits generalization error into approximation error and training error. Solutions of equations under reasonable, checkable assumptions are shown to be approximable by neural networks, so approximation error vanishes as networks widen.

The training process in the wide network limit follows an explicit gradient flow, which they derive and analyze. Applied mathematics readers get convergence understanding for a popular neural PDE solver family. High-dimensional equations are the target, where grid methods collapse and neural methods promise relief.

Catalog record

CREATORChenguang Liu, Antonis Papapantoleon, Jasper Rou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE120 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-25017

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Convergence of the generalization error for Deep Gradient Flow Methods for PDEs Chenguang Liu, Antonis Papapantoleon, Jasper Rou Abstract . The aim of this article is to provide a firm mathematical foundation for the application of deep gradient flow methods (DGFMs) for the solution of (high-dimensional) partial differential equations (PDEs). We decompose the generalization error of DGFMs into an approximation and a training error. We first show that the solution of PDEs that satisfy reasonable and verifiable assumptions can be approximated by neural networks, thus the approximation error tends to zero as the number of neurons tends to infinity. Then, we derive the gradient flow that the training process follows in the “wide network limit” and analyze the limit of this flow as the training time tends to infinity. These results combined show that the generalization error of DGFMs tends to zero as the number of neurons and the training time tend to infinity. 2020 Mathematics Subject Classification 68T07, 65M12 1. Introduction Deep learning methods for the solution of high-dimensional partial differential equations (PDEs) have gained tremendous popularity in the last few years, since they can tackle equations in dimensions that were not attainable by classical methods, such as finite difference and finite element schemes. This ability allows the modeling of more realistic phenomena across various fields of science and technology, including engineering, biology, economics, and finance. The seminal articles of Sirignano and Spiliopoulos 2018 on the Deep Galerkin Method (DGM) and of Raissi and Karniadakis 2018 and Raissi et al. 2019 on physics-informed neural networks (PINNs), building on the earlier work of Lagaris et al. 1998 and Lagaris et al. 2000 , incorporate the PDE residual and the initial and boundary conditions into the loss function of a neural network, which is then minimized by stochastic gradient descent (SGD). These methods have laid the foundations for a variety of extensions and applications, including among many others, fractional differential equations ( Pang et al. 2019 ), variational PINNs ( Kharazmi et al. 2021 ), Bayesian variants ( Yang et al. 2021 ) and mean-field games ( Carmona and Zeng 2024 ). On the other hand, deep gradient flow methods (DGFMs), also known as deep Ritz methods, formulate the PDE as an energy minimization problem, where the energy is derived from the differential operator, which typically leads to a loss function that is easier to compute. Moreover, they usually discretize the equation in time and train one network for each time step, instead of using a monolithic space-time discretization; see e.g. , E and Yu 2018 , Liao and Ming 2021 , Georgoulis et al. 2023 , Park et al. 2023 , Papapantoleon and Rou 2025 , Bruna et al. 2024 for differential operators, and Georgoulis et al. 2024 for an integro-differential operator. A comprehensive review of the available methods appears in the forthcoming book of Jentzen et al. 2025 . In the present article, we are interested in analyzing the error of deep gradient flow methods for the solution of PDEs. Let us consider the PDE (1.1) \begin{split}u_{t}+\mathcal{A}u&=0,\quad(t,x)\in[0,T]\times D,\\ u(0,x)&=\Phi(x),\quad x\in\partial D,\end{split} where \mathcal{A} is a differential operator, \Phi determines the initial condition, T is a (finite) time horizon, and D\subseteq\mathbb{R}^{d} is the domain of the PDE. The DGFMs translate the PDE into an energy minimization problem, which is then computed using stochastic gradient descent or one of its variants ( e.g. ADAM), and can be described in the following manner: (1.2) u^{\star}_{\theta,n}=\argmin_{v\in\mathcal{C}^{n}_{\theta}}\int\ell(v(x))\mathrm{d}x, where \mathcal{C}^{n}_{\theta} denotes the space of neural networks with n neurons where \theta is the set of trainable parameters, while \ell denotes the energy functional associated to the operator \mathcal{A} . Let u^{\star} denote the unique solution of ( 1.1 ). We would like to analyze and study the difference between the true solution of ( 1.1 ) and the solution computed by the deep gradient flow methods, i.e. by the outcome of the minimization problem ( 1.2 ). This difference is known as the generalization error in the machine learning literature, i.e. \mathcal{E}_{\text{gen}}=\|u^{\star}-u^{\star}_{\theta,n}\|. The generalization error \mathcal{E}_{\text{gen}} can be decomposed in three separate components: • the quadrature error \mathcal{E}_{\text{quad}} , which refers to how well the integral in ( 1.2 ) is approximated by Monte Carlo simulations or another quadrature method; • the approximation error \mathcal{E}_{\text{approx}} , which refers to how well the neural network v can approximate the continuous function u that solves the PDE ( 1.1 ); • the training error \mathcal{E}_{\text{train}} , which refers to how well GD or SGD approximate the true solution of the minimization problem ( 1.2 ). Then, we have the error decomposition (1.3) \displaystyle\mathcal{E}_{\text{gen}}=\mathcal{E}_{\text{train}}+\mathcal{E}_{\text{quad}}+\mathcal{E}_{\text{approx}}, and the aim of the present paper is to study these errors and show that, as the numbers of neurons tends to infinity and the training time also tends to infinity, then the generalization error tends to zero, and the outcome of the deep gradient flow method indeed approximates the solution of the PDE. There are several articles available that study the generalization error of deep learning methods for PDEs, typically focusing on the popular DGM and PINN methods. These methods rely on approximability properties of neural networks and properties of quadrature methods in order to control the generalization error, while they typically consider only a posteriori estimates for the training error. We refer the interested reader to Mishra and Molinaro 2022 and Gazoulis et al. 2025 for results on PINNs, and the related while more general article of Loulakis and Makridakis 2023 . Moreover, several articles consider the approximation error of DMG and PINNs; see, for example, Sirignano and Spiliopoulos 2018 and Shin et al. 2020 . The recent article of Jiang et al. 2023 considers the “global” convergence of DGM and PINNs, which amounts to the convergence of the training error in our notation. Combined with other available results, this article allows to deduce the convergence of the generalization error of these methods. Compared to the extended literature on DGM and PINNs, there are significantly fewer papers on DGFMs; let us mention here the articles of Dondl et al. 2022 which focuses on the approximation error, and Jiao et al. 2024 which provides a convergence rate using the Rademacher complexities. The aim of the present article is to provide convergence results on the generalization error of DGFMs under reasonable and verifiable hypothesis on the underlying PDEs. The first part of this work focuses on the analysis of the approximation error, i.e. we show that there exists a neural network that approximates the solution of the PDE. This result uses ideas from PDE theory, optimization and the calculus of variations, and is inspired by the seminal paper of Sirignano and Spiliopoulos 2018 . The second part of this work focuses on the analysis of the training error, and we show that as the number of neurons tend to infinity and the training time also tends to infinity, then the outcome of the deep gradient flow method tends to the true solution of the PDE. This result is inspired by the work of Jiang et al. 2023 . The quadrature error is the most well-understood error of the three, thus this work focuses on the other two errors. Moreover, let us mention for the

Read the complete text