Neural Networks for Singular Perturbations – Finite Regularity

F. Rohner, Ch. Schwab, C. Xenophontos · 2026

Read Neural Networks for Singular Perturbations – Finite Regularity by F. Rohner, Ch. Schwab, C. Xenophontos (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14459.txt.

Introduction

Rohner, Schwab and Xenophontos bound how well finite elements and feedforward neural networks can express solutions of a singularly perturbed elliptic boundary-value problem on a bounded interval. Working with low-regularity data, they prove parameter-robust algebraic rates for linear finite elements on graded meshes. Shallow and deep ReLU networks get matching robust rates, and deep networks with tanh units are claimed to double the finite-element rate.

Catalog record

CREATORF. Rohner, Ch. Schwab, C. Xenophontos
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79960
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.14459

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Neural Networks for Singular Perturbations – Finite Regularity F. Rohner, Ch. Schwab, C. Xenophontos Abstract We study finite-element and deep feedforward neural network (DNN for short) expressivity rate bounds for solution sets of a model linear, second order singularly perturbed, elliptic two-point boundary value problem, in Sobolev norms on a bounded interval ( − 1 , 1 ) (-1,1) , with explicit dependence on the singular perturbation parameter ε ∈ ( 0 , 1 ] \varepsilon\in(0,1] . Emphasis is on low Sobolev regularity of the data, i.e., source term f f and reaction coefficient b b . A proof of ε \varepsilon -explicit solution regularity based on exponentially weighted energy-norm bounds is developed, and ε \varepsilon -robust, algebraic expression rate bounds in Sobolev norms for ℙ 1 \mathbb{P}_{1} Finite-Elements on exponential and Shishkin type meshes is proved. Expression rates for shallow (fixed depth) ReLU \ReLU -NNs are shown which are robust w.r. to ε \varepsilon and explicit in terms of the NN size. Robust NN expression rate bounds are further studied for deep feedforward DNNs with ReLU and tanh-activations. As in 15 , tanh- and sigmoid-activated sub-NNs allow to include exponential boundary layer functions exactly into the NN feature space, leading to reduced NN sizes. Recent bitstring encoding techniques for deep NNs with ReLU activations afford, still under low data regularity f , b ∈ H 1 ​ ( I ) f,b\in H^{1}(I) twice the (robust) convergence rate of ℙ 1 \mathbb{P}_{1} Finite-Elements achievable with “eXp” or Shishkin meshes. MSC Classification]34B08, 34D15, 65L11, 65N30 keywords Singular Perturbations, Robust Convergence, Shishkin Meshes, Neural Networks, Superconvergence pacs [ 1 Introduction Elliptic and parabolic PDEs which depend on a small, non-dimensional physical parameter \varepsilon>0 arise in many areas of science. Their numerical solution is well-known to be challenging due to multiscale phenomena governing solution regularity. A particular class of such problems are singular perturbation problems in solid and fluid dynamics; we refer to 11 ; 8 ; 10 ; 18 and the references there. In all these applications, PDEs depend on a small parameter \varepsilon\in(0,1] in physically relevant regimes of input data. Standard numerical approximation methods (Finite Volume-, Finite Difference- or Finite Element Methods) generally do not perform uniformly w.r. to the physical perturbation parameter \varepsilon : in general, a so-called scale resolution condition relating the discretization parameters (such as the mesh width h in Finite Element Methods) and \varepsilon needs to hold, in order to exhibit theoretical approximation rates. In this paper, we continue and complement our study 15 on \varepsilon -robust expression rate bounds for solution families \{u_{\varepsilon}\}_{0<\varepsilon\leq 1} of elliptic singular perturbation problems. As in 15 ; 19 , we consider the model linear, singularly perturbed, reaction-diffusion boundary value problem (BVP): find u_{\varepsilon}(x) such that \displaystyle\mathcal{L}_{\varepsilon}u_{\varepsilon}:=-\varepsilon^{2}u^{\prime\prime}_{\varepsilon}(x)+b(x)u_{\varepsilon}(x) \displaystyle= \displaystyle f(x)\;,\;x\in I=(-1,1)\;, (1.1) \displaystyle u_{\varepsilon}(\pm 1) \displaystyle= \displaystyle 0, (1.2) where \varepsilon\in(0,1] is a small parameter that can approach zero, and b(x),f(x) are given functions on \overline{I}=[-1,1] , with the reaction coefficient satisfying \overline{b}\geq b(x)\geq\underline{b}>0\;\mbox{a.e. on}\;\overline{I}\;\mbox{ for some constants}\;\overline{b},\;\underline{b}. (1.3) While in 15 , analytic data b,f\in C^{\infty}(\overline{I}) were assumed, we concentrate in the present note on data b,f of finite, low Sobolev regularity in I . A common trait of singularly perturbed, linear, elliptic PDEs is an additive decomposition of the solution u_{\varepsilon} into a regular part u^{S}_{\varepsilon} and into boundary layer components u^{BL}_{\varepsilon} (see, e.g., 8 and the references there). The regular solution part u^{S}_{\varepsilon} may depend on the singular perturbation parameter \varepsilon , but is smooth independent of \varepsilon , i.e. derivatives of u^{S}_{\varepsilon} satisfy bounds in Sobolev norms which are bounded uniformly in terms of \varepsilon . A typical result for solutions u_{\varepsilon} of ( 1.1 ), ( 1.2 ) is as follows (see, e.g. 8 ; 18 ; 10 and the references there). Assume that for a nonnegative integer k , the data in ( 1.1 ) satisfy b,f\in C^{k+2}(\bar{I}) . Then u_{\varepsilon}=u^{S}_{\varepsilon}+u^{BL}_{\varepsilon}\;, (1.4) where the smooth part u^{S}_{\varepsilon} and the boundary layer u^{BL}_{\varepsilon} satisfy: for integer j=0,1,2,...,k+2 exists C_{j}>0 such that, for \varepsilon\in(0,1] and for x\in I holds |(u^{S}_{\varepsilon})^{(j)}(x)|\leq C_{j}\;,\quad|(u^{BL}_{\varepsilon})^{(j)}(x)|\leq C_{j}\varepsilon^{-j}\exp(-({\rm dist}(x,\partial I))/\varepsilon)\;. (1.5) The solution part u^{BL}_{\varepsilon} is not uniformly smooth in terms of \varepsilon . Its k -th derivative typically grows as O(\varepsilon^{-k}) . Due to their exponential decay off \partial I , the u^{BL}_{\varepsilon} are referred to as boundary layers . 1.1 Contributions While the pointwise bounds ( 1.5 ) are, in terms of their \varepsilon -dependence, optimal, one observes that at least C^{2}(\overline{I}) -regularity of the data b and f in ( 1.1 ), ( 1.2 ) is required for them to hold. We establish in Prop. 2.1 for a model, linear elliptic singular perturbation problem with source term f\in H^{1}(I) and variable reaction coefficient b\in H^{1}(I) in a bounded interval I=(-1,1) an \varepsilon -explicit solution decomposition ( 1.4 ) into a regular part u^{S}_{\varepsilon} of low, finite Sobolev regularity bounded uniformly with respect to the perturbation parameter \varepsilon , and into exponential boundary layer functions u^{BL}_{\varepsilon} . Unlike 19 ; 12 where analytic in \overline{I} coefficients and source terms were assumed, we admit data of finite Sobolev regularity H^{1}(I) . Unlike decompositions with pointwise derivative bounds ( 1.5 ), we establish \varepsilon -explicit bounds on u^{BL}_{\varepsilon} in exponentially weighted Sobolev spaces. Based on the solution decomposition, we prove that the solution can be expressed robustly (i.e. with constants and rates independent of \varepsilon ) at algebraic (w.r. to the NN size as measured in terms of the number of neurons) rates by various adapted Finite-Element Methods and DNN architectures, in various Sobolev norms. We show, in particular, in Thm. 4.6 that fixed depth, strict \ReLU -activated NNs afford the same robust algebraic rates as \mathbb{P}_{1} -FEM on \varepsilon -dependent Shishkin, Bakhvalov and exponential meshes, in the ( \varepsilon -dependent) energy norm. For singular perturbations of constant coefficient differential operators, we exhibit novel, fixed-depth NN architectures that are based on combination of \ReLU and \tanh activations which allow robust solution approximation rates with fewer neurons than strict \ReLU -activated NNs. We prove that certain deep, so-called super-expressive NN architectures from 24 afford, under the same finite Sobolev regularity assumptions on the coefficient b and the source terms f in ( 1.1 ), higher orders of robust solution approximation than the best results achievable by shallow NNs: Specifically, with a given budget of N neurons we show there are deep NNs that afford a robust convergence rate which is higher than the robust rate afforded with CpwL spline approximations. 1.2 Notation We shall use standard notation: \mathbb{N}=\{1,2,3,...\} and \mathbb{R} denote the natural resp. the real numbers, and I=(a,b)\subset\mathbb{R} shall denote an open, bounded interval. For a summability index q\in[1,\infty] , we denote by L^{q}(I) the Banach space of q -Lebesgue integrable, real-valued functions on I . For integer differentiation order k\in\mathbb{N}_{0}=\{0,1,2,...\} , we denote by W^{k,q}(I) the Banach space of k -times weakly differentiable functions in L^{q}(I) , whose k -th weak derivative belongs to L^{q}(I) . For k\in\mathbb{N} , W^{k,q}_{0}(I) denotes the closed subspace of functions in W^{k,q}(I) with vanishing traces at \partial I . In the hilbertian case q=2 , we write H^{k}(I)=W^{k,2}(I) , and H^{k}_{0}(I) . Often we shall use the space {\mathcal{S}}^{p}(I,\Delta) of continuous, real-valued functions f:I\to\mathbb{R} on a

Read the complete text