Discontinuous hybrid neural networks for the one-dimensional partial differential equations
Xiaoyu Wang Long Yuan and Yao Yu · 2025
Read Discontinuous hybrid neural networks for the one-dimensional partial differential equations by Xiaoyu Wang Long Yuan and Yao Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-09911.txt.
Introduction
Xiaoyu Wang, Long Yuan, and Yao Yu published a paper in 2025 on neural networks that can handle jumps when solving physics equations. Many equations of motion have solutions with sharp breaks, and ordinary smooth networks smear them. Allowing the network itself to be discontinuous promises accuracy at lower cost, since smooth networks approximate well in standard senses but waste effort at a break.
The authors build a hybrid method with a new loss that adds three ingredients: a variational form of the equation, penalties for jumps across internal interfaces, and boundary conditions. Two optimizers split the work, one tuning the network's nonlinear parameters and a Galerkin scheme handling the linear ones. The design aims to guarantee the loss settles while keeping the solution sharp where the physics demands it.
Catalog record
| CREATOR | Xiaoyu Wang Long Yuan and Yao Yu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-09911
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Discontinuous hybrid neural networks for the one-dimensional partial differential equations Xiaoyu Wang Long Yuan and Yao Yu Abstract A feedforward neural network, including hidden layers, motivated by nonlinear functions (such as Tanh, ReLU, and Sigmoid functions), exhibits uniform approximation properties in Sobolev space, and discontinuous neural networks can reduce computational complexity. In this work, we present a discontinuous hybrid neural network method for solving the partial differential equations, construct a new hybrid loss functional that incorporates the variational of the approximation equation, interface jump stencil and boundary constraints. The RMSprop algorithm and discontinuous Galerkin method are employed to update the nonlinear parameters and linear parameters in neural networks, respectively. This approach guarantees the convergence of the loss functional and provides an approximate solution with high accuracy. Keywords: Discontinuous hybrid neural networks, loss functional, discontinuous Galerkin, weak convergence, a posterior error estimate Mathematics Subject Classification (2010) 65N30, 65N55, 68T07 1 Introduction In numerical methods for solving partial differential equations, neural networks possess excellent adaptive capabilities and have gradually become a powerful tool [ 8 ] . A Physics-Informed Neural Network(PINN) [ 11 ] , as a classical neural network method, embeds the physical laws or boundary conditions directly into the training process of the neural network, which ensures that the training results conform to the physical laws. This approach can be used for control optimization, and forward and inverse problems [ 3 ] . Compared to purely data-driven neural networks, PINNs learn more generalized models with fewer data samples. Furthermore, [ 7 ] proposed a Variational Physics-Informed Neural Network(VPINN) within the Petrov-Galerkin framework. Unlike the PINN that incorporates the strong form of the equation into the network, a VPINN incorporates the variational weak formulation of the problem and constructs a variational formulation of loss function. While such PINNs and VPINN approaches have enjoyed success in particular cases, in practice, the achievable relative error is prone to stagnate around 0.1-0.01 no matter how many neurons or layers are uesd to define the underlying network architecture (see [ 6 ] ). To overcome this plight, [ 2 ] proposed Galerkin neural networks to approximate the variational equations by adaptively constructing a sequence of finite dimensional neural network subspaces, which are used as the discrete space of the variational equations. It constructs discrete basis functions under Galerkin iteration and provides a novel framework for solving elliptic, parabolic, and hyperbolic partial differential equations. However, the approach utilizes continuous neural networks, which typically require a large number of test points to ensure accuracy, thereby increasing computational complexity. In this paper, we use discontinuous neural networks to solve boundary value problems for time-harmonic partial differential equations. For convenience, we call the proposed method as DHNN method. For each element, fewer neural network nodes are used to construct the network structure, which better adapts to the local features and reduces the requirement of test point density [ 14 ] . Inspired by the construction of loss functionals in variational physics-informed neural networks [ 7 ] , we propose an innovative hybrid loss functional in section 3 that incorporates the variational of the approximation equation, interface jump stencil and boundary constraints. The RMSprop optimizer is employed to search nonlinear network parameters, such as weights and biases. After updating the nonlinear parameters, based on the associated linear neural network space, the discontinuous Galerkin method is employed to solve the linear parameters in the neural network. We derive the weak convergence of the approximation solution in the sense of the loss functional in section 4. Finally, we report some numerical results to confirm the effectiveness of the proposed method in section 5. 2 The model and discontinuous neural networks 2.1 The model We consider the solution of boundary value problems for one-dimensional time-harmonic partial differential equations using discontinuous neural networks. However, the proposed method can be extended to two-dimensional and three-dimensional cases. Let \Omega\subset R be a bounded domain, we perform a mesh discretization of the solution domain, with nodes being: x_{0}<x_{1}<\cdots<x_{N}. Here the subdomain \Omega_{i}=[x_{i-1},x_{i}] between the adjacent nodes x_{i-1} and x_{i} is i -th element. Let {\cal T}_{h} denote the partitioning \{\Omega_{k}\}_{k=1}^{N} of domain \Omega . Consider the following linear problem \left\{\begin{aligned} &\mathcal{A}u(x)=f\quad\quad\quad\text{in}\quad\Omega,\\ &\mathcal{B}u(x)=g(x)\quad\quad\text{on}\quad\partial\Omega,\end{aligned}\right. (2.1) f\in L^{2}(\Omega) , g\in H^{\frac{1}{2}}(\partial\Omega) . Let (V({\cal T}_{h}),||\cdot||_{V}) , (Y,||\cdot||_{Y}) , (Z,||\cdot||_{Z}) and (W,||\cdot||_{W}) be Banach spaces. Suppose that \mathcal{A}:V({\cal T}_{h})\longrightarrow Y is a bounded linear operator and \mathcal{B}:V({\cal T}_{h})\longrightarrow Z is also a bounded linear operator. Besides, suppose that C_{i}(i=1,2):V({\cal T}_{h})\longrightarrow W are bounded linear operators related to jump quantities on the set of all interior points \{x_{i}\}_{i=1}^{N-1} . 2.2 Discontinuous neural networks A general discontinuous neural network consists of a single hidden layer of n\in\mathbb{N} neurons on each elemen \Omega_{k}\in{\cal T}_{h} , defining a function U_{NN}:\mathbb{R}\longrightarrow\mathbb{C} as follows: u_{NN}(x;\theta)|_{\Omega_{k}}=\sum_{j=1}^{n}c_{j}^{(k)}\sigma(W_{j}^{(k)}\cdot x+b_{j}^{(k)}),\quad\quad\quad\forall\Omega_{k}\in V({\cal T}_{h}), (2.2) where n is called as the width of the discontinuous network; W_{j}^{(k)}\in\mathbb{R} , b_{j}^{(k)}\in\mathbb{R} are elementwise nonlinear parameters; c_{j}^{(k)}\in\mathbb{C} are elementwise linear parameters; and \sigma:\mathbb{R}\longrightarrow\mathbb{C} is a bounded, elementwise smooth activation function. For simplicity, we shall use the elementwise notation W|_{\Omega_{k}}=[W_{1}^{(k)},\cdots,W_{n}^{(k)}]\in\mathbb{R}^{1\times n} referred as weights, b|_{\Omega_{k}}=[b_{1}^{(k)},\cdots,b_{n}^{(k)}]\in\mathbb{R}^{n} referred as biases, and c|_{\Omega_{k}}=[c_{1}^{(k)},\cdots,c_{n}^{(k)}]^{T}\in\mathbb{C}^{n} . The set of nonlinear and linear parameters is collectively denoted by \Theta=[W,b,c] . In particular, a set of nonlinear parameters defined on every element is collectively denoted by \Phi=\{W,b\} . Denote V_{n}^{\sigma}({\cal T}_{h}) as the set of all functions of the form(2.2), which is defined by V_{n}^{\sigma}({\cal T}_{h}):=\{v:v|_{\Omega_{k}}=\sum_{j=1}^{n}c_{j}^{(k)}\sigma(W_{j}^{(k)}\cdot x+b_{j}^{(k)}),\quad k=1,\cdots,N\}. Although neural network space V_{n}^{\sigma}({\cal T}_{h}) is neither closed nor compact in C(\Omega) or L^{p}(\Omega) , by restricting the boundedness of network parameters, the neural network space with bounded parameters is both closed and compact in C(\Omega) and L^{p}(\Omega) . For the well-posedness of the discretized optimization problem (3.3) defined below, the subset V_{n,C}^{\sigma}({\cal T}_{h}) of V_{n}^{\sigma}({\cal T}_{h}) consisting of functions with bounded parameters needs to be defined (see [ 2 ] ). Given a sequence of positive increasing numbers C_{n} , satisfying \underset{n\longrightarrow\infty}{\lim}C_{n}=\infty , we define V_{n,C}^{\sigma}({\cal T}_{h}):=\{v\in V_{n}^{\sigma}({\cal T}_{h}):||\Theta||<C_{n}\}, where ||\Theta||=max_{k}||\Theta_{k}||,||\Theta_{k}||:=\max\limits_{j}|W_{j}^{(k)}|+\max\limits_{j}|b_{j}^{(k)}|+\max\limits_{j}|c_{j}^{(k)}| . 3 A discontinuous hybrid neural network algorithm Suppose \{\varphi_{k,i},k=1,\cdots,N,i=1,\cdots,\infty\} forms a complete orthonormal basis in a Hilbert space Y. Also, let \{\varphi_{k,i}\}_{i=1}^{\infty} forms a complete set of orthogonal basis functions in the Hilbert space Y_{k}=Y|_{\Omega_{k}} . We denote the PDE residual loss by L_{r}^{h}(\Theta):=\sum\limits_{k=1}^{N}\sum\limits_{i=1}^{\infty}|(\mathcal{A}v-f,\varphi_{k,i})_{Y_{k}}|^{2} , the boundary residual loss by L_{b}(\Theta):=||\mathcal{B}v-g||_{Z}^{2} , and the interface residual loss L_{int}^{h}(\Theta):=||[[C_{1}v]]||_{W}^{2}+||[[C_{2}v]]||_{W}^{2} . Then we can define the continuous hybrid loss functional as follows: \mathcal{J}_{\tau}^{h}(v)=L_{r}^{h}(\Theta)+\tau(L_{b}(\Theta)+L_{int}^{h}(\Theta)),\quad\quad\quad\forall\tau>0,v\in V({\cal T}_{h}). (3.3) Correspondingly, define its corresponding truncation by L_{r}^{h,\mathbf{M}}(\Theta):=\sum\limits_{k=1}^{N}\sum\limits_{i=1}^{M_{k}}|(\mathcal{A}v-f,\varphi_{k,i})_{Y_{k}}|^{2} and further \mathcal{J}_{\tau}^{h,\mathbf{M}}(v)=L_{r}^{h,\mathbf{M}}(\Theta)+\tau(L_{b}(\Theta)+L_{int}^{h}(\Theta)),\quad\quad\quad\forall\tau>0,v\in V({\cal T}_{h}), (3.4) where \mathbf{M}=(M_{1},\cdots,M_{N}) . Now we consider the minimization problem of the loss functionals constrained on the discontinuous neural network space. That is \underset{v\in V_{n,C_{n}}^{\sigma}}{\min}\mathcal{J}_{\tau}^{h,\mathbf{M}}(v). (3.5) To calculate optima of the objective function \mathcal{J}_{\tau}^{h,\mathbf{M}}(v) in neural network trainings, the RMSprop algorithm and discontinuous Galerkin method are employed to update the nonlinear parameters and linear parameters, respectively. The learning rate annealing strategy [ 12 ] is used to adaptively update the \tau for balancing the the PDE residual loss L_{r}^{h,\mathbf{M}}(\Theta) and the boundary residual loss L_{b}(\Theta)+L_{int}^{h}(\Theta) . 3.1 Updating linear parameters For known nonlinear parameters \Phi^{l} with l\geq 0 , let V_{\Phi^{l}}({\cal T}_{h}) denote the space V_{n}^{\sigma}({\cal T}_{h}) with the given nonlinear parameters \Phi^{l} . Linear parameters c^{l} are obtained using the standard discontinuous Galerkin method. Define