Learning on the Temporal Tangent Bundle for Physics-Informed Neural Networks
Jamal Adetola 1,∗, Charbel Mamlankou 1, Koffi Wilfrid Houédanou 2, Guy Aymard Degla 3 · 2026
Read Learning on the Temporal Tangent Bundle for Physics-Informed Neural Networks by Jamal Adetola 1,∗, Charbel Mamlankou 1, Koffi Wilfrid Houédanou 2, Guy Aymard Degla 3 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11829.txt.
Introduction
Jamal, Mamlankou, Houédanou, and Degla rework physics-informed neural networks for time-dependent problems. Instead of approximating the solution directly, they model its time derivative and rebuild the state with a Volterra integral that fixes initial conditions exactly. That removes competing soft constraints and sharpens high-frequency errors, countering spectral bias. On advection, Burgers, and Klein-Gordon tests, their compact three-layer networks cut errors by 100 to 200 times versus standard methods, with better shock capture and long-time accuracy.
Catalog record
| CREATOR | Jamal Adetola 1,∗, Charbel Mamlankou 1, Koffi Wilfrid Houédanou 2, Guy Aymard Degla 3 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 119765 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.11829
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning on the Temporal Tangent Bundle for Physics-Informed Neural Networks Thanks: ∗ Corresponding author Thanks: 1 École Nationale Supérieure de Génie Mathématique et Modélisation (ENSGMM), Université Nationale des Sciences, Technologies, Ingénierie et Mathématiques (UNSTIM), Bénin Thanks: 2 Département de Mathématiques (DM), Université d’Abomey-Calavi (UAC), Bénin Thanks: 3 Institut de Mathématiques et de Sciences Physiques (IMSP), Université d’Abomey-Calavi (UAC), Bénin Thanks: Email: [email protected], [email protected], [email protected], [email protected] Jamal Adetola 1,∗, Charbel Mamlankou 1, Koffi Wilfrid Houédanou 2, Guy Aymard Degla 3 Abstract . This paper addresses the limitations of Physics-Informed Neural Networks for time-dependent problems by introducing a tangent bundle learning framework. Instead of directly approximating the solution, we parameterize its temporal derivative and reconstruct the state through a Volterra integral operator that enforces initial conditions exactly. This approach eliminates competing soft constraints and naturally amplifies high-frequency errors through differentiation, countering spectral bias. We prove theoretical equivalence between minimizing the differentiated residual and solving the original partial differential equation. Experiments on advection, Burgers, and Klein-Gordon equations show that the proposed method achieves 100 to 200 times lower errors than standard approaches using compact three-layer networks, with superior shock-capturing and long-time accuracy. Keywords: Physics-Informed Neural Networks, Time-Dependent PDEs, Tangent Bundle Learning, Volterra Operators, Spectral Bias, Scientific Machine Learning, Temporal Causality 2020 Mathematics Subject Classification: 65M99, 35K05, 35Q53, 68T07 1. Introduction The quantitative description of natural phenomena through partial differential equations (PDEs) constitutes the bedrock of modern physics and engineering. Since the mid-20th century, the numerical approximation of these equations has been dominated by grid-based discretization strategies. Finite Difference Methods (FDM) [ 42 ] , Finite Element Methods (FEM) [ 4 , 8 ] , and Finite Volume Methods (FVM) [ 26 ] have reached a high degree of maturity, providing robust tools for simulating everything from fluid turbulence to structural deformation. Parallel to these local approximations, Spectral Methods [ 3 , 6 , 14 ] have offered exponential convergence rates for smooth problems by utilizing global basis functions. However, despite their ubiquity, these classical paradigms face intrinsic limitations when confronted with the "curse of dimensionality" [ 2 ] . As the dimension of the problem increases, the computational cost of mesh generation and linear system solving grows exponentially, rendering many high-dimensional problems in quantum mechanics or finance computationally intractable [ 15 , 13 ] . The last decade has witnessed the emergence of Scientific Machine Learning (SciML) as a transformative methodology aiming to circumvent these limitations. Leveraging the Universal Approximation Theorem [ 9 , 16 ] , Deep Neural Networks (DNNs) have been proposed as mesh-free function approximators capable of solving PDEs. The seminal introduction of Physics-Informed Neural Networks (PINNs) by Raissi, Perdikaris, and Karniadakis [ 38 , 36 , 37 ] formalized this approach. By embedding the PDE residual directly into the loss function of the network, PINNs constrain the neural search space to the manifold of physically admissible solutions, removing the need for labeled training data [ 19 ] . This paradigm has been successfully applied across a diverse spectrum of fields, including cardiovascular flow modeling [ 21 ] , material identification [ 40 ] , and inverse scattering problems [ 30 ] . Notwithstanding these successes, the application of standard PINNs to time-dependent evolution equations remains fraught with difficulties. The optimization landscape of the associated non-convex loss function is often pathological, characterized by stiff gradients and sharp local minima [ 43 ] . A primary culprit is the "spectral bias" or "F-principle" inherent to deep networks [ 35 , 48 ] , which predisposes them to learn low-frequency components of the target function rapidly while struggling to capture high-frequency details. In the context of time-dependent PDEs, this manifests as a failure to resolve sharp transients or high-frequency oscillations, leading to solutions that are overly diffusive or physically inaccurate [ 44 ] . Furthermore, the standard formulation treats the temporal coordinate t merely as an additional spatial dimension, ignoring the fundamental principle of causality [ 45 ] . Consequently, the optimization process does not respect the arrow of time, attempting to resolve the solution at t=T before correctly approximating the dynamics at t=0 , which frequently leads to convergence failures [ 23 , 11 ] . To mitigate these issues, researchers have proposed various heuristic modifications. Adaptive loss weighting schemes [ 47 , 46 , 31 ] attempt to dynamically balance the contributions of the PDE residual, boundary conditions, and initial conditions during training. Curriculum learning strategies [ 23 ] and temporal decomposition methods [ 17 , 32 , 41 ] break the time domain into smaller segments, solving the problem sequentially to respect causality. While these methods offer improvements, they often introduce additional hyperparameters and computational overhead without addressing the root cause: the difficulty of approximating a complex spatiotemporal trajectory directly from a static, undifferentiated loss landscape. More recently, operator learning frameworks like DeepONets [ 29 ] and Fourier Neural Operators (FNO) [ 27 , 22 ] have shifted focus to learning the mapping between infinite-dimensional function spaces. Yet, for single-instance PDE solving, the primal formulation remains the dominant approach. In this work, we propose a foundational architectural shift designed to address the specific pathologies of time-dependent neural PDE solvers. We introduce Physics-Informed Time Derivative Networks (PITDNs). Instead of targeting the solution manifold u(x,t) directly, our method learns the temporal rate of change, \partial_{t}u(x,t) , effectively lifting the learning problem to the tangent bundle of the solution space. The state variable is then recovered via a strictly differentiable Volterra integral operator. This formulation provides a rigorous mechanism to enforce initial conditions by construction, reducing the multi-objective optimization problem to a simpler constrained form. Moreover, we employ a novel loss functional based on the time-differentiated residual , which we prove acts as a high-pass filter on the optimization landscape, countering spectral bias. Our contributions are threefold. First, we establish a mathematical framework using Bochner spaces to define the learning problem on the tangent bundle. Second, we prove the well-posedness of the differentiated residual minimization, showing its equivalence to the strong form of the Cauchy problem under specific consistency conditions. Third, we demonstrate through extensive numerical benchmarks on hyperbolic (Advection), parabolic (Burgers), and dispersive (Klein-Gordon) equations that PITDNs outperform standard baselines in terms of accuracy and long-time stability. 2. Mathematical Formulation In this section, we establish the theoretical foundations of the proposed method. We adopt the language of functional analysis to ensure rigor in our definitions and subsequent proofs. 2.1. Problem Setup and functional spaces Let \Omega\subset\mathbb{R}^{d} be a bounded open domain with a Lipschitz continuous boundary \partial\Omega , and let \mathcal{T}=[0,T] be the time interval of interest. We denote by L^{2}(\Omega) the Hilbert space of square-integrable functions equipped with the standard inner product \left\langle\cdot,\cdot\right\rangle and norm \left\|\cdot\right\|_{L^{2}(\Omega)} . For spatial regularity, we utilize the Sobolev spaces H^{k}(\Omega) , comprising functions whose weak derivatives up to order k exist and reside in L^{2}(\Omega) , equipped with the norm \|u\|_{H^{k}(\Omega)}^{2}=\sum_{|\alpha|\leq k}\|\partial^{\alpha}u\|_{L^{2}(\Omega)}^{2}. To rigorously describe time-dependent functions, we employ Bochner spaces. Specifically, let X be a Banach space (e.g., H^{k}(\Omega) ). The space L^{p}(0,T;X) consists of measurable functions u:[0,T]\to X such that \int_{0}^{T}\left\|u(t)\right\|_{X}^{p}\,dt<\infty. We define the solution space for our problem as \mathcal{V}:=C^{1}([0,T];H^{k}(\Omega))\cap L^{\infty}(0,T;H^{k}(\Omega)), ensuring continuous differentiability in time and sufficient spatial smoothness with