Towards deep physics-informed Kolmogorov–Arnold networks Journal: arXiv
Spyros Rigas, Fotios Anagnostopoulos, Michalis Papachristou, Georgios Alexandridis · 2025
Read Towards deep physics-informed Kolmogorov–Arnold networks Journal: arXiv by Spyros Rigas, Fotios Anagnostopoulos, Michalis Papachristou, Georgios Alexandridis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-23501.txt.
Introduction
Spyros Rigas and three co-authors fix a training problem in physics-informed Kolmogorov-Arnold Networks, models that solve physics equations using flexible curve-based layers. The Chebyshev-based versions train fast and are now standard, but they grow unstable when stacked deep, which blocks many uses. Simply starting the weights the usual way does not keep deep versions on track.
The authors propose a Glorot-like starting scheme that keeps signal strength steady across layers, improving stability and accuracy. For the deepest cases they add a design called Residual-Gated Adaptive KANs, which borrows a gating idea from PirateNet. On nine standard equation benchmarks, their models beat matched rivals, often by orders of magnitude, and stayed stable where the others broke down. A study of how information flows during training helps explain why the gated design keeps learning.
Catalog record
| CREATOR | Spyros Rigas, Fotios Anagnostopoulos, Michalis Papachristou, Georgios Alexandridis |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 126 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-23501
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards deep physics-informed Kolmogorov–Arnold networks Journal: arXiv Spyros Rigas, Fotios Anagnostopoulos, Michalis Papachristou, Georgios Alexandridis Abstract Since their introduction, Kolmogorov–Arnold Networks (KANs) have been successfully applied across several domains, with physics-informed machine learning (PIML) emerging as one of the areas where they have thrived. In the PIML setting, Chebyshev-based physics-informed KANs (cPIKANs) have become the standard due to their computational efficiency. However, like their multilayer perceptron-based counterparts, cPIKANs face significant challenges when scaled to depth, leading to training instabilities that limit their applicability to several PDE problems. To address this, we propose a basis-agnostic, Glorot-like initialization scheme that preserves activation variance and yields substantial improvements in stability and accuracy over the default initialization of cPIKANs. Inspired by the PirateNet architecture, we further introduce Residual-Gated Adaptive KANs (RGA KANs), designed to mitigate divergence in deep cPIKANs where initialization alone is not sufficient. Through empirical tests and information bottleneck analysis, we show that RGA KANs successfully traverse all training phases, unlike baseline cPIKANs, which stagnate in the diffusion phase in specific PDE settings. Evaluations on seven standard forward PDE benchmarks under a fixed training pipeline with adaptive components demonstrate that RGA KANs consistently outperform parameter-matched cPIKANs and PirateNets – often by several orders of magnitude – while remaining stable in settings where the others diverge. 1 Introduction The widespread adoption and integration of machine learning into computational science has profoundly influenced the way complex physical phenomena are modeled and analyzed. One of the most striking advances is the Physics - Informed Machine Learning (PIML) framework [ 1 , 2 ] , which offers a compelling alternative to traditional discretization - based solvers for both forward and inverse problems involving partial differential equations (PDEs). Within the PIML framework, the governing equations, alongside boundary and initial conditions, plus any observational data, are embedded into a differentiable loss function, while a neural network parametrizes the unknown solution field. Leveraging automatic differentiation [ 3 ] to evaluate differential operators exactly, PIML eliminates the need for mesh generation and yields continuous, high - fidelity predictions with reduced computational cost. As a result, it has found success across a broad spectrum of scientific and engineering disciplines, from fluid mechanics [ 4 , 5 , 6 , 7 ] and materials science [ 8 , 9 ] to medicine [ 10 , 11 , 12 ] and chemistry [ 13 , 14 ] . While a variety of neural architectures have been explored within the PIML paradigm, including convolutional neural networks (CNNs) [ 15 ] , generative adversarial networks (GANs) [ 16 ] , and long short-term memory (LSTM) networks [ 17 ] , the fully connected multilayer perceptron (MLP) is the predominant backbone. When an MLP parametrizes the solution field, the variant is conventionally termed a Physics-Informed Neural Network (PINN), which is also the original formulation of PIML [ 18 ] . Despite their widespread use, PINNs exhibit several well - documented shortcomings, including spectral bias toward low - frequency modes [ 19 ] , restricted interpretability and limited scalability with depth, among other challenges in their training dynamics. To address these issues, numerous mitigation strategies have been proposed, ranging from architectural modifications [ 20 , 21 , 22 , 23 ] , to adaptive training techniques [ 24 , 25 , 26 ] . A complementary approach is to forgo the MLP backbone altogether in favor of alternative architectures that mitigate several of these issues by design. One such emerging alternative is the Kolmogorov–Arnold Network (KAN) [ 27 ] . Whereas the expressivity of MLPs is supported by the universal approximation theorem [ 28 ] , KANs are grounded in the Kolmogorov–Arnold representation theorem [ 29 ] . In practice, a KAN layer replaces fixed nonlinear activations with a learnable linear combination of basis functions; the original implementation employs B-splines, but other basis functions such as radial basis functions [ 30 ] , Chebyshev polynomials [ 31 ] and Rectified Linear Unit (ReLU)-based functions [ 32 ] have also been explored. This design offers several benefits, most notably in terms of enhanced interpretability [ 33 ] and robustness against spectral bias [ 34 ] . These have motivated the development of Physics-Informed Kolmogorov–Arnold Networks (PIKANs) [ 35 , 36 ] , where the MLP backbone is substituted by a KAN in the PINN framework. Initial studies have demonstrated that PIKANs can attain higher accuracy on benchmark PDEs, or comparable accuracy with considerably smaller network architectures than their MLP-based counterparts [ 27 , 35 , 37 ] . Consequently, they have already seen practical deployment in a variety of scientific and engineering contexts [ 38 , 39 , 40 ] . Despite their promising early results, PIKANs present their own challenges. Computational overhead is the most immediate: evaluating and differentiating the B - spline basis in the original KAN formulation quickly becomes a bottleneck, leading most physics-informed implementations to adopt the more efficient Chebyshev variant (cPIKAN) [ 35 ] . Scalability is another concern; empirical studies report training instabilities as the number of network parameters increases beyond a point [ 41 , 42 ] , limiting applicability in deep learning regimes. Similar issues are observed in deep PINNs, although the PirateNet architecture appears to mitigate them [ 23 ] . Methodological gaps also persist. Weight initialization schemes are still largely ad hoc: each basis family (B - spline, radial - basis, Chebyshev, etc.) provides its own default, yet no analogue to the well-studied Glorot initialization for PINNs [ 43 ] has been empirically or theoretically analyzed. Systematic experimentation with initialization schemes is at a nascent stage and has so far concentrated exclusively on B-spline-based KANs [ 44 ] . The picture is similar for adaptive training strategies. While several PINN-oriented techniques have been ported to PIKANs [ 37 , 45 , 46 ] , a unified training pipeline, comparable to the one codified for PINNs [ 47 ] , has yet to be established. Motivated by these gaps, we concentrate on cPIKANs, which are a fitting choice for physics-informed applications in terms of their computational efficiency and accuracy. We first observe that the reported depth-related instabilities may be closely tied to weight initialization: an initialization that preserves activation variance can prevent vanishing or exploding gradients, just as the Glorot scheme does in MLPs. Accordingly, we derive a “Glorot - like” initialization for KANs that makes no assumptions about the specific basis and is therefore applicable to any KAN variant. On a series of function-fitting and PDE benchmarks, we show that this initialization improves optimization stability and yields significantly more accurate solutions than the default initialization of Chebyshev-based KANs and cPIKANs. Building on this foundation, we address the depth-scaling issue. Because each KAN layer carries multiple learnable basis coefficients, a KAN layer of the same width as an MLP layer is substantially more parameter-heavy; real-world tasks that need larger capacity must therefore rely on greater depth, which in turn demands stable training. To this end, we introduce a Residual-Gated Adaptive KAN (RGA KAN) architecture. We analyze its training dynamics through the lens of the Information Bottleneck (IB) framework [ 48 ] and empirically demonstrate that RGA KANs remain stable and train effectively at depths where baseline cPIKANs diverge. Finally,