Transferable Physics-Informed Representations via Closed-Form Head Adaptation
Jian Cheng Wong 1 *, Isaac Yin Chung Lai 2 , Pao-Hsiung Chiu 1 , Chin Chun Ooi 1 , Abhishek Gupta 3 , Yew-Soon Ong 1 4 · 2026
Read Transferable Physics-Informed Representations via Closed-Form Head Adaptation by Jian Cheng Wong 1 *, Isaac Yin Chung Lai 2 , Pao-Hsiung Chiu 1 , Chin Chun Ooi 1 , Abhishek Gupta 3 , Yew-Soon Ong 1 4 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21761.txt.
Introduction
Wong and five colleagues make physics-informed neural networks transfer to new equations with little or no training data. Their Pi-PINN framework learns a shared physics-informed representation, then solves known and unknown partial differential equation instances through closed-form head adaptation with a least-squares-optimal pseudoinverse. Tests cover Poisson, Helmholtz, and Burgers equations. Predictions run 100 to 1000 times faster than a typical PINN, with far lower error than a typical data-driven model.
Catalog record
| CREATOR | Jian Cheng Wong 1 *, Isaac Yin Chung Lai 2 , Pao-Hsiung Chiu 1 , Chin Chun Ooi 1 , Abhishek Gupta 3 , Yew-Soon Ong 1 4 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34599 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.21761
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Transferable Physics-Informed Representations via Closed-Form Head Adaptation *Corresponding author: [email protected] Jian Cheng Wong 1 *, Isaac Yin Chung Lai 2 , Pao-Hsiung Chiu 1 , Chin Chun Ooi 1 , Abhishek Gupta 3 , Yew-Soon Ong 1 4 Abstract Physics-informed neural networks (PINNs) have garnered significant interest for their potential in solving partial differential equations (PDEs) that govern a wide range of physical phenomena. By incorporating physical laws into the learning process, PINN models have demonstrated the ability to learn physical outcomes reasonably well. However, current PINN approaches struggle to predict or solve new PDEs effectively when there is a lack of training examples, indicating they do not generalize well to unseen problem instances. In this paper, we present a transferable learning approach for PINNs premised on a fast Pseudoinverse PINN framework (Pi-PINN). Pi-PINN learns a transferable physics-informed representation in a shared embedding space and enables rapid solving of both known and unknown PDE instances via closed-form head adaptation using a least-squares-optimal pseudoinverse under PDE constraints. We further investigate the synergies between data-driven multi-task learning loss and physics-informed loss, providing insights into the design of more performant PINNs. We demonstrate the effectiveness of Pi-PINN on various PDE problems, including Poisson’s equation, Helmholtz equation, and Burgers’ equation, achieving fast and accurate physics-informed solutions without requiring any data for unseen instances. Pi-PINN can produce predictions 100–1000 times faster than a typical PINN, while producing predictions with 10–100 times lower relative error than a typical data-driven model even with only two training samples. Overall, our findings highlight the potential of transferable representations with closed-form head adaptation to enhance the efficiency and generalization of PINNs across PDE families and scientific and engineering applications. I Introduction Physics-informed neural networks (PINNs) have attracted significant attention as a neural approach for solving partial differential equations (PDEs) that govern a wide range of physical phenomena across science and engineering [ 1 , 2 ] . PINNs incorporate governing laws directly into the learning objective by treating PDE residuals, and associated boundary conditions (BCs) and initial conditions (ICs), as training constraints, resulting in a “physics-informed” loss [ 3 , 4 , 5 , 6 ] . Despite their promise, two practical limitations have repeatedly emerged. First, PINNs often train slowly and can exhibit poor optimization behavior compared to purely data-driven networks, largely due to the stiffness and complex loss landscapes introduced by physics-informed constraints [ 7 , 8 , 9 ] . Second, even after expensive training, standard PINNs generalize poorly to new PDE instances (e.g., new coefficients, source terms, BCs/ICs, or parameter regimes), limiting their usefulness when extrapolation or rapid redeployment is needed [ 10 ] . As a result, PINNs are still commonly used in a single-instance setting where the physics-informed loss is enforced directly for the target PDE, while extending a trained model to new instances typically requires substantial re-training and re-tuning. In many real-world settings, however, one does not start from scratch: there may exist a small collection of related PDE instances with solutions (or partial labels) that can be leveraged to learn representations that transfer. This motivates a transfer-learning perspective for PINNs, with the objective to build PINN models that are more reusable and adaptable across PDE families and parameter regimes. Hence, in this paper, we present Pi-PINN, a fast Pseudoinverse PINN framework that learns transferable physics-informed representations and enables closed-form head adaptation via a least-squares-optimal pseudoinverse under PDE constraints. The core idea is to decouple learning into (i) a shared embedding that captures transferable structure across related PDE instances and (ii) a task-specific output head that can be adapted efficiently through a closed-form linear solve, avoiding costly gradient-based re-optimization for each new instance. In addition, we study how combining data-driven multi-task learning losses with physics-informed residuals can produce more performant and transferable PINN models. The key contribution of our work are: (1) we introduce Pi-PINN, a pseudoinverse-based physics-informed learning framework that supports closed-form, least-squares-optimal head adaptation under PDE constraints, substantially reducing the computational cost of adapting PINNs to new PDE instances (see schematic in Figure 1 ). (2) We propose a representation-learning formulation for PINNs that learns transferable deep embeddings from related PDE instances, improving generalization across PDE families and parameter regimes. (3) We analyze the synergy between data-driven multi-task learning objectives and physics-informed residual losses, providing practical insights for training PINNs that are both accurate and reusable. (4) Empirically, across various PDE problems including Poisson, Helmholtz, and Burgers equations, Pi-PINN achieves 100–1000 \times faster prediction/adaptation than conventional PINNs, while obtaining 10–100 \times lower relative error than typical data-driven models in sparse-data regimes (e.g., 2–4 training samples). Fig. 1: The Schematic diagram of the fast Pseudoinverse PINN framework, with the example PDE instances (Burgers’ equation). II Pseudoinverse PINN Without loss of generality, we consider the use of PINNs to model the outcomes u in any spatial-temporal domain x\in\Omega,t\in(0,T] by satisfying the following governing equations: PDE: \displaystyle\mathcal{N}[u(x,t)] \displaystyle=h(x,t), \displaystyle x\in\Omega,t\in(0,T] (1a) BC: \displaystyle\mathcal{B}[u(x,t)] \displaystyle=g(x,t), \displaystyle x\in\partial\Omega,t\in(0,T] (1b) IC: \displaystyle u(x,t=0) \displaystyle=u_{0}(x), \displaystyle x\in\Omega (1c) where \mathcal{N}[u(x,t)] is a general differential operator which can include linear and nonlinear combinations of temporal and spatial derivatives, and h(x,t) is an arbitrary source term. Eq. 1b and Eq. 1c specify the BC and IC of the physical system, where \mathcal{B}[u(x,t)] is also a differential operator. To simplify the notation, we consider that the variation across different PDE instances (including variations in BC and IC) can be parameterized by a set of task parameters \vartheta . Every new PDE instance leads to different physical outcomes, u_{\vartheta} , spanning the entire spatial-temporal domain x\in\Omega,t\in(0,T] . A PINN model with sufficient capacity (number of nodes n_{\text{node}} , hidden layers L ) can effectively learn a good approximation to the target output, u_{\text{PINN}}(x,t;\boldsymbol{w}) , by minimizing the PINN loss function w.r.t. its network parameters \boldsymbol{w}=(\boldsymbol{W}_{1},\mathbf{b}_{1},...,\boldsymbol{W}_{L-1},\mathbf{b}_{L-1},\mathbf{w}_{L}) . II-A Pseudoinverse Physics-Informed [Pi]^{2} Computation. There has been some prior work on transfer learning in the PINN context [ 11 , 12 , 13 ] . These typically involve the transfer of network weights from a similar, previously-solved physics scenario as a better initialization for PINN training on a new task, i.e., minimizing the physics-informed loss by penalizing any violation of the PDE, BC/IC constraints made by the PINN model outputs on the target PDE. Generally, only the final few layers of the networks are fine-tuned in transfer learning, facilitating the exchange of valuable features acquired when training the PINN model for the previous task. In the extreme, we can transfer the entire learned representation up to the final nonlinear hidden layer, i.e., \mathbf{x}_{L} , and only fine-tune the weight parameters in the linear output layer \mathbf{w}_{L} . In the absence of target labels for a new PDE instance, this can be done by means of physics-informed fine-tuning that purely minimizes violations of the PDE and BC/IC constraints by the PINN output u_{\text{PINN}}=\mathbf{w}_{L}^{T}\mathbf{x}_{L} for the target PDE instance. For linear PDEs, the process of fine-tuning the output weight parameters \mathbf{w}_{L} reduces to a linear least-squares computation. This can be observed by examining the PDE, BC/IC constraints of Eq. 1 , where the terms \mathcal{N}[u_{\text{PINN}}(x,t;\boldsymbol{w})]-h(x,t) , u_{\text{PINN}}(x,t=0;\boldsymbol{w})-u_{0}(x) , and \mathcal{B}[u_{\text{PINN}}(x,t;\boldsymbol{w})]-g(x,t) , are all linear in terms of the output weights \mathbf{w}_{L} , as long as \mathcal{N}[\cdot] and