Fast PINN Eigensolvers via Biconvex Reformulation

Akshay Sai Banderwaar, Abhishek Gupta · 2025

Read Fast PINN Eigensolvers via Biconvex Reformulation by Akshay Sai Banderwaar, Abhishek Gupta (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00792.txt.

Introduction

Akshay Sai Banderwaar and Abhishek Gupta look for a faster way to solve eigenvalue problems with physics informed neural networks. Such problems reveal a system's heat response, stability, and natural modes of vibration. Neural solvers need no mesh, but they often train far more slowly than classic numerical methods.

The authors recast the search for eigenpairs, the paired values and shapes that solve these problems, as a biconvex task, meaning each half is well behaved when the other is held fixed. That form allows an alternating search with updates that can be computed directly and are proven to converge. In experiments, their method matched high accuracy while running up to five hundred times faster than gradient based training.

Catalog record

CREATORAkshay Sai Banderwaar, Abhishek Gupta
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-00792

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Fast PINN Eigensolvers via Biconvex Reformulation Akshay Sai Banderwaar, Abhishek Gupta Abstract Eigenvalue problems have a distinctive forward-inverse structure and are fundamental to characterizing a system’s thermal response, stability, and natural modes. Physics-Informed Neural Networks (PINNs) offer a mesh-free alternative for solving such problems but are often orders of magnitude slower than classical numerical schemes. In this paper, we introduce a reformulated PINN approach that casts the search for eigenpairs as a biconvex optimization problem, enabling fast and provably convergent alternating convex search (ACS) over eigenvalues and eigenfunctions using analytically optimal updates. Numerical experiments show that PINN-ACS attains high accuracy with convergence speeds up to 500 × \times faster than gradient-based PINN training. We release our codes at https://github.com/NeurIPS-ML4PS-2025/PINN_ACS_CODES . 1 Introduction Solving eigenvalue problems for differential operators involves finding non-trivial eigenvalue–eigenfunction pairs (eigenpairs) where the operator scales the function by its eigenvalue. Determining the function for a given eigenvalue resembles a forward problem, while extracting the eigenvalue from a function mimics an inverse problem. Physics-Informed Neural Networks (PINNs) have shown promise as high-resolution, mesh-free solvers for forward and inverse problems individually raissi2019physics ; toscano2025pinns , and these strengths could be brought to bear on the uniquely coupled forward–inverse structure of eigenvalue problems as well. However, existing PINN algorithms are known to be orders of magnitude slower than conventional numerical schemes mcgreivy2024weak ; grossmann2024can , primarily due to the difficulty of optimizing the highly non-convex PINN loss function. This loss combines terms representing the underlying differential equation and the initial and/or boundary conditions, which are often of incommensurable scale and can conflict under finite network capacity bischof2025multi . Moreover, the initial flatness of the network’s output—a symptom of the vanishing gradient problem wang2021understanding —tends to draw PINNs towards trivial solutions wong2022learning ; rohrhofer2023role , an issue that is especially deleterious for eigenvalue problems. Recent PINN methods for discovering eigenpairs rely on standard gradient-based iterative optimization ben2023deep ; yoo2025physics ; rowan2025solving ; bonder2025pinns , and therefore remain highly susceptible to conflicting loss terms and spurious basins of attraction in non-convex loss landscapes krishnapriyan2021characterizing ; wong2501evolutionary . One way to mitigate the conflict is to enforce boundary conditions directly into the network’s output, but applications have largely been limited to Dirichlet boundaries rowan2025solving ; bonder2025pinns . To reinforce the mesh-free flexibility of PINNs with accelerated convergence, this paper introduces a simple yet powerful trick: reformulating the PINN loss as a biconvex optimization problem. The key idea is to restrict training to the linear output layer of the PINN, keeping its hidden layers fixed to values initialized either randomly or according to some heuristic gallicchio2020deep . For linear differential operators, this makes the physics residual linear in terms of the network’s unknown parameters dong2023method ; wang2024piratenets . Consequently, the highly non-convex loss landscape is transformed into a biconvex one: convex (linear least-squares) in terms of the parameterized eigenfunction when the eigenvalue is fixed, and vice versa. We show that this structure admits a theoretically convergent alternating convex search (ACS) over eigenpairs, where each convex subproblem is solved optimally via analytical solutions. The overall framework not only yields substantial speedups compared to gradient-based optimization, but also improves accuracy through the analytic updates. The main contributions of this work are summarized below. • The differential eigenvalue problem is recast as a biconvex PINN optimization. Unlike prior PINN approaches, our reformulation accommodates varied boundary conditions and even applies to cases where the eigenvalue appears explicitly in the boundary operator (e.g., Euler’s buckling of a column with a free end). • Alternating convex search for PINN optimization is proposed. Starting at a random initial eigenvalue estimate, ACS iteratively fixes one variable (the eigenvalue or the eigenfunction) and updates the other via analytic results. PINN-ACS converges rapidly and can be parallelized over a population of estimates for simultaneous discovery of multiple eigenpairs. • We demonstrate the performance of our method on Euler’s buckling problem, for solving Helmholtz equation on an L-shaped domain, and for obtaining the natural frequencies of a thin plate (governed by a fourth-order biharmonic equation). PINN-ACS achieves speedups exceeding 10 \times , and up to 500 \times , compared to gradient-based PINN training. 2 Mathematical Preliminaries The general form of an eigenvalue problem for a differential operator defined over domain \Omega , with boundary \partial\Omega , can be written as: \displaystyle\mathcal{D}(u_{i})(\mathbf{x})+\lambda_{i}h(u_{i})(\mathbf{x}) \displaystyle=0, \displaystyle\mathbf{x}\in\Omega, (1a) \displaystyle\mathcal{B}(u_{i})(\mathbf{x}) \displaystyle=0, \displaystyle\mathbf{x}\in\partial\Omega. (1b) Here, \mathcal{D}(\cdot) is the differential operator, h(\cdot) is a function, \mathbf{x} represents the spatial coordinate, \mathcal{B}(\cdot) is the boundary operator, \lambda_{i} is the i^{\mathrm{th}} eigenvalue and u_{i}(\mathbf{x}) is the corresponding eigenfunction. In this work, we focus on linear self-adjoint operators, a common setting in recent studies and across many scientific applications ben2023deep ; mishra2025eig . Under this condition, the eigenfunctions are mutually orthogonal: \langle u_{i},u_{j}\rangle=\int_{\Omega}u_{i}(\mathbf{x})\,u_{j}(\mathbf{x})\,d\mathbf{x}=0,\;\text{for}\;i\neq j. (2) 3 Methodology 3.1 PINN Loss for Eigenvalue Problems Consider a multilayer perceptron parameterized by \mathbf{\theta} to approximate the i^{\mathrm{th}} eigenfunction as u_{i}(\mathbf{x};\mathbf{\theta}) . The PINN loss \mathcal{L}(\lambda_{i},\mathbf{\theta}) in terms of the model parameters and eigenvalue \lambda_{i} can then be defined as: \displaystyle\operatorname*{arg\,min}_{\lambda_{i},\,\mathbf{\theta}}\!\;\mathcal{L}(\lambda_{i},\mathbf{\theta})= \displaystyle\int_{\Omega}\!\left|\mathcal{D}(u_{i}(\mathbf{x};\mathbf{\theta}))+\lambda_{i}h(u_{i}(\mathbf{x};\mathbf{\theta}))\right|^{2}d\Omega+\alpha_{\mathrm{BC}}\!\int_{\partial\Omega}\!\mathcal{B}(u_{i}(\mathbf{x};\mathbf{\theta}))^{2}d\left(\partial\Omega\right) \displaystyle+\alpha_{\mathrm{ref}}\!\left(u_{i}(\mathbf{x}^{\mathrm{ref}};\mathbf{\theta})-u_{\mathrm{ref}}\right)^{2}+\alpha_{\mathrm{ortho}}\!\sum_{j=1}^{i-1}\left(\int_{\Omega}u_{i}(\mathbf{x};\mathbf{\theta})\,u_{j}(\mathbf{x})\,d\Omega\right)^{2}. (3) The first term captures the error in satisfying the differential equation; the second term enforces the boundary conditions; the third term prevents a trivial solution by prescribing u_{\mathrm{ref}}\neq 0 at some reference point \mathbf{x}^{\mathrm{ref}} ; the final term enforces orthogonality to eigenfunctions already discovered; \alpha_{\mathrm{BC}} , \alpha_{\mathrm{ref}} , \alpha_{\mathrm{ortho}} control the trade-offs between components. The loss is usually evaluated numerically by computing the integrand at randomly sampled collocation points and summing the results. An alternate way to achieve solution non-triviality is to substitute the third term by a normalization condition: \int_{\Omega}u_{i}(\mathbf{x};\mathbf{\theta})^{2}\,d\Omega=1 . However, this would induce nonlinear interactions between network parameters, hampering the biconvexification enabled by the form of Eq. ( 3 ), discussed next. 3.2 Biconvex Reformulation In a depth- L neural net, \theta=\{W^{[l]},\mathbf{b}^{[l]}\}_{l=1}^{L} and the transformation performed at the l^{\mathrm{th}} layer is a^{[l]}(\mathbf{x})=\sigma^{[l]}(W^{[l]}a^{[l-1]}(\mathbf{x})+\mathbf{b}^{[l]}) , where a^{[0]}(\mathbf{x})=\mathbf{x} . While \sigma^{[L]} is typically set to the identity function and \mathbf{b}^{[L]}=\mathbf{0} , the other non-linear activations, \sigma^{[l]} , render the network’s output non-linear in terms of the hidden layers’ weights and biases. This turns Eq. ( 3 ) into a mathematical program with a highly non-convex loss landscape, posing stiff optimization challenges unique to PINNs wong2501evolutionary ; sung2023neuroevolution . Figure 1: Illustrative plots for Euler’s buckling show that PINN-ACS leads to monotonic convergence as predicted by Theorem 1. Gradient descent using Adam yoo2025physics ; kingma2014adam oscillates due to conflicting loss terms. Tabulated average runtimes indicate substantial speedups with ACS alongside higher accuracy. In this paper, we reformulate the PINN eigensolver by restricting training to just the network’s output layer. Hidden layers are either set to random values or according to some heuristic (e.g., by transferring weights from a related source problem) and are left untrained. Randomization preserves a network’s universal approximation capacity under sufficient width and appropriate selection of random parameter distributions igelnik1995stochastic ; li2023powerful . As a result, we can write u_{i}(\mathbf{x};\mathbf{\theta}) as u_{i}(\mathbf{x};W^{[L]}) which is linear in terms of W^{[L]} . Plugging this simplified form of the eigenfunction into Eq. ( 3 ) gives \mathcal{L}(\lambda_{i},W^{[L]}) where the third and fourth terms turn into linear least-squares expressions in W^{[L]} . Note that this would not be the case if the normalization condition was employed (as

Read the complete text