MetaColloc: Optimization-Free PDE Solving via Meta-Learned Basis Functions
Zichuan Yang · 2026
Read MetaColloc: Optimization-Free PDE Solving via Meta-Learned Basis Functions by Zichuan Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12368.txt.
Introduction
Yang removes the retraining bottleneck from machine-learning solvers for partial differential equations. His MetaColloc system first meta-trains a two-branch network on varied random fields, building a shared dictionary of basis functions. At test time the network stays frozen, and each new equation is solved by one linear least-squares step, with Newton steps for nonlinear cases. Tests on six two- and three-dimensional equations reach leading accuracy while cutting test-time compute by orders of magnitude.
Catalog record
| CREATOR | Zichuan Yang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51768 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12368
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MetaColloc: Optimization-Free PDE Solving via Meta-Learned Basis Functions Zichuan Yang Abstract Solving partial differential equations (PDEs) with machine learning typically requires training a new neural network for every new equation. This optimization is slow. We introduce MetaColloc. It is an optimization-free and data-free framework that removes this bottleneck completely. We decouple basis discovery from the solving process. We meta-train a dual-branch neural network on diverse Gaussian Random Fields. This offline process creates a universal dictionary of neural basis functions. At test time, we freeze the network. We solve the PDE by assembling a collocation matrix. We find the solution through a single linear least squares step. For non-linear PDEs, we apply the Newton-Raphson method to achieve fast quadratic convergence. Our experiments across six 2D and 3D PDEs show massive improvements. MetaColloc reaches state-of-the-art accuracy on smooth and non-linear problems. It also reduces test-time computation by several orders of magnitude. Finally, we provide a detailed frequency sweep analysis. This analysis reveals a critical mismatch between function approximation and operator stability at extremely high frequencies. This profound finding opens a clear path toward future operator-aware meta-learning. 1 Introduction Partial differential equations (PDEs) describe many physical systems. They appear in fluid flow, quantum models, and electromagnetic fields. Classical solvers work well but require complex meshes. Their cost grows fast in high dimensions. Recent machine learning methods try to avoid this cost. Physics-Informed Neural Networks (PINNs) [ 34 ] turn PDE solving into an optimization problem. They use neural networks and automatic differentiation to match the PDE residual. This idea removes the mesh and works in high-dimensional spaces. PINNs still face a core problem. They must train a new network for every new PDE. This training takes thousands of gradient steps. It is slow and often unstable. Neural networks also show spectral bias [ 31 ] . They learn low frequencies first and struggle with high-frequency waves. They also face challenges with stiff non-linear equations. Other methods try to speed up test-time solving. Fourier Neural Operators (FNO) [ 21 ] and DeepONets [ 27 ] learn mappings between function spaces. They solve new PDEs quickly. But they need large paired datasets. These datasets come from classical solvers and are expensive to generate. We take a different view. We look at the structure of PDE solvers. The hard part is not the final solve. The hard part is finding a good set of basis functions. If we have a strong basis, the PDE becomes a simple linear least-squares problem. This solve takes only a few seconds. This idea suggests a shift. We should learn the basis offline. We should keep the online stage simple and fast. Some earlier works try this idea. Extreme Learning Machines (ELMs) [ 8 ] and random feature models use random networks as basis functions. These methods are simple but limited. To approximate complex functions, random features often require a very large number to achieve effective learning outcomes [ 12 ] . Furthermore, they are highly sensitive to initialization. We propose MetaColloc to address these limits. MetaColloc is an optimization-free PDE solver, requiring neither solution data nor gradient-based optimization at test time. We use meta-learning to build a universal dictionary of neural basis functions. We train the network offline on many Gaussian Random Field (GRF) tasks. GRFs are natural priors for PDE solutions. They cover smooth and rough functions. This forces the network to learn a wide range of shapes. To handle extremely high frequencies, we use a dual-branch design. One branch processes raw coordinates. The other branch uses multi-scale Fourier features [ 38 ] . The network learns how to mix these two branches during meta-training. Figure 1 shows the full pipeline. At test time, we freeze the network. We compute exact derivatives with forward-mode automatic differentiation. We build a collocation matrix and solve it in one step. For non-linear PDEs, we apply the Newton–Raphson method. This gives fast and stable updates. We reach high accuracy in a few iterations. We test MetaColloc on six PDEs. It matches state-of-the-art accuracy on smooth problems. It outperforms existing methods on high-frequency and non-linear equations. It also reduces test-time cost by several orders of magnitude. Figure 1: Overview of MetaColloc. During meta-training (a), coordinate samples x are drawn from multi-scale Gaussian Random Fields (GRF). The dual-branch network processes inputs through a low-frequency branch (raw coordinates) and a high-frequency branch (multi-scale Fourier embedding), whose outputs are concatenated to form the basis matrix \Phi . Network parameters \theta are updated via least-squares loss on GRF tasks. At test time (b), \theta is frozen; given collocation points x_{int} from a new PDE, the network produces \Phi and its derivatives \Phi_{x},\Phi_{xx} , …, which are assembled into a linear system solved in closed form. The PDE solution is then recovered as u(x)=\Phi(x)\cdot w , requiring neither solution data nor gradient-based optimization. 2 Related Work 2.1 Physics-Informed Neural Networks Physics-Informed Neural Networks (PINNs) [ 34 ] encode PDE constraints directly into a loss function. They solve equations using gradient descent. Many studies improve this baseline. Researchers balance loss terms during training [ 40 , 19 , 24 ] , analyze gradients with Neural Tangent Kernels [ 41 ] , and enforce causality [ 42 ] . Other works design new optimization techniques [ 7 , 39 , 47 , 4 ] or propose new architectures like Kolmogorov-Arnold Networks [ 25 ] . To solve high-frequency problems, some methods use random Fourier features [ 41 ] or dynamic meshes [ 20 ] . However, these methods remain very sensitive to hyper-parameters. All these approaches share a core limitation. They perform per-instance optimization. A user must train a new network for every new PDE. This makes test-time computation very expensive. MetaColloc avoids this problem completely. We move the heavy learning process to an offline meta-training stage. At test time, MetaColloc only requires a single linear solve. 2.2 Gaussian Process and Generative PDE Solvers Gaussian Process (GP) methods provide a probabilistic way to solve PDEs [ 5 ] . Recent work like GP-HM [ 9 ] models the power spectrum of the solution directly in the frequency domain. GP-HM beats standard PINNs on high-frequency PDEs. Alongside GPs, recent years have shown a surge in diffusion models for PDE modeling. Researchers use generative diffusion to reconstruct physical fields, control complex systems, and predict dynamics [ 18 , 37 , 50 , 3 , 45 ] . We differ from these methods fundamentally. GP-HM is still a per-instance solver. It optimizes kernel parameters for hundreds of seconds on every new problem. It also requires data points on a strict Cartesian grid. Generative diffusion models often require slow, iterative sampling steps [ 36 ] . MetaColloc has no grid limits. Our test-time speed is extremely fast and remains independent of the PDE type. 2.3 Operator Learning and Meta-Learning for PDEs Many researchers use operator learning to accelerate PDE simulations. Fourier Neural Operators (FNO) [ 21 ] and DeepONets [ 27 ] learn mappings between function spaces. Recent advances scale these operators heavily. They use Transformers [ 35 , 1 , 16 , 43 ] , State-Space Models [ 49 ] , and geometry-aware deformations [ 22 , 44 ] . Some studies explore multi-physics