Self-Composing Neural Operators with Depth and Accuracy Scaling via Adaptive Train-and-Unroll Approach
Juncai He, Xinliang Liu, Jinchao Xu () · 2025
Read Self-Composing Neural Operators with Depth and Accuracy Scaling via Adaptive Train-and-Unroll Approach by Juncai He, Xinliang Liu, Jinchao Xu () (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-20650.txt.
Introduction
Juncai He and co-authors build deep equation solvers from one repeated block in a 2025 paper. High-frequency and multiscale equations strain neural operators, which blur fine detail. Their self-composing design mimics classic iterative solvers. A single small block is applied again and again, deepening the solve without new parameters.
Training grows the depth step by step, a curriculum from shallow to deep. A multigrid-inspired block fights the usual bias toward smooth answers. On ultrasound tomography waves, errors fall well below Fourier operator rivals in the hard frequency band. Theory backs the gains. The recipe offers accuracy scaling for wave problems that defeat standard networks.
Catalog record
| CREATOR | Juncai He, Xinliang Liu, Jinchao Xu () |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-20650
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Self-Composing Neural Operators with Depth and Accuracy Scaling via Adaptive Train-and-Unroll Approach Thanks: Submitted to the editors DATE. Juncai He, Xinliang Liu, Jinchao Xu () Abstract In this work, we propose a novel framework to enhance the efficiency and accuracy of neural operators through self-composition, offering both theoretical guarantees and practical benefits. Inspired by iterative methods in solving numerical partial differential equations (PDEs), we design a specific neural operator by repeatedly applying a single neural operator block, we progressively deepen the model without explicitly adding new blocks, improving the model’s capacity. To train these models efficiently, we introduce an adaptive train-and-unroll approach, where the depth of the neural operator is gradually increased during training. This approach reveals an accuracy scaling law with model depth and offers significant computational savings through our adaptive training strategy. Our architecture achieves state-of-the-art (SOTA) performance on standard benchmarks. We further demonstrate its efficacy on a challenging high-frequency ultrasound computed tomography (USCT) problem, where a multigrid-inspired backbone enables superior performance in resolving complex wave phenomena. The proposed framework provides a computationally tractable, accurate, and scalable solution for large-scale data-driven scientific machine learning applications. keywords Operator learning, self-composition, multigrid, efficient training, ultrasound computed tomography Funding. This work was funded by the KAUST research baseline. † † runningheads: Self-composing neural operators / J. He, X. Liu, J. Xu MSC 68Txx, 65Mxx 1 Introduction Partial differential equation (PDE) models are ubiquitous in physics, engineering, and other disciplines. Many scientific and engineering fields rely on solving PDEs, such as optimizing airfoil shapes for better airflow, predicting weather patterns by simulating the atmosphere, and testing the strength of structures in civil engineering. Tremendous efforts have been made to solve various PDEs arising from different areas. Solving PDEs usually requires designing numerical methods that are tailored to the specific problem and depend on the insights of the model. However, deep learning models have shown great promise in solving PDEs in a more general and efficient way. Recently, several novel methods have been developed to directly learn the operator (mapping) between infinite-dimensional parameter and solution spaces of PDEs. Notable examples include DeepONet [ 35 ] , which employs a branch-trunk architecture grounded in universal approximation theory [ 5 ] , and the Fourier Neural Operator (FNO) [ 33 ] , which parameterizes global convolutional operators using a fast Fourier transform. This line of research has expanded rapidly, yielding variants like geo-FNO [ 32 ] for complex geometries, the Latent Spectral Model (LSM) [ 46 ] that operates in a learned latent space, and Multiwavelet-based Transformers (MWT) [ 16 ] using wavelet transforms. Concurrently, attention mechanisms, popularized by Transformers [ 45 ] , were adapted for operator learning, as seen in Galerkin-type attention models [ 3 ] and the General Neural Operator Transformer (GNOT) [ 20 ] , which introduces heterogeneous normalized attention for handling multiple input functions and irregular meshes. More recently, explicitly convolution-based architectures have also been proposed as neural operators [ 42 ] , and scalable approaches like Transolver++ [ 37 ] have demonstrated the ability to handle million-scale geometries for industrial applications. Additionally, specialized frameworks for complex geometries include Point Cloud Neural Operators (PCNO) [ 50 ] , which efficiently handle variable domains with topological variations. MIOnet is incorporated into classical iterative solvers, enhancing their efficiency [ 28 , 26 ] . The NINO [ 19 ] has demonstrated how neural operators can learn to integrate traditional numerical techniques with Newton nonlinear solvers, effectively learning the nonlinear mapping at each iteration for multiple solutions. These approaches can be broadly categorized. Spectral-type neural operators (e.g., FNO, MWT, LSM) leverage global operations and have shown promising results, supported by universal approximation theorems [ 29 , 31 ] . However, they often struggle with complex boundary conditions, typically requiring boundary information to be encoded into the input data, and can exhibit a bias towards learning low-frequency components, as high-frequency modes might be truncated. In contrast, local and geometrical neural networks (e.g., ResNet [ 24 , 25 ] , UNet [ 43 ] ) offer greater flexibility in managing diverse boundary conditions. Yet, these were not originally designed for operator learning of PDEs and often require a substantial number of parameters to achieve high accuracy. They can also act as high-pass filters, focusing on local features, which may limit their ability to capture long-distance dependencies crucial in many physical phenomena, unless specific mechanisms like large kernels are incorporated, which can be hard to train [ 10 ] . To bridge this gap, Liu et al. proposed the Hierarchical Attention Neural Operator (HANO), which mitigates spectral bias by adaptively coupling information across scales via attention, thereby boosting accuracy on challenging multiscale benchmarks [ 34 ] . Thus, although deep neural operators have shown great potential to learn complex patterns and relationships from data, and have been applied to fields like biomechanical engineering and weather forecasting [ 49 , 40 ] , they still face challenges regarding accuracy, flexibility with complex domains and boundary conditions, and parameter efficiency when compared to highly optimized classical numerical methods. Motivated by the iterative nature of many classical PDE solvers, where a conventional procedure is repeated to progressively refine a solution, we pose the following question: Question 1.1 . Can we design and efficiently train an appropriate neural operator such that it repeats only a simple skeleton structure (low-cost) but can achieve high accuracy through this repetition? This question draws inspiration from some of the most powerful techniques in scientific computing. Among all iterative methods for numerical PDEs, multigrid methods [ 18 , 47 , 44 ] are renowned for their efficiency, especially for elliptic PDEs. The connection between multigrid and deep learning was first noted in the original ResNet paper [ 24 ] , citing multigrid as a rationale for residual connections. Subsequently, MgNet [ 22 , 23 ] established deeper links, demonstrating that a linear V-cycle multigrid for the Poisson equation could be represented as a CNN. While MgNet and its adaptations have been explored for numerical PDEs [ 8 , 54 ] and forecasting [ 55 ] , a general and effective framework for integrating multigrid principles directly into operator learning for a broader class of PDEs has remained an open area. In this work, we aim to fundamentally integrate multigrid methodologies with operator learning based on a very general framework. To address these challenges, we investigate the intrinsic properties of operator learning tasks governed by PDEs and propose a concise and elegant neural operator architecture based on the concept of self-composition, directly inspired by iterative numerical methods. We introduce the general self-composing formulation \mathcal{O}(\bm{v})=\mathcal{P}\circ(\mathcal{G}\circ)^{n}\circ\mathcal{L}(\bm{v}) for neural operators. This framework allows building deep operators with significant parameter sharing by repeatedly applying a backbone operator \mathcal{G}(\cdot) . The concept of repeatedly applying a computational block shares similarities with several existing paradigms. Classical iterative methods for PDEs, such as Jacobi or Gauss-Seidel, inherently involve repeating a fixed update rule. Recurrent Neural Networks (RNNs) [ 9 ] also apply the same transition function repeatedly over a sequence. Algorithm unrolling [ 14 ] explicitly maps iterations of an optimization algorithm onto layers of a deep network. Our self-composing framework is distinct in several ways. While motivated by iterative PDE solvers, it