Autoregression-Free Neural Operators for Time-Dependent PDEs

Jiaquan Zhang, Caiyan Qin, Haoyu Bian, Libin Cai, Yi Lu, Chaoning Zhang, Wei Dong, Yuanfang Guo, Yang Yang, Hen Tao Shen · 2026

Read Autoregression-Free Neural Operators for Time-Dependent PDEs by Jiaquan Zhang, Caiyan Qin, Haoyu Bian, Libin Cai, Yi Lu, Chaoning Zhang, Wei Dong, Yuanfang Guo, Yang Yang, Hen Tao Shen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25413.txt.

Introduction

Jiaquan Zhang and colleagues replace step-by-step rollout for time-dependent PDEs with continuous latent evolution. Standard neural operators feed each predicted state back as the next input, growing error over long horizons without time modeling. Their AFNO method maps time evolution into a latent space and learns its vector field with flow matching. That design enables steady evolution over long runs and handles varied physics settings through direct conditioning. Theory and tests on six equations show better long-horizon steadiness and smaller rollout errors than rivals.

Catalog record

CREATORJiaquan Zhang, Caiyan Qin, Haoyu Bian, Libin Cai, Yi Lu, Chaoning Zhang, Wei Dong, Yuanfang Guo, Yang Yang, Hen Tao Shen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81471
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.25413

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Autoregression-Free Neural Operators for Time-Dependent PDEs Jiaquan Zhang, Caiyan Qin, Haoyu Bian, Libin Cai, Yi Lu, Chaoning Zhang, Wei Dong, Yuanfang Guo, Yang Yang, Hen Tao Shen Abstract Neural operators learn mappings from function-dependent inputs to solutions, providing an effective framework for solving partial differential equations (PDEs). For time-dependent PDEs, existing methods typically perform long-horizon prediction through autoregressive rollout directly in high-dimensional physical field spaces, where each predicted state is recursively fed back as the input for the next step. Although effective for short-term prediction, this autoregressive rollout and the lack of continuous-time modeling lead to progressive error accumulation over long-horizon rollouts. In this work, we propose Autoregression-Free Neural Operators (AFNO), which map the time evolution of PDEs into a latent space and model continuous-time vector fields within it. AFNO uses flow matching to learn the latent vector field, thereby enabling continuous evolution over extended horizons, avoiding autoregressive rollout and capturing dynamics under varying parameter configurations through explicit conditioning on physical parameters. Theoretical analysis and extensive experiments on six PDEs demonstrate that AFNO improves long-horizon prediction stability and consistently reduces rollout errors compared with the baselines. I Introduction Time-dependent partial differential equations (PDEs) are fundamental in scientific computing and physical simulation, with applications in fluid mechanics, weather prediction, and material modeling [ 49 , 39 , 42 ] . Traditional numerical solvers rely on discretized time stepping and parameter-specific solves, resulting in high computational cost and limited scalability [ 53 ] . In recent years, deep learning methods for solving PDEs have achieved significant progress [ 26 ] , with neural operators learning mappings between function spaces and providing a new framework for equation modeling [ 2 , 30 ] . For time-dependent PDEs, the central objective is to learn the temporal evolution of physical fields, such as transport and diffusion in fluids [ 64 ] , wave propagation [ 13 ] , and phase evolution in materials [ 41 , 38 ] . Such a task requires approximating solution operators that propagate physical states across time, rather than merely predicting a static solution field. Methods such as Fourier Neural Operator (FNO) [ 33 ] address temporal evolution modeling by learning time-advancement operators in high-dimensional physical fields, enabling data-driven multi-step prediction. Subsequent studies incorporate self-supervised and multi-resolution training, together with low-rank structures and physical constraints [ 51 , 36 , 62 ] . These advances improve generalization across complex geometries, multiple physical fields, and high-dimensional settings, thereby broadening the applicability of neural operators to fluid simulation, weather forecasting, seismic modeling, and material design. Fig. 1: Comparison of autoregressive rollout and autoregression-free rollout for time-dependent PDEs. (Left) Existing methods perform autoregressive rollout in high-dimensional physical space, where predictions are recursively fed back as inputs, leading to amplified error accumulation over long horizons. (Right) Our method encodes physical fields into a parameter-conditioned latent space, with continuous latent space rollout via flow-matching dynamics yielding stable error propagation. However, most existing methods perform long-horizon prediction in an autoregressive manner, where the predicted state at each step is recursively fed back as the input for the next step [ 55 ] , as illustrated in Fig. 1 . Although effective for short-term forecasting, this strategy causes the mismatch between prediction and ground truth to be propagated through subsequent rollout steps during long-horizon extrapolation, resulting in error accumulation and prediction instability. This effect is particularly severe in high-dimensional physical fields with complex spatial coupling and multi-scale structures, where local errors spread spatially and compound across scales. For example, in turbulent flow, the coupling of velocity and pressure across multiple scales leads to rapid error accumulation [ 59 ] . Moreover, discrete time stepping for inherently continuous-time PDE evolution obscures the underlying system structure and limits adaptability to varying time steps and multiple temporal scales [ 52 ] . Beyond this issue, physical parameters such as viscosity, diffusion strength, and time step are typically treated implicitly. This further restricts generalization across different parameter regimes, e.g., varying Reynolds numbers in fluid simulations [ 21 ] . These limitations motivate us to ask whether long-horizon PDE prediction can be made autoregression-free. Recent advances in generative modeling, such as diffusion [ 9 ] , latent diffusion models (LDMs) [ 45 ] , and flow-based models [ 63 ] , describe temporal variation as continuous trajectories evolving within a structured latent space [ 17 ] . In particular, LDMs encode high-dimensional data into a compact latent space, thereby reformulating high-dimensional data modeling over structured latent representations rather than raw observations. [ 9 ] . This motivates us to transfer PDE temporal evolution from high-dimensional physical fields to a latent dynamical space, where essential dynamics can be modeled more compactly and stably. Under this formulation, long-horizon prediction is performed by numerically integrating the latent vector field rather than by recursively feeding predicted physical states back into a neural operator. By decoupling intrinsic dynamics from high-dimensional physical observations, the latent space facilitates the extraction of coherent temporal features. It further provides a compact manifold for consistent parameterization of temporal evolution. Flow matching supervises the local vector field through finite-difference derivative alignment, thereby improving multi-step rollout stability. The conceptual shift from autoregressive physical space rollout to our paradigm is illustrated in Fig. 1 . Motivated by this insight, we propose Autoregression-Free Neural Operators (AFNO), a framework for modeling PDE temporal evolution in a restructured latent space. Specifically, AFNO first employs an FNO-based autoencoder to reorganize high-dimensional spatial fields, mapping physical fields to latent representations that preserve multi-scale structures. Next, in the latent space, we perform multi-step extrapolation on the discrete observed data via explicit numerical integration. To this end, a latent flow matching module is designed to supervise the latent vector field using finite-difference temporal derivatives extracted from latent trajectories, thereby enabling stable continuous-time evolution through numerical integration. AFNO also explicitly conditions the latent vector field on physical parameters and time steps through learned embeddings, enabling adaptive dynamics across parameter configurations within the same PDE family. Finally, AFNO decodes latent representations back to the physical space by exploiting inverse fast Fourier transform and the symmetry of the FNO architecture. The entire framework is trained end-to-end, jointly optimizing latent encoding, latent flow modeling, and decoding. Extensive experiments on six PDEBench [ 54 ] benchmarks demonstrate that AFNO outperforms baseline models in terms of accuracy and long-horizon stability. In particular, AFNO exhibits more gradual error growth and stronger prediction stability during long-horizon rollouts. At 100 rollout steps, it reduces the final mean squared error (MSE) and relative L_{2} error by 50–85% compared with the best-performing baseline. We further compare AFNO with recent methods designed for long-horizon rollout stability, and AFNO reduces the final-step MSE and relative L_{2} error by over 90% against the best-performing baseline. Moreover, theoretical analysis indicates that AFNO alleviates the recursive error amplification of autoregressive physical space rollout by replacing repeated operator application with continuous latent space evolution. Our contributions are as follows: • We introduce an autoregression-free operator learning paradigm for time-dependent PDEs, reformulating long-horizon prediction from recursive physical space rollout to latent dynamics modeling. • Based on this paradigm, we develop AFNO, a parameter-conditioned autoregression-free neural operator that combines structured latent representations with flow matching for stable latent space evolution. •

Read the complete text