Striding Across Reynolds Numbers: Representation Geometry in Neural PDE Generalisation

Jianing Shi · 2026

Read Striding Across Reynolds Numbers: Representation Geometry in Neural PDE Generalisation by Jianing Shi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30112.txt.

Introduction

Jianing Shi examines why neural PDE solvers struggle when the Reynolds number shifts, on the forced two-dimensional Navier-Stokes benchmark. A trained Fourier Neural Operator errs by nearly half under a tenfold Reynolds shift, yet simple retrieval baselines matching states without any forward model do better. ConvAE-Relay matches states in a source-trained autoencoder space and borrows source dynamics, cutting error with no target fitting. Ablations pin matching quality as dominant, and a multi-scale U-Net does better still, suggesting representation geometry organizes cross-regime transfer.

Catalog record

CREATORJianing Shi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37140
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30112

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Striding Across Reynolds Numbers: Representation Geometry in Neural PDE Generalisation Jianing Shi Abstract Cross-Reynolds generalisation in neural PDE solvers remains poorly characterised. On the canonical forced 2D Navier–Stokes benchmark, a trained Fourier Neural Operator reaches 46.68 % 46.68\% relative L 2 L_{2} error under a 10 × 10{\times} Reynolds-number shift, yet zero-forward-model retrieval baselines already improve to 41 41 – 42 % 42\% . This suggests representation geometry as a major organising variable among the tested methods. We test this hypothesis through ConvAE-Relay, which matches states in a source-trained convolutional autoencoder latent space and borrows dynamics from a source-regime database, achieving 38.34 ± 0.07 % 38.34{\pm}0.07\% using only a source-regime database and no target-regime fitting, labels, or database entries. A 2 × 2 2{\times}2 ablation isolates matching quality as dominant over the update rule. Oracle experiments confirm that source-regime dynamics directions remain transferable (cosine similarity ∼ 0.84 {\sim}0.84 ) when matching stays on-manifold; autoregressive drift is the primary bottleneck ( ∼ 12 {\sim}12 percentage points). From the learned-prediction side, a U-Net with multi-scale skip connections achieves 34.72 ± 0.60 % 34.72{\pm}0.60\% , consistent with the retrieval-side finding that local, multi-scale representations organise cross-Reynolds transfer among tested methods. All claims are scoped to this benchmark. 1 Introduction A zero-forward-model analogue relay outperforms a trained Fourier Neural Operator under a 10{\times} Reynolds-number shift. Yet a learned U-Net predictor outperforms both. This apparent contradiction, retrieval beating spectral learning while a different learned architecture beats retrieval, points to a shared underlying variable: the geometry of the state representation used for cross-regime prediction. The Reynolds number \mathrm{Re}=UL/\nu governs the ratio of inertial to viscous forces; increasing Re activates finer spatial scales through weakened viscous damping ( Pope 2000 ) . Extrapolation across Re regimes is ubiquitous in engineering: wind-tunnel data applied to full-scale aerodynamics, moderate-resolution climate simulations extrapolated to finer regimes. This is not arbitrary covariate shift; it is extrapolation along an axis whose difficulty is governed by the Navier–Stokes equations themselves. Machine learning for fluid mechanics has advanced rapidly ( Brunton et al. 2020 ; Duraisamy et al. 2019 ) , yet cross-parameter generalisation remains underexplored. On the forced 2D Navier–Stokes benchmark of Li et al. 2021 , we evaluate six methods spanning learned predictors and zero-parameter retrieval. Three FNO-family variants concentrate at 46 – 48\% relative L_{2} error at \mathrm{Re}{\approx}10{,}000 , suggesting a persistent limitation of tested FNO-family protocols under this shift, although we do not claim exhaustive FNO optimisation. PCA relay ( 41.77\% ) and k NN-copy ( 41.01\% ), both zero-parameter methods, outperform FNO. This raises the question: is the bottleneck in the learned forward dynamics map, or in how states are represented? PCA relay shows that retrieval can transfer useful dynamics, but PCA imposes a global linear geometry on vorticity fields. We test whether a source-trained convolutional autoencoder provides a more indexable nonlinear representation: ConvAE-Relay achieves 38.34{\pm}0.07\% , an improvement of 3.4 percentage points (pp) over PCA relay with only the matching representation changed. From the learned-prediction side, a U-Net with skip connections achieves 34.72{\pm}0.60\% , the strongest tested result, while contrasting with the original in-distribution comparison where U-Net underperformed FNO ( Li et al. 2021 ) . Retrieval-side ablations and learned-predictor comparisons provide consistent evidence from two complementary perspectives that representation geometry organises cross-Re transfer. A 2{\times}2 ablation (matching space \times update rule) isolates matching quality as the dominant factor. Oracle experiments decompose the error into drift ( {\sim}12 pp), magnitude ( {\sim}3 pp), and a diagnostic floor ( 23.32\% ). Mechanism ablations (smart tracking, dynamics-aware AE, history matching) each fail to improve, sharpening the diagnosis by elimination. Boundary characterisation at \mathrm{Re}{=}100{,}000 reveals that relay’s advantage reverses, while long-horizon evaluation shows drift accumulation eroding matching quality by T{=}20 . Our contributions are: (1) On this benchmark under 10{\times} Re shift, tested methods organise by representation geometry, with local and multi-scale representations outperforming global spectral or global linear. (2) ConvAE-Relay instantiates learned-representation analogue forecasting without a learned forward dynamics map, enabling controlled matching/update diagnostics. (3) Oracle and ablation experiments identify autoregressive drift as the main deployable bottleneck, expose a matching-dynamics representation tradeoff, and separate predictive sufficiency from relay indexability. 2 Related Work Neural operators. FNO ( Li et al. 2021 ) established the benchmark for learned PDE surrogates via spectral-domain convolutions. Factorised variants ( Tran et al. 2023 ) and geometry-aware extensions ( Li et al. 2023 ) reduce cost or handle irregular domains. The neural operator framework has been formalised as learning maps between function spaces ( Kovachki et al. 2023 ) . Broader families, including DeepONet ( Lu et al. 2021 ) , graph-based simulators ( Sanchez-Gonzalez et al. 2020 ; Pfaff et al. 2021 ) , Transformer-based operators ( Cao 2021 ; Hao et al. 2023 ) , and multiwavelet operators ( Gupta et al. 2021 ) , deliver strong in-distribution performance, but cross-parameter generalisation is less studied. Out-of-distribution generalisation in scientific ML. Physics-informed approaches ( Raissi et al. 2019 ; Karniadakis et al. 2021 ) embed PDE structure into training but primarily target interpolation. Scale-consistent learning ( Li et al. 2024 ) modifies training through scale-aware augmentation. Subramanian et al. 2023 characterise scaling and transfer behaviour for scientific foundation models. Our work instead studies zero-target-regime diagnostics across representation families, including nonparametric transition reuse without a learned forward map. U-shaped neural operators (U-NO; Rahman et al. 2023 ) exist but evaluate in different settings. To our knowledge, no prior work reports systematic cross-Re zero-shot evaluation comparing spectral, retrieval, and multi-scale methods on the original forced FNO benchmark under a unified protocol. Autoregressive rollout strategies. Long-horizon neural PDE prediction suffers from error accumulation. Pushforward training ( Brandstetter et al. 2022 ) and diffusion-based refinement ( Lippe et al. 2023 ) address this from the training side. Data-driven weather forecasting ( Lam et al. 2023 ; Pathak et al. 2022 ) demonstrates that autoregressive neural surrogates can reach operational accuracy. Our oracle decomposition isolates drift as the dominant bottleneck, providing a diagnostic complement to these training-side solutions. Analogue forecasting. Lorenz 1969 introduced prediction from historical atmospheric archives via state matching; modern variants apply analogue methods to precipitation nowcasting ( Atencia & Zawadzki 2015 ) and other geophysical fields. k NN-LM ( Khandelwal et al. 2020 ) demonstrated that retrieval-augmented prediction can match or surpass trained language models, especially under domain shift. Representation quality for retrieval has been studied extensively in NLP ( Bengio et al. 2013 ) but not in PDE dynamics transfer. ConvAE-Relay bridges this gap: learned-representation analogue forecasting adapted to cross-regime PDE dynamics. 3 Problem Setup and Methods Figure 1 : Cross-Reynolds diagnostic summary. (a) Vorticity predictions and absolute error maps at \mathrm{Re}{=}10{,}000 (rollout step 5, median-error trajectory). FNO produces visibly smoother fields; ConvAE-Relay and U-Net preserve finer structure. (b) 2{\times}2 representation ablation (seed 42): upgrading matching space from PCA to ConvAE latent improves the best result by 2.37 pp, while the update-rule effect is {\leq}1.1 pp; matching quality dominates. (c) Oracle bottleneck decomposition: autoregressive drift accounts for {\sim}12 pp, magnitude mismatch {\sim}2 pp, and residual {\sim}1 pp. Oracle variants are target-informed diagnostic probes, not deployable methods. 3.1 Benchmark and Task We study the forced 2D vorticity Navier–Stokes equations on a 64{\times}64 periodic domain ( Li et

Read the complete text