Bridging Spectral Operator Learning and U-Net Hierarchies: SpectraNet for Stable Autoregressive PDE Surrogates

Enrique Hernández Noguera, Md Meftahul Ferdaus, Elias Ioup, Mahdi Abdelguerfi, Julian Simeonov, [10pt] University of New Orleans, New Orleans, LA, USA · 2026

Read Bridging Spectral Operator Learning and U-Net Hierarchies: SpectraNet for Stable Autoregressive PDE Surrogates by Enrique Hernández Noguera, Md Meftahul Ferdaus, Elias Ioup, Mahdi Abdelguerfi, Julian Simeonov, [10pt] University of New Orleans, New Orleans, LA, USA (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09096.txt.

Introduction

Enrique Hernández Noguera leads four co-authors in presenting SpectraNet, a neural model meant to forecast how systems governed by partial differential equations change over time. Their 2026 work tackles a known tension: spectral models let errors grow fast during long rollouts, while layered U-Net models lose some fine detail. SpectraNet places trimmed spectral steps inside a U-Net hierarchy and trains it with a consistency loss.

Tests against sixteen published baselines cover fluid flow and several other equation sets. The authors report lower error than the standard Fourier model with far fewer parameters, stable free rollouts where that model diverges, and fast running times on an ordinary processor. They are careful to note that a full attention Transformer still wins on raw error alone. The value lies in a light, stable surrogate that stays usable over long forecasts.

Catalog record

CREATOREnrique Hernández Noguera, Md Meftahul Ferdaus, Elias Ioup, Mahdi Abdelguerfi, Julian Simeonov, [10pt] University of New Orleans, New Orleans, LA, USA
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.09096

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bridging Spectral Operator Learning and U-Net Hierarchies: SpectraNet for Stable Autoregressive PDE Surrogates Enrique Hernández Noguera, Md Meftahul Ferdaus, Elias Ioup, Mahdi Abdelguerfi, Julian Simeonov, [10pt] University of New Orleans, New Orleans, LA, USA Abstract Neural operators for time-dependent PDEs face a structural tension: spectral architectures (FNO and descendants) inherit exponential rollout-error growth from their one-step Lipschitz constant, while hierarchical U-Net operators trade resolution invariance for multi-scale detail. We introduce SpectraNet , an autoregressive neural operator that composes truncated spectral convolutions inside a U-Net hierarchy with a Residual-Target Spectral Block trained under a Semigroup-Consistency Loss . The residual-target parametrization replaces L T L^{T} stability blow-up with linear T ​ δ T\delta drift (Theorem 1 , conditional on staying in 𝒦 \mathcal{K} ), and the spectral path’s parameter count is Θ ⁡ ( L ​ w 2 ​ M 2 ) \Theta(Lw^{2}M^{2}) , independent of grid N N . Under a single unified protocol against 16 16 published neural-operator baselines on Navier–Stokes ν = 10 − 5 \nu{=}10^{-5} at 64 × 64 64{\times}64 , SpectraNet reaches test relative L 2 = 0.0822 L^{2}{=}0.0822 at 2.04 ​ M 2.04\,\text{M} parameters — 2.33 × 2.33{\times} fewer than canonical FNO at ≈ 20 % \approx\!20\% lower error — and wins five of six rows in a cross-PDE comparison against FNO (NS at ν ∈ { 10 − 4 , 10 − 3 } \nu\!\in\!\{10^{-4},10^{-3}\} , PDEBench Shallow-Water 2D and Diffusion-Reaction, with the Active-Matter row going to FNO inside its seed spread). Trained from scratch at native 128 2 128^{2} under the same protocol, SpectraNet improves to 0.0724 0.0724 while FNO regresses to 0.3080 0.3080 ; we caveat this in Section 7.4 since the unified protocol is not tuned per-architecture per-resolution. Free rollout stays bounded for T = 100 T{=}100 where FNO diverges across all 200 200 test trajectories. On consumer CPU at B = 1 B{=}1 , SpectraNet runs sub- 200 ​ ms 200\,\text{ms} while the full-attention Transformer that wins raw L 2 L^{2} pays ∼ 60 × \sim\!60{\times} latency; we do not claim to beat that Transformer on raw L 2 L^{2} , only to dominate the lightweight ( ≤ 5 ​ M \leq\!5\,\text{M} parameter, sub- 200 ​ ms 200\,\text{ms} CPU) Pareto frontier. Source code is available at https://github.com/Enrikkk/spectranet . 1 Introduction Neural operators amortize the cost of solving time-dependent partial differential equations (PDEs) ( 10 ; 7 ; 19 ) by learning the solution operator \Phi_{\Delta t}:\omega_{t}\!\mapsto\!\omega_{t+\Delta t} from data, then rolling it out autoregressively (AR) to predict trajectories. Two failure modes dominate this regime. First, any one-step error compounds across T rollouts at a rate set by the operator’s one-step Lipschitz constant; for direct-prediction architectures this is generically exponential in T . Second, existing operators choose between resolution invariance and multi-scale detail rather than combining them: spectral architectures (the Fourier Neural Operator, FNO ( 10 ) , and its descendants) inherit a fixed band-limit but no spatial hierarchy; U-Net ( 18 ) hybrids recover detail but lose resolution invariance at down/up-sampling; and transformer operators replace the spectral mixer with attention that scales as \mathcal{O}(N^{2}) in the spatial token count N . Gap. No published neural operator combines all four ingredients that Theorem 1 and Proposition 2 below identify as jointly necessary at the 64^{2} – 256^{2} regime: (i) truncated spectral mixing for resolution-aware approximation, (ii) a U-Net hierarchy for multi-scale capture, (iii) a residual-target rollout parametrization that bounds long-horizon drift, and (iv) a training objective that enforces semigroup consistency at the trajectory level. FNO has only (i); U-FNO ( 24 ) and U-NO ( 16 ) have (i)+(ii); transformers have neither; (iii) and (iv) have not been combined with spectral or U-Net mixers in the AR-rollout regime. SpectraNet. We introduce SpectraNet , an autoregressive neural operator that bridges spectral operator learning and U-Net hierarchies through a Residual-Target Spectral Block that parametrizes the per-step update as f_{\theta}=\mathrm{id}+\Delta_{\theta} , together with a Semigroup-Consistency Loss that ties two single-step applications f_{\theta}\!\circ\!f_{\theta}(\omega_{t}) to the two-step ground truth \omega_{t+2} . The headline operating point uses three encoder–decoder levels, channel width w{=}32 , spectral truncation radius M{=}12 (the number of low-frequency Fourier modes retained per axis), and 2.04\,\text{M} parameters — 2.33{\times} fewer than canonical FNO at \approx\!20\% lower error on Navier–Stokes (NS) at viscosity \nu=10^{-5} . Contributions. (C1) Architecture ( Section 4 ): a 2D autoregressive spectral U-Net with the two named training-time innovations above; the canonical operating point sits on the lightweight ( \leq\!5\,\text{M} parameter, sub- 200\,\text{ms} CPU) Pareto frontier of the NS \nu{=}10^{-5} leaderboard, dominating every published baseline within that envelope ( Table 3 ). The full-attention NSL Transformer wins raw L^{2} outside this envelope and we do not claim to beat it on accuracy. (C2) Theory ( Section 5 ): an Approximation–Stability decomposition ( Theorem 1 ) bounding the T -step stability term by \mathcal{O}(T\delta) under residual-target parametrization vs \mathcal{O}(L^{T}\varepsilon_{0}) for direct prediction with one-step Lipschitz L\!>\!1 ; Lemma 1 (b) is the standard residual-network observation, and the contribution is anchoring it to an empirical operator with measured \delta,\widehat{L} ( Table 8 ). Proposition 2 bounds the spectral path at \Theta(Lw^{2}M^{2}) , independent of N (shared with FNO). (C3) Multi-PDE evidence ( Section 7.1 ): a unified-protocol evaluation over six PDE benchmarks in which SpectraNet beats FNO on five of six rows at 2.33{\times} smaller parameter count; the NS row is two-seed and the remaining five are single-seed ( Section 10 ). At native 128^{2} under the same protocol SpectraNet improves to 0.0724 while FNO regresses to 0.3080 ; caveat in Section 7.4 . (C4) Deployment cost ( Section 9 ): a CPU-extended inference-cost study showing SpectraNet runs sub- 200\,\text{ms} on consumer CPU while the full-attention Transformer that wins raw L^{2} takes \sim\!10\,\text{s} per sample. 2 Related Work We position SpectraNet against four prior families of regular-grid neural operators, each embodying one or two of the four ingredients from Section 1 . Spectral neural operators. The Fourier Neural Operator ( 10 ) is the ancestor of SpectraNet’s spectral path: pointwise channel mixes interleaved with truncated spectral convolutions, granting (i) resolution invariance and translation equivariance up to the bandlimit ( 3 ) . Variants address specific failure modes: F-FNO ( 22 ) factorizes 2D spectral convolutions for memory; U-FNO ( 24 ) adds a U-Net hierarchy on top; U-NO ( 16 ) extends with multi-resolution spectral mixing; LSM ( 25 ) learns a basis; MWT ( 4 ) uses multi-wavelets. SpectraNet inherits the FNO spectral convolution but adds (a) the Residual-Target Spectral Block ( Section 4.4 ) and (b) the Semigroup-Consistency Loss ( Section 4.5 ). U-FNO/U-NO add the hierarchy but neither has a residual-target rollout or a multi-step training objective; both inherit the direct-prediction L^{T} stability blow-up of Theorem 1 (i). Transformer operators. Galerkin ( 2 ) , FactFormer ( 9 ) , OFormer ( 8 ) , Transolver ( 26 ) , GNOT ( 5 ) all replace the spectral mixer with sub- \mathcal{O}(N^{2}) approximate attention. Only the NSL Transformer ( 27 ) retains exact softmax and clears FNO at 64^{2} — a resolution-dependent result documented in Appendix D . Proposition 2 predicts these transformers’ parameter cost grows polynomially with grid resolution, borne out by their 39 – 277\,\text{M} counts ( Table 3

Read the complete text