Predictivity and Utility of Neural Surrogates of Multiscale PDEs Journal: Computing in Science & Engineering

Karthik Duraisamy · 2026

Read Predictivity and Utility of Neural Surrogates of Multiscale PDEs Journal: Computing in Science & Engineering by Karthik Duraisamy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20061.txt.

Introduction

Duraisamy argues that neural surrogates for multiscale partial differential equations are strong but bounded, not all-purpose rivals for classical solvers. Many benchmarks sit on low-dimensional manifolds where any decent reduced model fills in well. Spectral bias leaves fine detail blurry, and coarse-graining destroys information for good, so no training can recover what coarse data discard. Weather outlooks on reanalysis data are a sweet spot, but chaotic multiscale cases will not carry over, and hybrids plus better reports are urged.

Catalog record

CREATORKarthik Duraisamy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45807
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.20061

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Predictivity and Utility of Neural Surrogates of Multiscale PDEs Journal: Computing in Science & Engineering Karthik Duraisamy Abstract Scientific machine learning is increasingly being spoken of as universal emulators for classical numerical solvers for multi-scale partial differential equations, but most apparent successes can be explained by facts that also define their limits. Many successful benchmarks live on low-dimensional solution manifolds where any competent reduced model will interpolate well. More fundamentally, neural surrogates systematically under-resolve high-frequency content due to spectral bias, and coarse-graining compounds this problem through irreversible information loss. In many multi-scale problems, no architecture or training procedure can fully recover what the coarse representation discards. Two simple examples are used to characterize spectral bias, coarse-graining and error accumulation. We discuss why medium-range weather prediction on reanalysis data sits in a favorable sweet spot and why this will not generalize to genuinely chaotic multi-scale scenarios. We identify domains where neural surrogates offer genuine value, propose further research on neural-classical hybrids, and call for better reporting standards. \jvol XX \jnum XX \paper XX \jmonth Month \jtitle Controversies and Perspectives on AI/ML in Science & Engineering \sptitle Theme Article: Controversies and Perspectives in AI/ML \chapteri Scientific machine learning (SciML) has produced a fast-growing zoo of neural surrogates for multiscale partial differential equations (PDEs), often accompanied with maximalist claims such as “replace the solver” or “1000 \times speedups.” Many successes reflect one of two situations: The first situation is interpolation on a benign manifold . When the parameter-to-solution map is smooth and the solution set has low intrinsic dimension: rapidly decaying Kolmogorov n-width [ 16 ] , any competent approximation method will succeed. Polynomial surrogates, projection-based Reduced Order Models (ROMs), or neural networks all achieve similar accuracy on such problems. The key diagnostic is whether classical reduced-order methods with comparable offline cost achieve similar accuracy [ 6 ] ; if so, the benchmark tests interpolation, not the capacity to handle multiscale physics. The second situation is confusing attractor sampling with prediction . Showing that a surrogate reproduces mean profiles or energy spectra on a chaotic attractor demonstrates that it has learned to sample the invariant measure [ 17 ] . This is useful for estimating long-time averages but categorically different from forecasting: the model may have no skill at predicting which region of the attractor the system will visit next. The distinction between learning dynamics and learning statistics is well-understood in dynamical systems but often glossed over in SciML benchmarks. An important issue cuts across both of the aforementioned situations: neural networks impose an implicit low-pass prior on what they learn, a phenomenon known as spectral bias . Rahaman et al. [ 7 ] showed that networks trained by gradient descent fit low-frequency components first, a phenomenon now understood through the Neural Tangent Kernel (NTK) [ 8 ] . In essence, the kernel eigenvalues associated with high-frequency Fourier modes decay polynomially, so that gradient-based training resolves low-frequency structure orders of magnitude faster than fine-scale content. Wider networks shift the spectrum but generally preserve the decay rate; more data cannot create gradient signal for modes where the eigenvalue is effectively zero. Architectural choices make this explicit: Fourier neural operators truncate to k_{\max} modes by design [ 5 ] , and even “super-resolution” variants cannot recover information never represented. For PDEs, spectral bias is not an aesthetic defect as in image processing: high- k content encodes boundary layers, sharp fronts, and fluctuations that control derivative-dependent quantities of interest: e.g. fluxes, stresses, dissipation rates. A surrogate can achieve excellent L^{2} error while systematically misrepresenting the physics that matters. This explains persistent reports of PINNs failing on “slightly harder” problems and neural operators degrading under autoregressive rollout [ 18 ] . Coarse-graining introduces a deeper problem: irreversible information loss . A network trained with mean-squared error converges to the conditional expectation \hat{u}(u_{c})=\mathbb{E}[u\mid u_{c}] [ 9 ] , i.e. the average over all fine states consistent with the coarse input. This is simply the optimal solution to Mean Squared Error (/MSE) minimization . The resulting prediction is over-smoothed by construction, and no architecture or training procedure can recover information the coarse representation discards. We note that many of these challenges (e.g. closure modeling, projection-induced information loss, and instability under long-horizon rollout) are well established in the classical ROM literature, and comments above apply equally to any coarse representation, neural or otherwise. The distinction is that classical intrusive ROMs retain explicit access to the governing equations in discrete form , whereas neural surrogates trade this structural access for flexibility and non-intrusiveness; the hybrid strategies discussed later in this paper can be viewed as attempts to recover some of these structural guarantees by periodically re-coupling the surrogate with a physics-based solver. These observations reframe how we should evaluate neural PDE surrogates. Related issues have been raised by others as well (e.g. [ 6 , 28 ] ). In this work, we provide some partial answers and focus on : What is being predicted: trajectories, statistics, or merely interpolated parameter responses? On what horizon, measured in relevant physical time scales? And do the metrics probe the scales that control the quantities of interest, or do they hide spectral deficiencies behind smooth headline numbers? This paper illustrates with simple examples, some interacting issues that limit predictive capabilities of neural surrogates for multiscale PDEs: spectral bias , coarse-graining information loss , and autoregressive error accumulation . We also identify problem classes where neural surrogates offer genuine value and discuss hybrid neural–classical strategies as a path forward. We distinguish two evaluation criteria: predictivity (accuracy on physically relevant quantities of interest under extrapolation) and utility (end-to-end cost advantage over classical methods), arguing that conflating the two has led to overoptimistic assessments. Predictivity and Utility We try to separate two questions that are often conflated. Predictivity asks whether a surrogate remains accurate under the extrapolations that matter for the scientific task: longer horizons, distribution shift, and/or resolution refinement, as measured on physically relevant Quantities of Interest (QoI), often derivative- or phase-sensitive). Utility asks whether the surrogate reduces end-to-end time-to-QoI relative to optimized solvers and classical reduced models once training cost, inference cost, and amortization over the intended query count are included. A method can be useful without being truly predictive (e.g., accurate long-time statistics with trajectory errors), and it can be predictive without being useful (e.g., accurate but not competitive end-to-end from a cost consideration standpoint). Also, these distinctions vary from task to task. In design optimization, end-to-end cost is extremely important. In short term weather prediction, only the inferencing cost is relevant. This paper exposes some of these issues in a simple setting (codes available at https://github.com/CaslabUM/NeuralSurrogates ), and provides some perspectives on how to move forward. A frequency-response lens on PDE surrogates The interplay between spectral bias and PDE structure is best understood by examining how three frequency-dependent quantities interact. This section introduces a unifying lens by connecting the PDE’s own spectral content, the operator’s frequency response, and the learning dynamics of the network. This lens will be used to diagnose when and why neural surrogates succeed or fail. For PDE surrogates, predictive ability becomes explanatory only when combined with the PDE’s own frequency response and with the metric/QoI being reported. Three “spectra” interact:

Read the complete text