Towards Multi-Fidelity Scaling Laws of Neural Surrogates in CFD

Paul Setinek Gianluca Galletti Johannes Brandstetter · 2025

Read Towards Multi-Fidelity Scaling Laws of Neural Surrogates in CFD by Paul Setinek Gianluca Galletti Johannes Brandstetter (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01830.txt.

Introduction

Paul Setinek, Gianluca Galletti, and Johannes Brandstetter ask how to budget simulation data for neural surrogates of fluid flow. Scaling laws in language and vision assume data is cheap. In computational fluid dynamics, every training example costs a simulation. But a simulation's fidelity can be traded for speed, by adjusting its assumptions. Other fields lack that dial.

Using low- and high-fidelity Reynolds-averaged Navier-Stokes simulations, the authors recast the classical scaling laws, splitting the dataset axis into compute budget and dataset composition. Experiments reveal clean compute-performance scaling and that the best mixture of cheap and expensive data depends on the budget available. Theirs is the first study of its kind on multi-fidelity scaling laws for neural surrogates, with practical advice for generating training data efficiently.

Catalog record

CREATORPaul Setinek Gianluca Galletti Johannes Brandstetter
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-01830

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Multi-Fidelity Scaling Laws of Neural Surrogates in CFD Paul Setinek Gianluca Galletti Johannes Brandstetter Abstract Scaling laws describe how model performance grows with data, parameters and compute. While large datasets can usually be collected at relatively low cost in domains such as language or vision, scientific machine learning is often limited by the high expense of generating training data through numerical simulations. However, by adjusting modeling assumptions and approximations, simulation fidelity can be traded for computational cost, an aspect absent in other domains. We investigate this trade-off between data fidelity and cost in neural surrogates using low- and high-fidelity Reynolds-Averaged Navier-Stokes (RANS) simulations. Reformulating classical scaling laws, we decompose the dataset axis into compute budget and dataset composition. Our experiments reveal compute-performance scaling behavior and exhibit budget-dependent optimal fidelity mixes for the given dataset configuration. These findings provide the first study of empirical scaling laws for multi-fidelity neural surrogate datasets and offer practical considerations for compute-efficient dataset generation in scientific machine learning. \workshoptitle AI for Science LLM Large Language Model CV Computer Vision PDE Partial Differential Equation CFD Computational Fluid Dynamics LES Large Eddy Simulation RANS Reynolds-Averaged Navier-Stokes Re Reynolds number AoA Angle of Attack FVM Finite Volume Method DNS Direct Numerical Simulation MSE Mean Squared Error SOTA state of the art nMAE normalized Mean Absolute Error 1 Introduction Machine learning has seen immense progress in recent years, which was not only driven by architectural or methodological innovations but also by the increasing availability of computational resources. This has enabled the scaling up of models, and as a result many SOTA models now contain hundreds of billions of parameters [ 38 , 2 ] . Scaling laws, which originated in the domain of LLM [ 20 , 18 ] , have expanded into various other areas like CV ( CV ) [ 46 ] or time series [ 35 , 44 ] . These empirical studies describe how models improve as a function of three axes : (i) parameter count ( N ), (ii) dataset size ( D ), and (iii) compute ( C ). In the meantime, scientific machine learning has similarly achieved remarkable success in modeling complex systems with neural surrogates. Notable examples include breakthroughs in weather and climate forecasting [ 21 , 31 , 29 , 32 , 8 ] , material design [ 27 , 45 , 41 ] or protein folding [ 19 , 1 ] . These advances have partly been enabled by large curated public datasets, such as ERA5 in weather modeling [ 17 ] or the Protein Data Bank (PDB) [ 6 ] . More recently, first large scale datasets have also been released in areas such as automotive aerodynamics [ 4 , 14 ] . However, in many areas of science and engineering, such datasets are either not public or do not exist and therefore researchers are required to generate their own problem specific dataset prior to model training. Since the systems of interest are usually governed by PDE [ 15 ] , generating training data requires solving these equations numerically, which is often coupled with significant computational costs [ 43 ] . This means that unlike other domains, where data can be sourced with little to no computation ( e.g. text, real-world images and videos), the dataset size D in scientific machine learning is often no longer “free” to scale . While prior works study how performance scales with dataset size, they do not take the computational cost associated with scaling the dataset axis into account [ 37 , 16 , 3 , 30 ] . Moreover, many neural surrogates exist with the goal of “amortizing” the training and dataset cost with repeated, cheap evaluations down the line; if dataset generation becomes prohibitively expensive, it fundamentally defeats the purpose of such models. The numerical solution of PDE is a long-running research topic, which can be very nuanced: when designing numerical simulations, modeling assumptions and simplifications of the underlying physics need to be made, which trade fidelity for compute. Ordered by computational complexity, common approaches include DNS ( DNS ) which resolves all turbulent scales at prohibitive cost; LES ( LES ), which resolves only large scales while modeling subgrid-scale dynamics; and RANS ( RANS ), where turbulence is entirely modeled through statistical averaging, making RANS the cheapest but also least accurate of the three. For example, a 3D LES over an airfoil can take several orders of magnitude longer than simulating the respective RANS equations. Even on simple scenarios, LES take an order of magnitude longer than RANS simulations [ 23 ] , a factor which grows substantially for more chaotic systems. This introduces a fundamental trade-off, which motivates our research question: Under a fixed budget constraint, what is the optimal training set composition of low- and high-fidelity samples in order to maximize model performance? We move towards an answer to this essential question by proposing a reformulation of classical scaling laws, devised to account for this phenomenon. We argue for splitting the simple dataset size D axis into two components, namely dataset compute budget D_{b} (core hours allocated for data generation) and dataset composition D_{c} (controls the fidelity distribution of the training data). Since training data generation is typically the primary bottleneck in scientific machine learning, we focus on these two axes while assuming model size and compute for model training are non-limiting. Although restrictive, this assumption allows us to directly address our research question that has not been studied in prior work. Our contributions can be summarized as follows: • We introduce a formulation of multi-fidelity scaling laws, extending classical scaling law analysis to settings where training data can be simulated at different fidelities. • We design a multi-fidelity dataset of external aerodynamics around airfoils, incorporating different modeling assumptions across fidelity levels. • We present the first empirical investigation of multi-fidelity scaling laws along the data axis on a CFD ( CFD ) dataset, evaluating how model performance scales under varying dataset compositions and generation budgets. 2 Related work Learning from multi-fidelity data. Multi-fidelity data exists in different fields, and can come in different shapes and forms (e.g., varying realism, accuracy or resolution). In CV , “resolution transfer” [ 7 , 34 ] or “super-resolution” [ 13 , 28 ] are well researched directions, where models learn to predict fine-scale details from coarse observations. Similar ideas appear in scientific machine learning under “discretization convergence” in neural operators [ 22 ] , where models are trained to generalize across mesh resolutions. While these methods may be invariant to changes in resolution (even though most of them have no theoretical guarantee [ 5 ] ), they do not capture underlying physics and fidelity shifts. Transfer learning for multi-fidelity data in scientific machine learning. Recent studies have explored transfer learning from low- to high-fidelity data [ 12 , 36 , 25 ] . However, in these works the distinction between fidelities is limited to changes in mesh resolution, while the underlying physical modeling assumptions remain the same. In contrast, a more recent study takes this further by transferring knowledge from low-fidelity RANS to high-fidelity LES simulations in the context of wind farm modeling, thereby altering the underlying modeling assumptions [

Read the complete text