NeurIPS 2024 ML4CFD Competition: Results and Retrospective Analysis

Mouadh Yagoubi, David Danan, Milad Leyli-Abadi, Ahmed Mazari, Jean-Patrick Brunet, Abbas Kabalan, Fabien Casenave, Yuxin Ma, Giovanni Catalani, Jean Fesquet, Jacob Helwig, Xuan Zhang, Haiyang Yu, Xavier Bertrand, Frederic Tost, Michael Baurheim, Joseph Morlier, Shuiwang Ji · 2025

Read NeurIPS 2024 ML4CFD Competition: Results and Retrospective Analysis by Mouadh Yagoubi, David Danan, Milad Leyli-Abadi, Ahmed Mazari, Jean-Patrick Brunet, Abbas Kabalan, Fabien Casenave, Yuxin Ma, Giovanni Catalani, Jean Fesquet, Jacob Helwig, Xuan Zhang, Haiyang Yu, Xavier Bertrand, Frederic Tost, Michael Baurheim, Joseph Morlier, Shuiwang Ji (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08516.txt.

Introduction

Mouadh Yagoubi and co-authors report on a contest to speed up airflow simulation with machine learning. Fluid flow codes are exact but slow, which limits design work. The contest asked teams to build stand in models for flow over wing shapes. Over two hundred forty teams took part, working from data made with an open source solver.

Entries were scored on accuracy, respect for physics, speed, and skill on unseen cases. Several beat the set baselines under the joint score. The best entry even topped the real solver on the combined measures. The authors draw lessons from the top designs and give advice for running later contests in science machine learning.

Catalog record

CREATORMouadh Yagoubi, David Danan, Milad Leyli-Abadi, Ahmed Mazari, Jean-Patrick Brunet, Abbas Kabalan, Fabien Casenave, Yuxin Ma, Giovanni Catalani, Jean Fesquet, Jacob Helwig, Xuan Zhang, Haiyang Yu, Xavier Bertrand, Frederic Tost, Michael Baurheim, Joseph Morlier, Shuiwang Ji
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE109 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-08516

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

NeurIPS 2024 ML4CFD Competition: Results and Retrospective Analysis Mouadh Yagoubi, David Danan, Milad Leyli-Abadi, Ahmed Mazari, Jean-Patrick Brunet, Abbas Kabalan, Fabien Casenave, Yuxin Ma, Giovanni Catalani, Jean Fesquet, Jacob Helwig, Xuan Zhang, Haiyang Yu, Xavier Bertrand, Frederic Tost, Michael Baurheim, Joseph Morlier, Shuiwang Ji Abstract The integration of machine learning (ML) into the physical sciences is reshaping computational paradigms, offering the potential to accelerate demanding simulations such as computational fluid dynamics (CFD). Yet, persistent challenges in accuracy, generalization, and physical consistency hinder the practical deployment of ML models in scientific domains. To address these limitations and systematically benchmark progress, we organized the ML4CFD competition, centered on surrogate modeling for aerodynamic simulations over two-dimensional airfoils. The competition attracted over 240 teams, who were provided with a curated dataset generated via OpenFOAM and evaluated through a multi-criteria framework encompassing predictive accuracy, physical fidelity, computational efficiency, and out-of-distribution generalization. This retrospective analysis reviews the competition outcomes, highlighting several approaches that outperformed baselines under our global evaluation score. Notably, the top entry exceeded the performance of the original OpenFOAM solver on aggregate metrics, illustrating the promise of ML-based surrogates to outperform traditional solvers under tailored criteria. Drawing from these results, we analyze the key design principles of top submissions, assess the robustness of our evaluation framework, and offer guidance for future scientific ML challenges. 1 Introduction Machine learning (ML) techniques, and deep learning (DL) in particular, have achieved transformative breakthroughs in domains such as computer vision, natural language processing, and speech recognition. Their influence is now extending to the physical sciences, where ML-based surrogate models offer the potential to accelerate computationally intensive tasks like those in computational fluid dynamics (CFD) (see e.g., tompson2016 ; kasim2021building ; rasp_deep_2018 ; sanchez2020learning ; menier2023cd ). By replacing or augmenting parts of traditional solvers, ML can significantly reduce simulation time, making high-resolution analysis feasible for real-time design and decision-making. However, real-world deployment poses challenges in stability, generalization, and physical consistency. CFD remains a critical bottleneck in many industrial workflows due to its high computational cost. Each simulation can take hours to run, particularly when resolving turbulent flows or boundary layers with fine spatial and temporal resolution. This hinders large-scale design space exploration, optimization, and uncertainty quantification. Moreover, CFD fields are inherently multi-scale: while pressure tends to vary more gradually than velocity in many regions, sharp gradients can still arise near leading edges, separation zones, or areas of adverse pressure recovery. In contrast, velocity fields often exhibit steep gradients within boundary layers, making surrogate learning particularly difficult in these regions. Bridging the gap between ML research and industrial deployment remains a key challenge. While the fusion of ML and physics is a vibrant area, much of the work remains limited to academic benchmarks or idealized settings. Initiatives like the Machine Learning and the Physical Sciences workshop ml4ps2022 , and competitions in molecular simulations open_catalyst , particle physics adam2015higgs , and robotics cea_challenge , reflect growing interest in translational ML. Recent efforts (e.g., physics4ml ; equer2023multiscale ; chughtai2023neural ; menier2025interpretable ) show how incorporating physical priors can improve robustness and interpretability. Still, few benchmarks address deployment bottlenecks in high-fidelity industrial CFD, where accuracy, latency, and generalization must be balanced. To address this, we organized a competition focused on a representative CFD task with industrial relevance: steady-state aerodynamic simulations of 2D airfoils, using a dataset designed with real-world constraints in mind. Participants predicted velocity, pressure, and turbulent viscosity from mesh-based inputs while ensuring physical consistency. Evaluation used the LIPS (Learning Industrial Physical Simulation) platform leyli2022lips , combining ML metrics (accuracy, inference time) and physics-aware criteria (e.g., OOD generalization, physical law compliance). The competition drew over 240 teams and highlighted a diverse range of methods from PCA-based Gaussian Processes to graph neural networks and coordinate-based neural fields. This methodological variety reflects the difficulty of building surrogates that are simultaneously accurate, fast, and deployment-ready. Generalization across different airfoil geometries, especially under out-of-distribution conditions, emerged as a core challenge. Successful methods often incorporated geometric inductive biases, which proved essential for learning physically coherent surrogates suitable for downstream tasks. In this retrospective, we revisit the competition’s structure, results, and lessons learned. We analyze the diversity of submitted approaches, assess the evaluation framework, and discuss practical trade-offs for ML deployment in CFD. Our goal is to guide the development of surrogates that are not only high-performing, but viable for real-world applications. 2 Competition Overview The airfoil use-case We consider a canonical benchmark in computational aerodynamics: the simulation of steady, incompressible, subsonic flow past 2D airfoils under high Reynolds number conditions. While originally framed as a shape optimization task seeking geometries with optimal lift-to-drag ( C_{L}/C_{D} ) ratios, our objective shifts towards data-driven emulation of the underlying physics. Specifically, we aim to learn surrogates that approximate the solution operator mapping geometric and flow boundary conditions to the full-field solution of the incompressible Reynolds-Averaged Navier–Stokes (RANS) equations. All geometries are parameterized by the NACA 4- and 5-digit series NACA , operating at sea-level, T=298.15 K, with Reynolds numbers ranging from 2\times 10^{6} to 6\times 10^{6} , ensuring subsonic Mach numbers ( M<0.3 ) and preserving incompressibility assumptions bonnet2022airfrans . Despite the 2D simplification, the regime remains highly nontrivial due to transitional and turbulent boundary-layer dynamics, particularly near-wall anisotropy and multi-scale interactions that emerge at high Reynolds numbers. Capturing these effects is essential for accurate prediction of full flow fields (velocity, pressure, turbulent viscosity) and integrated quantities such as lift and drag coefficients. These interactions pose challenges to both conventional solvers and neural surrogates, particularly in extrapolative regimes kochkov2021machine ; brandstetter2022message . Ground-truth simulations are generated using stabilized finite volume methods in OpenFOAM, relying on a structured C-mesh with 250 – 300 k cells and y^{+}\approx 1 near the wall bonnet2022airfrans . The governing PDE system incorporates a two-equation turbulence closure via the SST k – \omega model. The ensemble-averaged incompressible RANS equations are: \begin{cases}\partial_{i}\overline{u}_{i}=0\\ \partial_{j}(\overline{u}_{i}\overline{u}_{j})=-\partial_{i}(\frac{\overline{p}}{\rho})+(\nu+\nu_{t})\partial^{2}_{jj}\overline{u}_{i},\quad i\in\{1,2\},\end{cases}\, (1) where \overline{.} denotes an ensemble averaged quantity, \partial_{i} the partial derivative with respect to the i-th spatial components, u the fluid velocity, p an effective pressure, \rho the fluid specific mass, \nu the fluid kinematic viscosity, \nu_{t} the fluid kinematic turbulent viscosity and where we used the Einstein summation convention over repeated indices. For more details about the physical setting, we refer to Appendix A . Dataset specification and structure The dataset is derived from the AirfRANS suite bonnet2022airfrans ; yagoubi2024neurips , encompassing simulations over a range of aerodynamic configurations with controlled variation in Reynolds number and angle of attack. Three subsets are constructed to evaluate generalization under different ML regimes: • Training set (scarce regime) : 103 samples drawn from the ‘scarce’ scenario, filtered to retain only instances with 3\times 10^{6}\leq\text{Re}\leq 5\times 10^{6} . • In-distribution test set : 200 samples from the ‘full’ test split, within the same Reynolds regime. • Out-of-distribution (OOD) test set : 496 samples with extrapolated Reynolds ranges ( [2,3]\cup[5,6]\times 10^{6} ), absent from training. Each simulation is provided as an unstructured point cloud, cropped to [-2,4]\times[-1.5,1.5] m, with per-node input features: 2D inlet velocity (m/s), distance to the airfoil (m), and unit normals (m) zeroed outside the surface. Supervision targets include: velocity components ( \overline{u}_{x} , \overline{u}_{y} ), reduced pressure ( \overline{p}/\rho ), and

Read the complete text