Inpainting physics: self-supervised learning for context-driven fluid simulation
2026
Read Inpainting physics: self-supervised learning for context-driven fluid simulation (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08832.txt.
Introduction
Jonas Weidner, Yeray Martin-Ruisanchez, Daniel Rueckert, Benedikt Wiestler, and Julian Suk recast fluid simulation as inpainting: fill the unknown flow given the parts already fixed. Standard neural surrogates demand explicit geometry and boundary settings, tying them to training conditions and blocking reuse when boundaries shift. Their self supervised approach learns a prior over velocity fields from data alone, then imposes inlets, outlets, or unchanged regions only when solving.
A local neighborhood tokenizer compresses fine three dimensional fields into compact tokens for generative models. On blood flow in brain aneurysms, the method rebuilds full fields from sparse context, beats supervised rivals under shifts, and allows local geometry edits reusing prior context.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08832
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Inpainting physics: self-supervised learning for context-driven fluid simulation Abstract Neural surrogate models for computational fluid dynamics (CFD) are typically trained as forward operators that map explicit problem specifications, such as geometry and boundary conditions, to solution fields. This ties the model to the conditioning variables seen during training and limits reuse under boundary-condition shifts or local geometry changes. We propose to reformulate steady CFD inference as an inpainting problem: instead of training on explicit boundary conditions, we learn a self-supervised prior over velocity fields and impose boundary constraints only during inference by fixing known regions such as inlet, outlet or unchanged regions from previous simulations. To scale this idea to large 3D meshes, we introduce a local neighbourhood tokeniser that represents high-resolution velocity fields as compact spatial latent tokens and train latent flow-matching and masked-autoencoder models on these tokens. On intracranial aneurysm hemodynamics, our method reconstructs full velocity fields from sparse boundary context, outperforms supervised neural surrogates under boundary-condition and dataset shift and enables local geometry editing by reusing unchanged simulation context. These results suggest that viewing CFD inference as context-conditioned inpainting can turn neural surrogates from task-specific predictors into reusable flow priors. Jonas Weidner 1,2 , Yeray Martin-Ruisanchez 1 , Daniel Rueckert 2,3,4 , Benedikt Wiestler 1,2,∗ , Julian Suk 2,3,∗ [email protected] 1 AI for Image-Guided Diagnosis and Therapy, Technical University of Munich 2 Munich Center for Machine Learning (MCML) 3 AI in Healthcare and Medicine, Technical University of Munich 4 Imperial College London ∗ Shared senior authorship 1 Introduction Figure 1: Inpainting physics. (1) We tokenise raw velocity fields into local ball-shaped latent representations. (2) We train a self-supervised model on these tokenised velocity fields using latent flow matching or a masked autoencoder. (3) At inference, boundary conditions are explicitly enforced by fixing known regions like inflow and outflow during inpainting, enabling generalisation to unseen geometries and flow conditions, preserving local flow structure. Computational fluid dynamics (CFD) is a central tool in, e.g., automotive engineering, climate science and biomedical modelling, but high-fidelity numerical simulations remain expensive 3 ; 4 ; 21 . Each new geometry, boundary condition or design variant typically requires solving a large discretised system from scratch. Neural surrogates aim to reduce this cost by learning an approximate map from problem specification to quantities of interest. Neural surrogates for CFD are typically trained as supervised neural operators: given a defined geometry, physical parameters and boundary conditions, they directly regress the corresponding velocity or pressure field 30 ; 20 ; 2 . This formulation is powerful in-distribution, but strongly couples the learned model to the conditioning variables and data-generating process used during training. As a result, extrapolation to new boundary conditions, geometries or solver settings remains a major obstacle. We begin by observing that laminar fluid flow correlates highly with its spatial context. Due to viscosity effects and momentum, the union of upstream and downstream dynamics is an excellent prior for local flow. 1 1 1 The totality of such local interactions are the basis of continuum-based fluid simulation. Despite this, supervised neural operators do not explicitly leverage spatial context when producing flow estimates. We argue that including spatial context could greatly improve out-of-distribution performance in neural surrogates. Inspired by the success of generative modelling in computer vision 26 , we formulate (steady) CFD emulation not as deterministic regression but as (probabilistic) generation. In other words, we recast the supervised learning of flow states based on their underlying Dirichlet boundary conditions to self-supervised, unconditional learning in conjunction with conditioning at inference time . Our goal is to learn a reusable prior over all flow states and impose Dirichlet boundary conditions only at inference time via inpainting . The generative model can then leverage upstream and downstream observations and infer dynamical parameters implicitly from the context. 2 2 2 We remark that the inpainting formulation smoothly transitions into the regression formulation if we increase the size of the inpainting region (until the Dirichlet boundary). This separation of representation learning and context-driven inference changes how neural surrogates can be used. A pre-trained model can be queried with different conditioning patterns: sparse inlet-outlet constraints (similar to the supervised setting) as well as larger measured or otherwise known spatial context. Furthermore, the inpainting formulation enables local editing, i.e., adaptation of a given simulation to small geometric perturbations of the flow domain. Contributions. We (i) propose to shift the modelling objective and view CFD emulation as context-conditioned inpainting rather than supervised neural operator modelling, (ii) show that this approach has significantly better generalisation properties with boundary conditions and geometric variations and (iii) propose local editing, as a preliminary downstream task tailored to generative models. 2 Related work Neural surrogates for science and engineering have recently been gaining traction. Their underlying idea is to learn a mapping from the problem parameters to the quantities of interest in order to approximate high-fidelity numerical simulation in certain regimes where data is abundant. Proofs of concept exist, e.g., for weather forecasting 4 , automotive aerodynamics 2 , high-energy physics 29 and cardiovascular flow 30 . Neural operators. Popularised by architectures like DeepONet 19 and Fourier Neural Operator 15 , “neural operators" has become an umbrella term for models that map between continuous functions. In the context of neural surrogates, these models are typically set up as forward operators, mapping geometric and physical inputs to the solution field 20 ; 2 ; 23 ; 12 ; 33 ; 13 . Conceptually, these approaches are designed for regular grids or point clouds extracted from 3D surface or volume meshes. While grid-based methods lend themselves to Euclidean domains where isotropically high resolution is required, e.g. freestream flow within a volume element, point cloud methods can efficiently model irregular structures and their interaction with the medium, e.g., blood flow in human vasculature. Generative models. Due to their efficacy in domains like image processing and computer vision, generative models are gradually incorporated into neural surrogates. Diffusion models have been proposed for temporal forecasting of solutions to partial differential equations (PDE) where the main difficulty lies in achieving stable rollouts 31 ; 18 ; 28 ; 27 . Notably, 27 highlight the efficacy of encoding the data into a unified latent space in which system dynamics are generated. We adopt latent space modelling as a design choice in our approach. Instead of autoregressively evolving an initial state over time, 16 propose to learn the manifold of possible flow states in a certain (turbulent) regime constituted by the training data. In contrast to our approach, 16 impose Dirichlet boundary condition already during training and focus on directly modelling the distribution of turbulence rather than emulating a numerical solver. 17 propose to generate converged flow states via graph-based latent diffusion conditioned on parameters of the dynamics, such as Reynolds number. While this more closely emulates a numerical solver, 17 also impose Dirichlet boundary conditions during training which means their model is tied to these parameters, different from our context-driven approach. 8 propose an unconditional approach to temporal turbulence modelling using a latent space learned via conditional neural fields. In contrast to their approach, we include variations in mass flow and domain geometry in our framework while 8 focus on generating complex turbulence patterns within