Large-Scale Simulations of Turbulent Flows using Lattice Boltzmann Methods on Heterogeneous High Performance Computers
Adrian Kummerländer, Fedor Bukreev, Yuji Shimojima, Shota Ito and Mathias J. Krause · 2025
Read Large-Scale Simulations of Turbulent Flows using Lattice Boltzmann Methods on Heterogeneous High Performance Computers by Adrian Kummerländer, Fedor Bukreev, Yuji Shimojima, Shota Ito and Mathias J. Krause (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-21804.txt.
Introduction
Adrian Kummerländer and four co-authors simulate turbulent flow on modern supercomputers in a 2025 paper. Their lattice Boltzmann method suits parallel machines because its work splits naturally across many processors. They describe a new variant for wall-modeled simulation in complex shapes, built in an open framework. Scaling tests reached billions of cells across many nodes, showing such giant turbulence runs are practical on mixed processor machines.
Catalog record
| CREATOR | Adrian Kummerländer, Fedor Bukreev, Yuji Shimojima, Shota Ito and Mathias J. Krause |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-21804
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Large-Scale Simulations of Turbulent Flows using Lattice Boltzmann Methods on Heterogeneous High Performance Computers Adrian Kummerländer, Fedor Bukreev, Yuji Shimojima, Shota Ito and Mathias J. Krause Abstract Current GPU-accelerated supercomputers promise to enable large-scale simulations of turbulent flows. Lattice Boltzmann Methods (LBM) are particularly well-suited to fulfilling this promise due to their intrinsic compatibility with highly parallel execution on both SIMD CPUs and GPUs. A novel LBM scheme for wall-modeled LES in complex geometries is described with a special focus on the efficient implementation in the open source LBM framework OpenLB. Detailed scalability results are provided for all HoreKa partitions, utilizing up to 128 nodes and covering problem sizes up to 18 billion cells. 1 Introduction Multi-physics transport problems are predominant in the physical reality; capturing them in high-fidelity simulations enjoys ever-growing demand in both fundamental research and industrial applications. This present report summarizes our efforts on modeling and simulating such problems on the HoreKa supercomputer using the open source software framework OpenLB openlb2020 ; kummerlander_openlb_2025 . A particular focus is placed on the efficient implementation of large-scale simulations of turbulent fluid flows on GPU clusters. (a) Fully-coupled FSI simulation of a wind park (b) Wall-modelled FSI of a propeller aircraft Figure 1 : Volumetric rendering for large-scale multi-GPU LES in OpenLB Reynolds‐averaged Navier–Stokes (RANS) models and Large‐Eddy Simulation (LES) are two often compared approaches to turbulent flow modeling: although RANS approaches outperform LES in terms of computational cost, they suffer from significant accuracy shortcomings in aerodynamic calculations blocken_application_2011 . Despite the availability of modern GPU-accelerated clusters such as HoreKa, LES is often considered as too expensive to replace RANS in practice. Addressing the high computational cost of accurate LES for turbulent flows in large-scale, complex geometries is an ongoing research question. Modern approaches such as lattice Boltzmann methods (LBM) chen_lattice_1998 offer a promising alternative to conventional finite‐volume methods (FVM) for conducting LES. OpenLB provides a platform-independent and fully differentiable framework openlb2020 ; kummerlanderImplicitPropagationDirectly2023 ; Kummerlaender.deRSE23 ; kummerlaender25 for implementing and efficiently executing LBM-based simulations on heterogeneous high-performance computers. In a fair comparison with the FVM solver OpenFOAM, it has previously demonstrated a 32× faster time-to-solution—even without leveraging its multi-GPU capabilities HAUSSMANN20 . As covering all recent applications teutscher2025 ; Ito2025 ; Marquardt2025 ; Hettel2025 ; Zhong2024 ; SIMONIS23 ; Bukreev2025 ; Simonis2024 ; Marquardt2024a ; Marquardt2024b ; Ito2024 ; Prinz2024 ; Hafen2023 ; Bukreev2023 of such a multi-physics simulation tool within LBRG, an interdisciplinary group of mathematicians and engineers is beyond the scope of a single report, we focus instead in detail on recent work specifically on modeling large-scale turbulent flows in urban areas teutscher2025 . Studies PASQUIER23 ; ahmad_large-eddy_2017 ; jacob2021 ; inagaki_numerical_2017 ; onodera_real-time_2021 ; lenz_towards_2019 show that accurate LES are fundamentally feasible even in larger built‐up areas. LBMs have already been used for evaluating wind comfort in complex urban geometries ahmad_large-eddy_2017 , for simulating moist air convection or pressure loads on high‐rise buildings jacob2021 , for analyzing atmospheric boundary‐layer dynamics inagaki_numerical_2017 , or for investigating pollutant dispersion onodera_real-time_2021 . In particular, real‐time numerical simulations of urban flows with grid sizes on the order of one meter are possible onodera_real-time_2021 ; lenz_towards_2019 . Section 2 describes macroscopic equations used to model air flows of urban geometries, their solution by a novel LBM scheme and its efficient platform-transparent implementation in OpenLB. Detailed parallel performance results on all partitions of HoreKa as well as a summary of the experimental validation of the developed numerical wind channel setup is provided in Section 3 . 2 Methodology Modeling air flows and pollution transport in urban geometries is particularly challenging for computational fluid dynamics (CFD) due to the combination of large (landscape, buildings) and small length scales (building surfaces, trees). In our digital twin study teutscher2025 we modeled this complex problem using a filtered Brinkman–Navier–Stokes equations (FBNSE), capturing both LES for subgrid-scale turbulence modeling and homogenized porous media for tree modeling in a single target equation. This equation was solved efficiently on HoreKa Green using a homogenized lattice Boltzmann method (HLBM) discretization in OpenLB. 2.1 Filtered Brinkmann–Navier–Stokes Equations The macroscopic motion of fluids is commonly described using the Navier–Stokes equations (NSE). Incompressible flows in heterogeneous domains consisting of both fully fluid regions and porous media can be described using the filtered Brinkman–Navier–Stokes equations (FBNSE) \displaystyle\begin{cases}\bm{\nabla}\cdot\bar{\bm{u}}=0,&\quad\text{in }\Omega\times I,\\ \frac{\partial\bar{\bm{u}}}{\partial t}+\bar{\bm{u}}\cdot\bm{\nabla}\bar{\bm{u}}=-\frac{\bm{\nabla}\bar{p}}{\rho}+\nu_{\mathrm{mo}}\bm{\nabla}^{2}\bar{\bm{u}}+\frac{\nu_{\mathrm{mo}}}{K}\bar{\bm{u}}-\bm{\nabla}\cdot\mathbf{T}_{\mathrm{sgs}},&\quad\text{in }\Omega\times I,\end{cases} (1) for filtered pressure \bar{p} , velocity \bar{\bm{u}} density \rho on spatial domain \Omega\subseteq\mathbb{R}^{3} and time I\subseteq\mathbb{R}_{>0} . The molecular kinematic viscosity is defined as \nu_{\mathrm{mo}} and the permeability coefficient K>0 of the porous medium is given by the Forchheimer equation K=\frac{\mu_{F}Q}{A\left(\frac{\Delta P}{L}-\frac{\rho}{K_{\beta}}\frac{Q^{2}}{A^{2}}\right)}, (2) with dynamic viscosity \mu , volume flow rate Q , characteristic length L , projected area A , pressure difference \triangle P and nonlinear permeability coefficient K_{\beta} . The term \bm{\nabla}\cdot\mathbf{T}_{\mathrm{sgs}} models the subgrid-scale turbulence using the Smagorinsky LES approach \displaystyle\mathbf{T}_{\mathrm{sgs}} \displaystyle=2\nu_{\mathrm{turb}}\bar{\mathbf{S}}, (3) \displaystyle\nu_{\mathrm{turb}} \displaystyle=\left(C_{\mathrm{S}}\triangle_{\bm{x}}\right)^{2}\left|\bar{\mathbf{S}}\right|, (4) where C_{\mathrm{S}}>0 is the Smagorinsky constant, \triangle_{\bm{x}} is the filter width, and \bar{\mathbf{S}} is the filtered strain rate tensor: \bar{S}_{\alpha\beta}=\frac{1}{2}\left(\frac{\partial\bar{u}_{\alpha}}{\partial x_{\beta}}+\frac{\partial\bar{u}_{\beta}}{\partial x_{\alpha}}\right). (5) 2.2 Homogenized Lattice Boltzmann Method The HLBM is used to discretize the FBNSE ( 1 ) on a regular space-time grid with the D3Q19 velocity stencil (cf. Figure 2 ). Specifically, we utilize a homogenized hybrid regularized recursive Lattice Boltzmann method with Smagorinsky LES model (HHRRLBM-LES) that extends the classic HLBM KRAUSE20171 with a hybrid third-order recursive regularized collision model coreixas2017 ; JACOB18 . x y z Figure 2 : Schematic illustration of the discrete velocity set D3Q19 . Figure from simonis2023pde . The filtered and homogenized LB equation is given by f_{i}(\bm{x}+\xi_{i}\triangle t,t+\triangle t)=f_{i}^{\mathrm{eq}}(\bm{x},t)+\left(1-\frac{1}{\tau_{\mathrm{eff}}(\bm{x},t)}\right)\tilde{f}_{i}^{(1)}(\bm{x},t),\quad\text{in }\Omega_{\triangle x}\times I_{\triangle t}, (6) for distribution functions f_{i} along q discrete velocities \xi_{i} on a regular lattice \Omega_{\triangle x}\subset\Omega\subseteq\mathbb{R}^{3} with cell size \triangle x at discrete times I_{\triangle t}\subset I\subseteq\mathbb{R}_{\geq 0} separated by step size \triangle t . Here, the non-equilibrium distribution \tilde{f}_{i}^{(1)} is computed as a linear combination \displaystyle\tilde{f}_{i}^{(1)}(\bm{x},t)=\sigma f_{i}^{(1)}(\bm{x},t)-(1-\sigma)f_{i}^{(1,FD)}\text{ for }\sigma\in[0,1]. (7) That is, the distribution is hybridized between reconstructions using the rate of strain tensor obtained either from local macroscopic moments or from a non-local finite difference (FD) approximation. For the local part, the non-equilibrium distribution function f_{i}^{(1)} is expanded in terms of Hermite polynomials \mathbf{H}_{i}^{(n)} of the discrete velocity \xi_{i} as f_{i}^{(1)}(\bm{x},t)=\omega_{i}\sum_{n=0}^{N=3}\frac{1}{c_{\mathrm{s}}^{2n}n!}\mathbf{H}_{i}^{(n)}:\mathbf{a}_{1}^{(n)}(\bm{x},t), (8) where \omega_{i} are the lattice weights. The Hermite expansion coefficients are defined as \mathbf{a}_{1}^{(n)}(\bm{x},t)=\sum_{i=0}^{q-1}\mathbf{H}_{i}^{(n)}f_{i}^{(1)}(\bm{x},t). (9) For the non-local part, the FD non-equilibrium distribution function is defined as f_{i}^{(1,FD)}:=\frac{\rho\tau}{c_{\mathrm{s}}^{2}}\mathbf{H}_{i}^{(2)}:\mathbf{S}^{\mathrm{FD}}(\bm{x},t). (10) The equilibrium distribution function is defined as f_{i}^{\mathrm{eq}}(\bm{x},t)=\omega_{i}\left(\rho+\frac{\xi_{i}\cdot\rho\,\mathbf{\widehat{u}}}{c_{\mathrm{s}}^{2}}+\frac{\mathbf{H}_{i}^{(2)}:\widehat{\mathbf{a}}_{0}^{(2)}}{2c_{\mathrm{s}}^{4}}+\frac{\mathbf{H}_{i}^{(3)}:\widehat{\mathbf{a}}_{0}^{(3)}}{2c_{\mathrm{s}}^{6}}\right) (11) using Hermite coefficients \widehat{\mathbf{a}}_{0}^{(0)}=\rho(\bm{x},t) and \widehat{\mathbf{a}}_{0}^{(n)}=\mathbf{a}_{0}^{(n-1)}\widehat{\bm{u}}(\bm{x},t) . In the general case KRAUSE20171 , we define the homogenized velocity \widehat{\bm{u}} as a convex combination of the fluid velocity moment \bm{u} and the solid velocity \bm{u}^{\mathrm{B}} , given by \widehat{\bm{u}}(\bm{x},t)=(1-d(\bm{x},t))\bm{u}(\bm{x},t)+d(\bm{x},t)\bm{u}^{\mathrm{B}}(\bm{x},t), (12) where d is the so-called lattice porosity d(\bm{x},t)=1-\frac{\triangle x^{2}\nu\tau_{\mathrm{mo}}}{K(\bm{x},t)}. (13) and \tau_{\mathrm{mo}} is the molecular relaxation time. While the the velocity of the solid geometry is set to zero for our urban flow cases, non-zero solid velocities are dynamically prescribed by the wall model in Section 2.2 . Finally, the subgrid scale turbulence is accounted for by locally computing the effective relaxation time \tau_{\mathrm{eff}}(\bm{x},t) using the Smagorinsky BGK model \tau_{\mathrm{eff}}(\bm{x},t)=\frac{\nu_{\mathrm{eff}}(\bm{x},t)}{c_{\mathrm{s}}^{2}}\frac{\triangle t}{\triangle x^{2}}+\frac{1}{2}. (14) Connecting the HHRRLBM ( 6 ) to the FBNSE ( 1 ) target equation, we