Adaptive Mesh-Quantization for Neural PDE Solvers

Winfried van den dool [email protected], Maksim Zhdanov [email protected], Yuki M. Asano [email protected], Max Welling [email protected] · 2025

Read Adaptive Mesh-Quantization for Neural PDE Solvers by Winfried van den dool [email protected], Maksim Zhdanov [email protected], Yuki M. Asano [email protected], Max Welling [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-18474.txt.

Introduction

Winfried van den Dool and three co-authors make neural solvers for partial differential equations spend their effort where the physics is hard. Graph networks handle irregular meshes well, but they normally give every node the same bit width, wasting computation on simple regions.

Their adaptive mesh quantization lets a light auxiliary model spot high loss regions. It then assigns higher bit width there and lower elsewhere across node, edge, and cluster features. Integrated with two leading models, the framework improves the accuracy cost tradeoff on flow, elasticity, and fluid tasks, with gains up to about half at the same cost.

Catalog record

CREATORWinfried van den dool [email protected], Maksim Zhdanov [email protected], Yuki M. Asano [email protected], Max Welling [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-18474

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adaptive Mesh-Quantization for Neural PDE Solvers Winfried van den dool [email protected], Maksim Zhdanov [email protected], Yuki M. Asano [email protected], Max Welling [email protected] Abstract Physical systems commonly exhibit spatially varying complexity, presenting a significant challenge for neural PDE solvers. While Graph Neural Networks can handle the irregular meshes required for complex geometries and boundary conditions, they still apply uniform computational effort across all nodes regardless of the underlying physics complexity. This leads to inefficient resource allocation where computationally simple regions receive the same treatment as complex phenomena. We address this challenge by introducing Adaptive Mesh Quantization: spatially adaptive quantization across mesh node, edge and cluster features, dynamically adjusting the bit-width used by a quantized model. We propose an adaptive bit-width allocation strategy driven by a lightweight auxiliary model that identifies high-loss regions in the input mesh. This enables dynamic resource distribution in the main model, where regions of higher difficulty are allocated increased bit-width, optimizing computational resource utilization. We demonstrate our framework’s effectiveness by integrating it with two state-of-the-art models, MP-PDE and GraphViT, to evaluate performance across multiple tasks: 2D Darcy flow, large-scale unsteady fluid dynamics in 2D, steady-state Navier–Stokes simulations in 3D, and a 2D hyper-elasticity problem. Our framework demonstrates consistent Pareto improvements over uniformly quantized baselines, yielding up to 50% improvements in performance at the same cost. 1 Introduction In recent years, there has been a surge in the application of machine learning algorithms to build neural surrogates for Partial Differential Equations (PDEs). Using neural networks, PDEs can be modeled as a next-frame prediction problem, where the dynamics of the system (such as fluid movements) are implicitly learned. In real-world applications, traditional numerical methods such as the finite element method predominantly operate on meshes or irregular grids ( Mavriplis, 1997 ) , as these representations naturally adapt to complex geometries. Graph Neural Networks (GNNs) ( Brandstetter et al., 2022 ; Pfaff et al., 2020 ; Gilmer et al., 2017a ) have emerged as a natural solution, since they can operate directly on unstructured meshes, thereby avoiding the mesh-to-grid interpolation otherwise needed to represent irregular domains on regular grids. However, since computation scales directly with input graph size, efficient inference of GNNs remains challenging ( Zhang et al., 2022 ) . Furthermore, reducing the operational cost is crucial for the practical deployment of GNN-based neural surrogates, as lower costs enable higher data resolution - a key factor in improving modeling accuracy ( Iles et al., 2020 ; Bauer et al., 2015 ; Randall et al., 2007 ) . Figure 1: Overview of the proposed framework. Given a point cloud or graph, a bit assigner returns a node-wise quantization scheme for a larger model. The goal is to assign higher precision to more difficult and complex regions. A common way to lower the computational budget required for inference in deep learning is (integer) quantization ( Jacob et al., 2017 ) . Converting model weights and activations from floating-point to integer representations can drastically reduce computing and memory requirements. Although quantization was initially associated with resource-limited devices, it is now gaining attention in broader applications including Large Language Models ( Wang et al., 2023 ) and large-scale physical simulations ( Lang et al., 2021 ) . In the latter, reduced-precision arithmetic has shown particular promise in mathematical (climate) modeling ( Kimpson et al., 2023 ) and neural surrogates ( Dool et al., 2023 ) , enabling higher data resolutions within fixed computational budgets. While quantization typically applies uniform precision reduction across the entire model, many physical systems exhibit complexity that varies in both location and time. Phenomena such as turbulence, shock waves, and steep gradients create regions where accurate prediction requires substantially higher numerical precision than in other, smoother, more predictable areas. ( Strogatz, 2000 ) Current neural PDE solvers treat all mesh nodes uniformly, allocating the same computational resources regardless of the underlying modeling difficulty at each location. This uniform approach leads to inefficient resource utilization, where computationally straightforward regions consume the same resources as challenging phenomena. We address this inefficiency by introducing Adaptive Mesh Quantization (AMQ). In mesh quantization, we apply quantization to all the mesh-based features - including node features, edge features, and possibly cluster representations - processed by graph neural networks operating on meshes. Our adaptive approach dynamically varies this quantization - allocating higher bit-width to regions of higher complexity. More concretely, relatively easy nodes - as to be selected by a small auxiliary model - will be processed using low bit-width, freeing up resources for more complex regions of the input requiring high precision (Figure 1 ). The light-weight auxiliary model is trained to predict the loss of the main model at each node, which serves as a proxy for local complexity. This proxy is thus based on both aleatoric and epistemic uncertainty, signaling not only where prediction is hard in general but also where prediction is hard for the quantized main model specifically. Figure 2: Overview of the training phase ( A ). A lightweight auxiliary model assigns a complexity weight to every mesh node to guide the resource allocation of the large main message-passing model. Gradients do not flow through the dotted black arrows, i.e., backpropagation happens through the solid black arrows. ( B ). The bit assignment process is schematically shown in ( C ). Overall, we make the following contributions: • We introduce Adaptive Mesh Quantization, a framework that learns to allocate resources locally on a mesh depending on the input data. We additionally propose an efficient implementation for the mixed-precision quantization of MLPs. • We investigate the particular setup where an auxiliary model learns which input regions are more complex based on the main model’s loss landscape, training both models simultaneously. • We validate the framework on multiple PDE datasets, ranging in complexity and scale, and demonstrate consistent compute-performance Pareto improvements. 2 Related work 2.1 Neural PDE solvers The intersection of PDE solving and deep learning has emerged as an active research area ( McGreivy and Hakim, 2024 ) . The usual task is to learn a mapping from an initial state to a solution of a PDE with a neural network, in order to accelerate complex simulations in fields like fluid dynamics and climate modeling ( Wang et al., 2024 ) . Originally focusing on regular grids ( Li et al., 2020 ) , multiple solutions were suggested to deal with irregular meshes, with message-passing-based approaches ( Brandstetter et al., 2022 ) most relevant to our work. For these networks the input comes in the form of a graph and a model learns evolution for every node. Lately, mesh attention ( Janny et al., 2023 ) was also introduced to capture long-range dependencies further enhancing the performance of mesh-based models. We do not introduce a new pde-solving model, but rather a new method that is applicable to families of Graph Neural Networks in particular. To showcase our method we use both the original MPNN ( Brandstetter et al., 2022 ) , as well as the more elaborate Graph Attention architecture ( Janny et al., 2023 ) . Apart from being state-of-the-art on several datasets, the latter model has the advantage of covering many different layer types (e.g., graph pooling, rnn, attention), ensuring

Read the complete text