GeoTransolver: Learning Physics on Irregular Domains using Multi-scale Geometry Aware Physics Attention Transformer
Corey Adams, Rishikesh Ranade, Ram Cherukuri, Sanjay Choudhry · 2025
Read GeoTransolver: Learning Physics on Irregular Domains using Multi-scale Geometry Aware Physics Attention Transformer by Corey Adams, Rishikesh Ranade, Ram Cherukuri, Sanjay Choudhry (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-20399.txt.
Introduction
Corey Adams, Rishikesh Ranade, Ram Cherukuri, and Sanjay Choudhry present GeoTransolver, a Transformer that learns physics on awkward, irregular engineering domains. Computer-aided design work needs fast stand-ins for runs such as airflow over cars. Their model extends the Transolver design with geometry-aware attention, branded GALE.
GALE pairs physics-aware self-attention on learned state slices with cross-attention to shared geometry and global context, gathered by multi-scale ball queries and reused in every block. Released inside NVIDIA's PhysicsNeMo, the model is benchmarked on automotive and wing data sets against strong benchmarks. It also handles crash-dynamics problems for bumper beams and full vehicle bodies. Accuracy, toughness to geometry shifts, and data thrift all improve over the comparison models.
Catalog record
| CREATOR | Corey Adams, Rishikesh Ranade, Ram Cherukuri, Sanjay Choudhry |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-20399
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GeoTransolver: Learning Physics on Irregular Domains using Multi-scale Geometry Aware Physics Attention Transformer Corey Adams, Rishikesh Ranade, Ram Cherukuri, Sanjay Choudhry Abstract We present GeoTransolver, a Multiscale Geometry-Aware Physics Attention Transformer for CAE that replaces standard attention with GALE, coupling physics-aware self-attention on learned state slices with cross-attention to a shared geometry/global/boundary-condition context computed from multi-scale ball queries (inspired by DoMINO) and reused in every block. Implemented and released in NVIDIA PhysicsNeMo, GeoTransolver persistently projects geometry, global and boundary condition parameters into physical state spaces to anchor latent computations to domain structure and operating regimes. We benchmark GeoTransolver on DrivAerML, Luminary SHIFT-SUV, and Luminary SHIFT-Wing, comparing against Domino, Transolver (as released in PhysicsNeMo), and literature-reported AB-UPT, and evaluate drag/lift R 2 R^{2} and Relative L 1 L_{1} errors for field variables. GeoTransolver delivers better accuracy, improved robustness to geometry/regime shifts, and favorable data efficiency; we include ablations on DrivAerML and qualitative results such as contour plots and design trends for the best GeoTransolver models. By unifying multiscale geometry-aware context with physics-based attention in a scalable transformer, GeoTransolver advances operator learning for high-fidelity surrogate modeling across complex, irregular domains and non-linear physical regimes. K eywords Transformers \cdot Multiscale Geometry Aware Models \cdot CAE Surrogate Modeling 1 Introduction Computer-aided engineering (CAE) increasingly depends on AI-based surrogates to accelerate design exploration to reduce reliance on costly high-fidelity simulation [ 53 , 14 , 6 , 44 ] . However, building reliable AI models for physics remains a challenge. Real-world geometries are heterogeneous and irregular with multiscale features; highly non-linear physical phenomena arising from local effects can produce long-range dependencies which span the entire computational domain; physical regimes and boundary conditions must be encoded and represented efficiently; and models must respect physical constraints (e.g., incompressibility with \nabla u=0 ) without sacrificing performance and scalability. Data is often scarce or biased by solver settings, domain shifts are common across shape families and physical regimes, and deep operator learners can accumulate error layer-by-layer, particularly in stiff regimes [ 4 ] . These factors have limited the stability, robustness, generalization, and trustworthiness of existing AI based approaches. This paper introduces GeoTransolver, a Multiscale Geometry-Aware Physics Attention Transformer for CAE applications. GeoTransolver extends the Transolver [ 50 ] backbone by replacing standard attention with GALE (Geometry-Aware Latent Embeddings) attention, which unifies physics-aware self-attention on learned state slices with cross-attention to geometry and global context embeddings. Inspired from Domino’s multi-scale ball query formulations [ 43 ] , GeoTransolver learns global geometry encodings and local latent encodings that capture neighborhoods at multiple radii, preserving fine-grained near-boundary behavior and far-field interactions. Crucially, geometry and global features are projected into physical state spaces and injected as context in every transformer block, ensuring persistent conditioning and alignment between evolving latent states and the underlying domain. GALE directly targets core challenges in AI physics modeling. By structuring self-attention around physics-aware slices, GeoTransolver encourages interactions that reflect operator couplings (e.g., pressure–velocity or field–material). Multi-scale ball queries enforce locality where needed while maintaining access to global signals, balancing efficiency with nonlocal reasoning. Continuous geometry-context projection at depth mitigates representation drift and improves stability, while providing a natural interface for constraint-aware training and regularization. Together, these design choices enhance accuracy, robustness to geometric and regime shifts, and scalability on large, irregular discretizations. GeoTransolver is implemented using the NVIDIA PhysicsNeMo framework [ 36 ] , which provides geometry data pipelines, multi-physics conditioning utilities, and scalable training/inference infrastructure. PhysicsNeMo enables efficient integration of multi-scale neighborhood construction, global context handling, and distributed training necessary for high-resolution CAE workloads. We benchmark GeoTransolver against state-of-the-art architectures - Domino [ 43 ] , AB-UPT [ 1 ] , and Transolver [ 50 ] on three widely used CAE datasets: DrivAerML [ 5 ] , Luminary SHIFT-SUV [ 11 ] , and Luminary SHIFT-Wing [ 12 ] . The benchmarking is carried out using the framework developed in PhysicsNeMo [ 48 ] . These tasks span aerodynamic flows over parameterized vehicle and wing geometries across diverse operating regimes. We evaluate field-level reconstruction (e.g., velocity, pressure and wall shear) and integral quantities (e.g., drag and lift coefficients), along with generalization to unseen geometries and design trends. Across these benchmarks, GeoTransolver demonstrates better accuracy, improved data efficiency, and strong robustness, underscoring the value of geometry-aware, globally conditioned attention. Our key contributions are: 1. GeoTransolver, a transformer architecture with GALE attention that pairs physics-aware self-attention on learned state slices with cross-attention to multi-scale geometric neighborhoods and global context. 2. A geometry-context projection strategy (inspired from Transolver [ 50 ] ) that maps geometry and global features into physical state spaces and injects them into every block, enabling persistent conditioning and reducing layer-wise drift. 3. An adaptation of multi-scale ball queries (inspired from Domino [ 43 ] ) that balances local fidelity and global coupling on irregular meshes and point clouds, improving efficiency and stability. 4. A comprehensive benchmark on DrivAerML, SHIFT-SUV, and SHIFT-Wing against Domino, AB-UPT, and Transolver, demonstrating gains in accuracy. 2 Background and Related Work AI-based surrogates for CAE span operator learning, mesh/geometry-aware representations, multi-scale architectures, and physics-informed training. Below we survey the most relevant lines of work and position GeoTransolver relative to them. 2.1 Neural operators Neural operators learn mappings between function spaces, enabling resolution-agnostic surrogates for PDEs [ 22 ] . DeepONet [ 32 ] pioneered branch–trunk architectures for continuous operators, while Fourier Neural Operator (FNO) [ 25 , 27 ] introduced spectral convolution to capture nonlocal interactions efficiently. Variants extend FNO with learnable bases, adaptive grids, or hybrid spectral–spatial blocks, and graph-based neural operators (GNO) (citations) target irregular discretizations via message passing on meshes [ 26 , 24 ] . Transformer-based operators have shown to improve long-range coupling and scalability due to the self- and cross- attention capability in many scientific disciplines [ 49 ] . Transformer neural operators [ 18 , 7 , 45 , 29 , 37 , 50 ] apply global attention over latent fields, sometimes with sparsification or hierarchical tokens to manage cost. Other efforts integrate attention with spectral operators or U-Net hierarchies to balance locality and global mixing. GeoTransolver builds on this trajectory but replaces standard attention with GALE, structuring self-attention around physics-aware state slices while using cross-attention to geometry and global context. This differs from purely field-centric attention by persistently anchoring computations to domain and regime signals at every block. 2.2 Geometry and mesh-aware encoders MeshGraphNet and related GNNs [ 39 , 38 , 40 , 16 ] popularized message passing on simulation meshes, capturing local interactions via edge neighborhoods and respecting mesh topology. Extensions incorporate learnable stencils, anisotropic kernels, and multigrid coarsening to handle large geometries [ 34 , 15 ] . Point-cloud networks [ 10 , 9 , 13 ] introduced multi-scale representations and local neighborhood aggregation for unstructured geometry, with hierarchical encoders that preserve fine detail while summarizing global shape. Domino [ 43 ] recently demonstrated that multi-scale ball queries paired with global shape descriptors can significantly improve generalization across shape families in CAE settings. AB-UPT [ 1 ] explores unified physics transformers with geometry-grounded tokens and attention that mixes local neighborhoods with global descriptors. GeoTransolver adopts multi-scale ball query formulations to learn both local latent encodings and global geometry encodings, but goes