Compact SO(3) Equivariant Atomistic Foundation Models via Structural Pruning
Chen Wang, Siyu Hu, Guangming Tan, Weile Jia · 2026
Read Compact SO(3) Equivariant Atomistic Foundation Models via Structural Pruning by Chen Wang, Siyu Hu, Guangming Tan, Weile Jia (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08885.txt.
Introduction
Chen Wang, Siyu Hu, Guangming Tan, and Weile Jia shrink atomistic foundation models that respect three dimensional rotation symmetry, without breaking the symmetry that makes them accurate. These equivariant networks pay heavily for higher order tensor operations. Their structural pruning cuts whole blocks along channel and order dimensions, keeping or removing each symmetry unit entire so rotational equivariance survives intact.
Starting from a large trained checkpoint beats training a small model from scratch: the pruned flagship wins on most leaderboard metrics, carries far fewer parameters, and needs far less pretraining compute. Fine tuned, it slashes energy and force errors versus task specific training, and the method transfers to other equivariant architectures and stacks with quantization.
Catalog record
| CREATOR | Chen Wang, Siyu Hu, Guangming Tan, Weile Jia |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08885
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Compact SO(3) Equivariant Atomistic Foundation Models via Structural Pruning Chen Wang, Siyu Hu, Guangming Tan, Weile Jia Abstract SO(3) equivariant graph neural networks have become the dominant paradigm for atomistic foundation models, achieving high accuracy and data efficiency by building rotational symmetry directly into the architecture. Yet the computational cost of their higher-order tensor operations creates a tough trade-off between model accuracy and inference efficiency. In this paper, we propose a structural pruning method for SO(3) equivariant atomistic foundation models to bridge this accuracy-efficiency gap. The pruning is applied along the channel and order dimensions, with each irreducible representation kept or removed as a complete block, thereby retaining SO(3) equivariance. Starting from a large checkpoint, the pruned model substantially reduces the inference cost while retaining higher accuracy than an independently trained small model. The pruned MACE-MP model outperforms the official from-scratch trained small model on 7 of 9 metrics on the Matbench Discovery leaderboard. In terms of efficiency, compressed MACE-MP and MACE-OFF models contain 1.5 × \times to 4 × \times fewer parameters and require 2.5 × \times to 4 × \times less pre-training compute than training a small model from scratch. For downstream applications, fine-tuning the pruned model reduces energy and force errors by 70.1% and 34.4% compared to training task-specific models from scratch across eight representative downstream datasets. We demonstrate that the method generalizes to other SO(3) equivariant architectures (SevenNet, eSCN) and can be combined with quantization and knowledge distillation for further gains. 1 Introduction Figure 1: Pruning results . Pruned models outperform scratch-trained models of the same size at 2.5 \times –4 \times lower training cost. Point size indicates parameter count. Atomistic foundation models have recently emerged as a transformative paradigm in computational chemistry and materials science [ 28 ] . These models [ 5 , 42 , 60 , 46 , 37 ] are available at different scales, placing each at a different point on the accuracy-efficiency frontier. Larger foundation models often achieve stronger benchmark accuracy than smaller ones, but their training cost, inference latency [ 33 ] , and memory footprint become major bottlenecks for practical deployment. This persistent tension between the accuracy of large checkpoints and the efficiency of smaller ones remains a central challenge for atomistic foundation models. Among atomistic foundation models, SO(3) equivariant architectures have shown particularly strong accuracy on widely used benchmarks by explicitly encoding rotational symmetry [ 53 , 19 , 43 ] . This accuracy gain, however, comes with substantial computational overhead: equivariance is commonly implemented through tensor product operations over higher-order geometric features, whose cost scales up to \mathcal{O}(L_{\max}^{6}) , where L_{\max} is the maximum order. This makes the accuracy-efficiency trade-off especially tough for SO(3) equivariant atomistic foundation models. Model compression is therefore an important route to enhancing model efficiency, with three common paradigms: structural pruning, knowledge distillation (KD) [ 23 ] , and quantization [ 21 ] . As summarized in Table 1 , these methods target different aspects of the efficiency problem. Structural pruning removes redundant architectural units from a pre-trained checkpoint, producing a smaller model in the same family with reduced parameters and FLOPs. KD trains a smaller student model to mimic a larger teacher, which can reduce inference cost but typically depends on the chosen student architecture and does not directly reuse the teacher weights. Quantization keeps the architecture unchanged and reduces numerical precision, improving memory and hardware efficiency without reducing the number of parameters or tensor-product paths. Table 1: Comparison of model compression paradigms for atomistic foundation models. Our structural pruning is distinct from and can be combined with Knowledge Distillation and Quantization. Criterion Structural Pruning (Ours) Knowledge Distillation Quantization Weight source Sliced from pre-trained Random init (student) Pre-trained (bit-reduced) Architecture scope Same family Often cross-arch Same arch Reduces FLOPs & params ✓ ✓(smaller student) \times Reduces bit-precision \times \times ✓ SO(3) equivariance Exactly preserved Depends on student Approx. preserved Among these compression strategies, we focus on structural pruning because it can directly inherit weights from a large checkpoint and produce a smaller pre-trained model, offering a natural way to bridge this accuracy-efficiency gap. However, pruning strategies developed for large language models [ 24 , 61 , 2 , 38 ] cannot be directly transferred to SO(3) equivariant models: unlike scalar-featured networks, these models operate on higher-order geometric tensors, so naively removing feature dimensions breaks equivariance. To the best of our knowledge, existing pruning methods for atomistic foundation models remain coarse-grained, typically removing interaction blocks whose marginal accuracy contribution is limited [ 22 , 30 ] . While such block-level pruning preserves SO(3) equivariance by removing entire equivariant modules, it effectively reduces the receptive field of the model and can be especially harmful for shallow model architectures [ 6 ] . This raises a central challenge: how can we identify removable structures and compress the weights of a large pre-trained SO(3) equivariant model while preserving equivariance and retaining accuracy? To address this challenge, we propose a structural pruning framework for SO(3) equivariant atomistic foundation models. Our key contributions are as follows: • We introduce a structural pruning method for higher-order SO(3) equivariant tensors across both channel and order dimensions while preserving SO(3) equivariance. • We propose a four-stage framework to compress foundation models for downstream fine-tuning: (1) calibration, (2) pruning, (3) retraining, and (4) fine-tuning. • We validate our approach on MACE-MP and MACE-OFF, achieving 1.5 \times to 4 \times parameter reduction and 2.5 \times to 4 \times training cost reduction (Figure 1 ). The pruned models outperform official models trained from scratch on accuracy and achieve 2.7 \times inference speedup. When fine-tuned on downstream tasks, they reduce energy and force errors by 70.1% and 34.4% compared to training from scratch. • We demonstrate the generalizability of our method across architectures (SevenNet, eSCN) and show that structural pruning can be combined with quantization and knowledge distillation for further efficiency and accuracy gains. 2 Related Work Atomistic Foundation Models. Atomistic foundation models are MLIPs trained on massive chemical datasets to predict interatomic potentials across the periodic table. Unlike earlier MLIPs [ 54 , 50 , 17 , 7 ] tailored for specific chemical systems, these models [ 5 , 42 , 60 , 46 , 37 , 52 , 35 , 64 , 63 , 44 , 67 ] are pre-trained on vast, diverse quantum mechanical datasets [ 11 , 13 , 3 , 34 , 20 , 51 ] to learn transferable representations of interatomic interactions. By effectively bridging the accuracy of ab initio methods with the efficiency of classical force fields [ 49 ] , these foundation models enable accurate simulations at unprecedented scales [ 39 ] . In practical applications such as drug discovery and materials design, foundation models can be fine-tuned on domain-specific datasets [ 45 , 29 ] . However, their massive parameter counts and computational costs create a deployment bottleneck, motivating our work to develop efficient compression strategies that make these foundation models practical for downstream applications. Model Compression. Model compression [ 12 ] encompasses a suite of techniques, including pruning [ 15 ] , knowledge distillation (KD) [ 23 , 14 , 1 ] and quantization [ 21 , 8 ] , designed to reduce the memory