From Evaluation to Design: Using Potential Energy Surface Smoothness Metrics to Guide Machine Learning Interatomic Potential Architectures Ryan Liu 1,2, Eric Qu 2, Tobias Kreiman 2, Samuel M. Blau 3, Aditi S. Krishnapriyan 2,3 Abstract Machine Learning Interatomic Potentials (MLIPs) sometimes fail to reproduce the physical smoothness of the quantum potential energy surface (PES), leading to erroneous behavior in downstream simulations that standard energy and force regression evaluations can miss. Existing evaluations, such as microcanonical molecular dynamics (MD), are computationally expensive and primarily probe near-equilibrium states. To improve evaluation metrics for MLIPs, we introduce the Bond Smoothness Characterization Test (BSCT). This efficient benchmark probes the PES via controlled bond deformations and detects non-smoothness, including discontinuities, artificial minima, and spurious forces, both near and far from equilibrium. We show that BSCT correlates strongly with MD stability while requiring a fraction of the cost of MD. To demonstrate how BSCT can guide iterative model design, we utilize an unconstrained Transformer backbone as a testbed, illustrating how refinements such as a new differentiable k k -nearest neighbors algorithm and temperature-controlled attention reduce artifacts identified by our metric. By optimizing model design systematically based on BSCT, the resulting MLIP simultaneously achieves a low conventional E/F regression error, stable MD simulations, and robust atomistic property predictions. Our results establish BSCT as both a validation metric and as an “in-the-loop” model design proxy that alerts MLIP developers to physical challenges that cannot be efficiently evaluated by current MLIP benchmarks. 1 Introduction Interatomic potentials are fundamental to computational chemistry and materials science, enabling essential tasks such as molecular dynamics (MD) and geometry optimization. Quantum mechanical calculations, such as Density Functional Theory (DFT) [ 25 ] are widely used to derive these potentials, supporting applications from drug discovery to catalyst design [ 12 , 23 , 21 ] . However, DFT’s O(n^{3}) computational scaling makes it prohibitively expensive for large systems or long timescale simulations. To overcome this bottleneck, Machine Learning Interatomic Potentials (MLIPs) have emerged as powerful surrogates, offering orders-of-magnitude speedups while approaching DFT accuracy. Typically, MLIPs are trained to minimize regression error on the energies and forces of atomistic structures, with a goal of reproducing the underlying potential energy surface (PES). However, even given a low test-set regression error, the curvature of the PES may not necessarily be correct. During dynamic simulations, this can manifest as unstable trajectories or nonphysical behavior [ 16 , 4 , 37 , 46 ] . The true quantum mechanical PES has an inherent degree of smoothness that MLIPs are not necessarily guaranteed to capture. While the MLIP community often uses “smoothness” informally to mean bounded derivatives [ 15 ] , rigorous chemical smoothness requires the absence of artificial extrema or inflection points [ 53 ] . This property is especially crucial in far-from-equilibrium regimes, where data are more limited while spurious PES features can still cause simulation instability. However, current evaluations rely either on computationally expensive MD simulations or test sets limited to near-equilibrium states, which may not efficiently capture these instabilities [ 42 , 15 ] . To bridge this gap, we introduce the Bond Smoothness Characterization Test (BSCT), a benchmark that efficiently evaluates PES smoothness in both near- and far-from-equilibrium regimes. BSCT probes 1D bond deformations where the true PES is known to be smooth, such that artifacts created by MLIP can be easily isolated and detected. From this, we define the Force Smoothness Deviation (FSD), a low-cost metric defined on BSCT that correlates strongly with MD stability, providing an early indicator of physical reliability. We demonstrate BSCT’s utility both as a validation metric and as a guide for architectural design. Using an expressive, unconstrained attention-based neural network as a test bed, we show how BSCT acts as an “in-the-loop” diagnostic to identify and reduce unphysical artifacts. Through targeted modifications, including a new differentiable k -nearest neighbors (Diff-kNN) algorithm, controllable Gaussian smearing, and temperature-controlled attention, we systematically improve the model’s physical soundness. This case study establishes BSCT as a practical framework for evaluating and developing MLIPs, providing a critical signal of reliability that is not fully captured by standard MLIP benchmarks. 2 Related Works The development of reliable MLIPs has driven a dual focus in the community: the creation of rigorous benchmarks to validate physical consistency and the design of expressive architectures that can capture complex atomic interactions. MLIP Benchmarks. Many MLIP benchmarks have been developed to supplement energy and force regression errors, and we list some examples of these here. The TorsionNet-500 dataset includes 500 molecules’ torsion scan profiles, allowing MLIPs to compare their PES to DFT calculations [ 45 ] . 16 proposed to use MD simulation stability and h(r) reconstruction to benchmark models. 4 used NVE simulations and Jacobians to measure the non-conservative behavior of the models. 28 explicitly evaluated the generalization capabilities of foundational MLIPs. The NNP Arena provides an assortment of molecular and lattice property prediction benchmarks as well as inference speed test [ 50 ] . The Open Molecules 2025 (OMol25) dataset [ 30 ] and leaderboard [ 54 ] include molecular physics-based evaluations. Our benchmark evaluation, BSCT, differs from these by explicitly measuring PES smoothness far from equilibrium, and could be added to complement OMol25 evaluations. For materials, the Open Catalyst project assesses models by their ability to predict the correct relaxed energy of the catalyst-adsorbate system [ 6 ] . Matbench Discovery ranks models by their ability to predict structure stability [ 49 ] . The MDR phonon benchmark tests MLIP’s capability to predict the correct phonon structure of the lattice [ 42 ] . MLIP arena provides a wide array of benchmarks for materials ranging from homonuclear diatomics to equations of state [ 9 ] . The JARVIS-Leaderboard has multiple benchmarks related to materials design [ 11 ] . MLIPs for Atomistic Systems. There is a long history of ML interatomic potentials, including those based on neural networks [ 3 ] . Many of these neural networks have invariant features, i.e., featurizing embeddings to be invariant to group operations such as rotations and translations [ 51 , 62 , 19 , 60 , 8 , 52 , 7 , 41 , 57 , 61 , 17 ] . There are also several graph neural networks that build in symmetries such as rotational equivariance through featurizing via spherical harmonics [ 2 , 38 , 33 , 15 , 24 , 1 , 40 , 5 , 56 ] . There has also been increasing interest in models that do not explicitly build in all symmetries into the architecture, and instead focus on approaches to maximize learning these symmetries from the data [ 43 , 44 , 39 , 48 , 36 ] . 27 explored this direction further, demonstrating that Transformers can discover molecular structure without explicit graph priors. 3 Bond Smoothness Characterization Test Figure 1 : Example of how the Bond Smoothness and Characterization Test (BSCT) is constructed. We show a C-C bond in a C_{2}H_{2}F_{4} molecule from the BSCT-SPICE dataset. The DFT reference PES smoothly varies across the wide range of bond lengths, showing the behavior expected from reliable interatomic potentials. These 1D probes are a simple and efficient way to measure PES smoothness in a regime of highly varying energies, which is inherently out-of-distribution of many MLIP training datasets. Evaluating an MLIP for physical modeling requires more than measuring its energy and force prediction errors. Reliable simulations with an MLIP depend on accurately reproducing the inherent smoothness of the true quantum mechanical PES. In the quantum chemistry community, the chemical smoothness of a PES is defined as the absence of spurious discontinuities, extrema, and inflection points [ 53 ] . This definition differs from the MLIP community’s more recent use of “smoothness” to mean bounded PES derivatives [ 15 ] , which is typically evaluated by costly microcanonical MD simulations. Near vs. Far-From-Equilibrium Smoothness. A critical distinction exists between smoothness in the interpolation (near-equilibrium) and extrapolation (far-from-equilibrium) regimes. Near equilibrium, there is now enough training data that sufficiently expressive models can often improve smoothness by using this data [ 59 ] . For example, models on the Matbench Discovery leaderboard can achieve better near-equilibrium smoothness ( \kappa_{\mathrm{SRME}} ) by training the same architecture on larger datasets [ 49 ] . As larger datasets such as OMol25 and OPoly26 [ 30 , 31 ] become available, we believe that near-equilibrium smoothness will no longer be a constraining factor. However, while OMol25 and OPoly26 do include substantial reactive sampling, far-from-equilibrium regions of the PES remain less thoroughly sampled, making such configurations more often out-of-distribution (OOD). In this regime, physical reliability may also depend on the model’s architectural priors. As shown in 58 , MLIPs that are not explicitly trained on transition states data show degraded smoothness under bond breakage. BSCT targets this challenging OOD regime to expose instabilities that standard validations currently miss. What BSCT Measures. The Bond Smoothness Characterization Test (BSCT) evaluates how smoothly an MLIP predicts energies and forces as molecular bonds are systematically stretched and compressed beyond equilibrium. We focus on bonds because their ground-truth PES (i.e., dissociation curves) is known to be smooth, making deviations and erroneous non-smoothness easier to detect. By sampling a one-dimensional slice of the PES along the bond length and comparing MLIP predictions with density functional theory (DFT) references, BSCT isolates non-smooth behavior in regions with lower coverage in the training distribution. BSCT targets the challenging regime of far-from-equilibrium smoothness, enabling us to determine specific MLIP design choices that genuinely improve PES smoothness. This allows us to isolate the effect of architecture from raw model capacity, providing insight into how design choices influence the reliability and generalization of MLIPs. Constructing the BSCT Dataset. For each molecule, we select a bond that splits the molecule into two fragments and displace the fragments along the bond axis while keeping their internal geometries fixed. Formally, given atomic positions \{x_{i}\}\in\mathbb{R}^{N\times 3} , the fragment labels \{h_{i}\}\in\{-1,1\}^{N} , and the bond direction unit vector \hat{r}\in S^{2} , the perturbed positions for displacement \alpha are: x_{i}^{\prime}(\alpha)=x_{i}+\alpha h_{i}\hat{r}. (1) We construct the BSCT-SPICE dataset by applying this procedure to the SPICE test structures [ 14 , 26 ] . The dataset contains 485 molecules, each with 100 DFT single-point calculations computed at the same level of theory as SPICE ( \omega B97M-D3(BJ)/def2-TZVPPD) using Psi4 [ 35 , 47 , 22 , 55 ] , the same computational chemistry code as SPICE. We construct the dataset by systematically scanning bridge bonds in molecules, filtering out structures with isolated or overlapped atoms, and running DFT calculations on bond scans (See Appendix A for details). Figure 1 shows an example C–C bond scan, where the DFT PES varies smoothly across the sampled bond lengths; BSCT evaluates whether MLIPs preserve this physically correct behavior. \captionlistentry \captionlistentry \captionlistentry \captionlistentry Figure 2 : (a): Motivation for our proposed potential energy surface (PES) smoothness metric. We compare two hypothetical PES. Both PES1 and PES2 accurately reproduce the true quadratic PES near equilibrium. Away from equilibrium, PES1 slowly deviates from the quadratic PES reference but remains smooth, and PES2 has an artificial minimum (non-smoothness) enclosed by gold dashed lines. (b) Standard metrics, such as energy and forces mean absolute errors (MAEs), evaluated on the one-dimensional subset fail to detect PES2’s non-smoothness. (c) Our proposed force smoothness deviation (FSD) metric sensitively captures this non-smooth behavior. Quantifying Smoothness: The FSD metric. We introduce a new metric, the Force Smoothness Deviation (FSD), to quantitatively measure the PES smoothness. We define the “force norm due to perturbation” as: \|\Delta\vec{F}\|^{2}=\|\vec{F}-\vec{F}_{\mathrm{min}E}\|^{2}, (2) where \vec{F}_{\mathrm{min}E} is the force vector at the minimum-energy structure in the 1-D PES section. As shown in Figure 2 , the derivative of \|\Delta\vec{F}\|^{2} with respect to the perturbed coordinate \alpha sensitively detects non-smoothness (artificial minimum). Therefore, we define the force smoothness deviation (FSD) as: FSD \displaystyle=\max_{\alpha}\left|\frac{\frac{\mathrm{d}}{\mathrm{d}\alpha}\|\Delta\vec{F}_{\text{MLIP}}\|^{2}}{\|\Delta\vec{F}_{\text{MLIP}}\|^{2}}-\frac{\frac{\mathrm{d}}{\mathrm{d}\alpha}\|\Delta\vec{F}_{\text{DFT}}\|^{2}}{\|\Delta\vec{F}_{\text{DFT}}\|^{2}}\right| (3) \displaystyle=\max_{\alpha}\left|\frac{\mathrm{d}}{\mathrm{d}\alpha}\log\frac{\|\Delta\vec{F}_{\text{MLIP}}\|^{2}}{\|\Delta\vec{F}_{\text{DFT}}\|^{2}}\right|, (4) where \alpha is defined by Equation 1 , and the derivative is taken with respect to this one-dimensional perturbation parameter. We utilize the logarithmic derivative here because it captures the relative rate of change in the force norm, ensuring that smoothness artifacts are penalized equally in both high-force and low-force regions. A lower FSD indicates smoother and more physically sound PES predictions. The particular functional form makes FSD an indicator of chemical smoothness since the denominator \|\Delta\vec{F}_{\text{MLIP}}\|^{2} is small when an extremum is around, and the numerator \frac{\mathrm{d}}{\mathrm{d}\alpha}\|\Delta\vec{F}_{\text{MLIP}}\|^{2} is small when an inflection point is around. By comparing the ratio of this to the DFT reference, FSD can detect any artificial extrema or inflection points and measure the chemical smoothness of a PES. Validating BSCT through “In-the-Loop” Model Design. While we have motivated BSCT and FSD based on chemical principles, an important value lies in their ability to serve as actionable signals during model development. To demonstrate that BSCT is a viable proxy for expensive MD simulations, we must demonstrate that reducing FSD directly translates to improved physical stability. This requires a controlled experimental setup, and we start with an expressive, unconstrained baseline architecture that allows us to isolate specific sources of non-smoothness. In the following section, we introduce this backbone—not as an explicitly proposed state-of-the-art model, but as a neutral testbed designed to rigorously evaluate the diagnostic utility of BSCT in guiding iterative architectural design. 4 MinDScAIP: A Testbed for Smoothness Evaluations with BSCT To rigorously evaluate BSCT’s sensitivity, we require an expressive backbone architecture with minimal geometric constraints. Starting from a neutral, unconstrained baseline allows us to isolate specific sources of non-smoothness and ensures that any improvements captured by BSCT are directly attributable to our targeted design choices. We design the Min imally constrained D ifferentiable Sc aled A ttention I nteratomic P otential, MinDScAIP (MinDScAIP) to serve as this transparent testbed, enabling us to validate BSCT as a tool for iterative model development while keeping confounding architectural priors to a minimum. (a) (b) Figure 3 : To study how different model design choices impact or improve PES smoothness, we design a neural network backbone similar to Swin-Transformer, intending to create a neutral testbed for BSCT-guided architecture ablations. We generalize the shifted window attention of Swin Transformer to graphs by alternating in-and-out neighborhood attention. (a) The structural similarity to a generic Transformer is intentional, aiming to provide a neutral testbed for BSCT. (b) The interleaving windows allow information to propagate across the molecular graph. 4.1 Backbone Architecture Overview The MinDScAIP backbone, as shown in Figure 3 , is structured to be similar to a Transformer backbone, and differs from conventional design mainly in the graph construction method and the attention mechanism employed. These designs are proposed to balance expressiveness and computational scalability, while keeping the architecture unconstrained enough to serve as a neutral testbed for testing BSCT-guided modifications. Graph Construction Methods. Most attention-based MLIPs impose sparsity using radius graphs. However, this often results in inefficient representations due to the highly variable number of neighbors per atom (see Appendix B for detailed analysis). For a more regular and efficient representation, the MinDScAIP backbone adopts a k -nearest-neighbor (kNN) graph structure. We arrange kNN edges into an N_{atom}\times k array such that (i,j) -th element is the j -th shortest edge in node i ’s neighborhood [ 44 ] . Essentially, rather than padding all neighborhoods to the same number of neighbors, we truncate the neighbors to obtain a regular representation. To make the truncation differentiable, we further propose a Differentiable k-NN Algorithm, which will be detailed in Section 4.2 . Attention Mechanism. MinDScAIP employs a dual attention strategy inspired by Swin-Transformer’s shifted window attention [ 34 ] , as depicted in Figure 3(b) . Swin-Transformer partitions an image into windows (disjoint local sets). Similarly, on a directed graph, neighborhoods can partition edges into disjoint local sets. Similar to Swin-Transformer, where windows are shifted to create overlaps to propagate information, we can switch between in-neighborhood and out-neighborhood to achieve the same goal. As illustrated in Figure 3(b) , MinDScAIP interleaves attention between edges sharing the same source atom (out-neighborhood) and edges sharing the same destination atom (in-neighborhood), generalizing Swin-Transformer to directed graphs. 4.2 Differentiable kNN Algorithm A fundamental requirement for physically consistent Molecular Dynamics (MD) simulations is the use of a conservative force field, where atomic forces are defined as the negative gradient of the energy. While kNN graph construction offers computational advantages, standard kNN algorithms are inherently non-differentiable due to the truncation. To address this issue, we introduce the differentiable kNN algorithm (Diff-kNN), which ensures the potential remains differentiable, allowing for the prediction of conservative forces while maintaining the efficiency of a sparse graph. Standard kNN Algorithm. The standard kNN algorithm has two steps: first, a ranking is calculated for each edge (i,j) with length d_{ij} with respect to other edges in its neighborhood \mathcal{N}(i)=\{(i,j^{\prime})\}_{j^{\prime}} . This ranking is hard , as rankings can change discontinuously when edge lengths vary. \textstyle\mathrm{rank}((i,j)|\mathcal{N}(i))=\sum_{j^{\prime}}\mathbb{I}(d_{ij}>d_{ij^{\prime}}). (5) Second, it selects the k shortest edges: G=\left\{(i,j):\mathrm{rank}((i,j)|\mathcal{N}(i))1\\ \end{array}\right. (11) where g(x)\approx x when |x|\ll 1 and g(x)\equiv\mathbb{I}(x) when |x|>1 . Since g(x) is strictly zero when x<-1 , we can sort edge lengths and truncate at some rank k+\Delta before taking the summation, since the edges with large displacement will contribute 0 to the sum when calculating the envelope factor for the edges with small displacement that will be included in the graph. This can reduce the memory complexity of the gradient graph from O(N_{\text{atoms}}^{3}) to O(N_{\text{atoms}}(k+\Delta)^{2}+N_{\text{atoms}}^{2}) . Appendix D Gaussian Smearing In this section, we look at Gaussian smearing, as described in Section • ‣ 5.1 . We show that increasing \sigma upper bounds the derivatives of any linear combination of these basis functions relative to their infinity norm. We consider the derivative of an arbitrary linear combination of basis functions v_{i}(x)=e^{-(x-i)^{2}/2\sigma^{2}} specified by coefficients \{a_{i}\} : f(x)=\sum_{i}a_{i}v_{i}(x) , where the basis function has width \sigma . Now we are interested in the maximum derivative at some location y normalized by the infinity norm of such a linear combination. Without loss of generality, we can assume y=0 . \displaystyle\max_{\{a_{i}\}}\frac{\left.\frac{\partial}{\partial y}\sum_{i=-\infty}^{\infty}a_{i}e^{-(y-i)^{2}/2\sigma^{2}}\right|_{y=0}}{\left\|\sum_{i=-\infty}^{\infty}a_{i}e^{-(x-i)^{2}/2\sigma^{2}}\right\|_{\infty}} \displaystyle=\max_{\{a_{i}\}}\frac{\frac{1}{\sigma^{2}}\sum_{i=-\infty}^{\infty}ia_{i}e^{-i^{2}/2\sigma^{2}}}{\left\|\sum_{i=-\infty}^{\infty}a_{i}e^{-(x-i)^{2}/2\sigma^{2}}\right\|_{\infty}} (12) \displaystyle\approx\frac{1}{\sigma^{2}}\max_{a(\cdot)}\frac{\int^{\infty}_{-\infty}a(z)xe^{-z^{2}/2\sigma^{2}}\mathrm{d}z}{\left\|\int^{\infty}_{-\infty}a(z)e^{-(x-z)^{2}/2\sigma^{2}}\mathrm{d}z\right\|_{\infty}} (13) \displaystyle=\frac{1}{\sigma}\max_{a(\cdot)}\frac{\int^{\infty}_{-\infty}a(\sigma z)ze^{-z^{2}/2}\mathrm{d}z}{\left\|\int^{\infty}_{-\infty}a(\sigma z)e^{-(x-z)^{2}/2}\mathrm{d}z\right\|_{\infty}} (14) \displaystyle\propto\frac{1}{\sigma} (15) For the first equality, the numerator is derived and evaluated at x=0 . After that, the summation is approximated by an integral, and the coefficients become a continuous function. Subsequently, we change the variable of integration. The RHS has no dependence on \sigma other than the \sigma^{-1} factor since a(\sigma z) dependence can be absorbed by the maximization over a(\cdot) . We conclude that this quantity is upper bounded by O(\sigma^{-1}) . Therefore, the larger the smearing width, the more bounded the derivatives are. Appendix E Energy Conservation Tests in Molecular Dynamics (MD) Problem Setup. We assess the differentiability and smoothness of our models by performing MD simulations under the microcanonical (NVE) ensemble with the Verlocity Verlet integrator [ 29 ] . The degree of non-conservative behavior is upper bounded by the magnitude of the higher derivatives of the PES [ 20 ] . Thus, even if a model is infinitely differentiable, it can still manifest non-conservative behavior if the higher derivatives are not bounded. The “smoothness” here is defined as the boundedness of PES derivatives, which is different from the force smoothness deviation defined by BSCT. We follow the protocol proposed by [ 15 ] and use the seven molecules in the MD22 dataset [ 10 ] as out-of-distribution test systems for the MD simulation data relative to SPICE training. Each simulation integrates dynamics for 100\mathrm{ps} with 1\mathrm{fs} time steps. Results. Figure 8 compares energy drift among MinDScAIP models with different prediction heads and graph structures. The direct force MinDScAIP model cannot conserve energy due to its inherently non-conservative nature. The gradient-based force MinDScAIP model with a standard non-differentiable kNN graph cannot conserve energy due to the piecewise continuity of the standard kNN graph, which results in unbounded first and higher-order derivatives. In contrast, the gradient-based force MinDScAIP model with a differentiable kNN graph can conserve energy. It is infinitely differentiable, and this also demonstrates that its higher derivatives are bounded. Figure 8 : The energy drift of three different MinDScAIP models during microcanonical (NVE) ensembles averaged over seven MD22 molecular trajectories. The direct-force and non-differentiable kNN gradient models (blue and orange curves) show significant energy drift due to non-conservative or discontinuous predictions. The gradient-based Diff-kNN model employs the Diff-kNN algorithm described in Section 4.2 and conserves the energy (the non-zero drift is due to the first-step error of the Verlet integrator). Appendix F Inference Efficiency Benchmarks Benchmark Setup. We follow the benchmark described in 15 to test the throughput and memory usage of MinDScAIP, MACE-MP-16M, and eSEN-30M-OAM (smaller eSEN model weights are not public) on the diamond system. We vary the number of supercells included in the image to test model’s throughput and memory usage as a function of number of atoms. All benchmark is done on a single 80GB A100. Results Table 6 summarizes the inference benchmark results. We highlight that MinDScAIP-60M is substantially outperforms MACE-MP-16M in material stability predictions (Matbench F1) by including more parameters (4x the size of MACE-MP-16M), while being just slightly slower than MACE-MP-16M. Comparing against eSEN-30M-OAM, MinDScAIP offers comparable accuracy while being order-of-magnitude faster and memory-efficient than eSEN. Both results suggests that MinDScAIP is more scalable than the current architectures. Table 6 : Inference efficiency benchmark of three selected models. Models are tested on the diamond system with varying number of supercells. The benchmark is done on a single 80GB A100. MinDScAIP promises accuracy similar to eSEN while being just slightly slower than MACE-MP. Throughput (Millions of Steps Per Day) Memory Usage (GB) Number of Atoms MinDScAIP-60M MACE-MP-16M eSEN-30M-OAM MinDScAIP-60M MACE-MP-16M eSEN-30M-OAM 216 1.15 1.49 0.09 4.32 2.3 35.81 512 0.51 0.72 OOM 9.83 5.19 OOM 1000 0.26 0.37 OOM 19.7 10.0 OOM 1728 0.15 0.23 OOM 35.35 17.16 OOM 2744 0.09 0.15 OOM 59.57 27.06 OOM Appendix G Hyperparameters The hyperparameters used for the experiments are summarized in Table 7 For the MPTrj experiments, we adopt the training procedure proposed in [ 15 ] and models are pretrained with direct-force and DeNS targets [ 32 ] , then fine-tuned with a gradient-based prediction head following 15 . The parameters for pretraining are specified by the parentheses. Table 7 : The hyperparameters used for MinDScAIP experiments. The parameters used for pretraining are indicated by parentheses: direct force prediction is enabled during pretraining (✓in the parentheses) and gradient-based prediction is used for fine-tuning (✗that follows the parentheses). dataset SPICE MACE-OFF MPTrj-ablations hyperparameter Direct-Force Grad. kNN Grad. Diff-kNN Small Medium Large W. Decay Smearing Temperature Smear. & Temp. Weak Medium Strong final-60M Embedding dimension 512 512 512 128 256 512 512 512 512 512 256 256 256 512 Hidden factor 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Number of layers 8 8 8 8 8 8 8 8 8 8 8 8 8 8 Number of heads 32 32 32 8 16 32 32 32 32 32 16 16 16 32 kNN k 30 30 30 30 30 30 30 30 30 30 30 30 30 30 Diff-kNN ✗ ✗ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Diff-kNN d_{0} N/A N/A 0.2Å 0.2Å 0.2Å 0.2Å 0.2Å 0.2Å 0.2Å 0.2Å 0.2Å 0.2Å 0.2Å 0.2Å Diff-kNN \beta N/A N/A 10 10 10 10 10 10 10 10 10 10 10 10 Mem-eff. Diff-kNN \Delta N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A 20 20 20 20 Radius cutoff 6Å 6Å 6Å 6Å 6Å 6Å 6Å 6Å 6Å 6Å 6Å 6Å 6Å 6Å Number of radial basis 128 128 128 128 128 128 128 128 128 128 128 128 128 128 Smearing Scale 1 1 1 1 1 1 1 5 1 5 1 5 10 10 Angular l_{max} 5 5 5 5 5 5 5 5 5 5 5 5 5 5 Temperature \tau 1 1 1 1 1 1 1 1 10 10 1 5 10 10 Direct-Force Prediction ✓ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ (✓) ✗ (✓) ✗ (✓) ✗ (✓) ✗ Batch size 128 128 128 128 128 128 128 128 128 128 256 256 256 (128)256 Weight decay 10^{-3} 10^{-3} 10^{-3} 10^{-3} 10^{-3} 10^{-3} 5\times 10^{-2} 5\times 10^{-2} 5\times 10^{-2} 5\times 10^{-2} 10^{-3} 10^{-2} 5\times 10^{-2} 10^{-2} Warmup factor 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 (0.2) 0. (0.2) 0. (0.2) 0. (0.2) 0. Warmup epochs 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Number of Epochs 100 100 100 100 100 100 100 100 100 100 (60) 40 (60) 40 (60) 40 (60) 40 EMA decay 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 0.999 Gradient Clipping 100 100 100 100 100 100 100 100 100 100 100 100 100 100 Energy Loss Weight 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Forces Loss Weight 2 2 2 2 2 2 2 2 2 2 (2)4 (2)4 (2)4 (2)4 Stress Loss Weight N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A 10 10 10 10