Scaling Machine Learning Interatomic Potentials with Mixtures of Experts Yuzhi Liu, Duo Zhang, Anyang Peng, Weinan E, Linfeng Zhang, Han Wang Abstract Machine Learning Interatomic Potentials (MLIPs) enable accurate large-scale atomistic simulations, yet improving their expressive capacity efficiently remains challenging. Here we systematically develop Mixture-of-Experts (MoE) and Mixture-of-Linear-Experts (MoLE) architectures for MLIPs and analyze the effects of routing strategies and expert designs. We show that sparse activation combined with shared experts yields substantial performance gains, and that nonlinear MoE formulations outperform MoLE when shared experts are present, underscoring the importance of nonlinear expert specialization. Furthermore, element-wise routing consistently surpasses configuration-level routing, while global MoE routing often leads to numerical instability. The resulting element-wise MoE model achieves state-of-the-art accuracy across the OMol25, OMat24, and OC20M benchmarks. Analysis of routing patterns reveals chemically interpretable expert specialization aligned with periodic-table trends, indicating that the model effectively captures element-specific chemical characteristics for precise interatomic modeling. 1 Introduction Over the past few years, MLIPs have rapidly advanced as a powerful strategy to bridge the divide between the high accuracy of Quantum Mechanical (QM) methods and the computational efficiency of classical force fields 10 ; 6 ; 37 ; 53 ; 38 ; 23 ; 8 ; 18 ; 43 ; 32 ; 15 ; 45 ; 13 . The fundamental principle of MLIPs lies in approximating the high-dimensional Potential Energy Surface(PES) by learning from extensive datasets generated via first-principles calculations, such as Density Functional Theory(DFT) 25 ; 28 ; 1 ; 13 ; 35 ; 9 . Once trained, these potentials can be employed in large-scale molecular dynamics(MD) simulations, enabling the study of complex atomistic processes at a fraction of the computational cost of direct QM approaches 53 ; 18 ; 43 ; 32 . As a result, MLIPs have found successful applications across diverse fields, including drug discovery 3 ; 49 ; 42 ; 36 , materials science 5 ; 19 ; 44 ; 26 ; 31 , and catalysis design 47 ; 12 ; 14 , providing a scalable and accurate framework for exploring chemical space and predicting material properties. To more accurately model interatomic interactions, a variety of pioneering neural network architectures, such as MACE 7 , SevenNet 34 , DPA 50 ; 51 , eSEN 22 , and AlphaNet 48 , have been developed. In pursuit of broader applicability, researchers have progressively scaled up model parameters and trained these architectures on increasingly large datasets, often employing multi-task learning strategies 51 ; 46 ; 41 . A notable observation emerging from these efforts is the presence of scaling laws: similar to trends observed in Large Language Models (LLMs) 27 ; 24 , the performance of these interatomic potentials improves predictably with increases in both model size and training data volume 52 ; 46 . This scaling behavior provides a clear and encouraging roadmap for advancing the accuracy and capability of future machine-learned potentials. Despite the clear benefits of scaling, simply expanding model capacity through deeper or wider dense architectures remains computationally prohibitive in practice. Two primary bottlenecks hinder this approach: first, the all-to-all computational dependency in dense models limits parallel efficiency 33 ; 40 ; Second, as depth and width increase, the optimization landscape grows increasingly complex, resulting in diminishing returns and heightened training instability 27 ; 24 . To circumvent these limitations, MoE architectures have emerged as a cornerstone of large-scale modeling 39 ; 21 ; 29 ; 54 ; 20 ; 11 . An MoE layer replaces the standard dense feed-forward network with a collection of independent experts, governed by a learnable routing mechanism that activates only a sparse subset of parameters for any given input. This design effectively decouples the total model capacity from the computational cost per token 39 ; 29 . Architecturally, MoE is inherently more conducive to parallelism; by distributing experts across different devices, it reduces the communication bottlenecks typical of standard model parallelism. Consequently, for a fixed computational budget, an MoE model can leverage a significantly larger total parameter count to achieve lower validation loss and superior representational power compared to its dense counterparts 21 ; 54 ; 20 ; 2 . Although this scaling paradigm is highly attractive for MLIPs, its direct adoption faces two fundamental challenges. First, standard MoE designs are often incompatible with the equivariant representations employed by MLIP architectures based on equivariant graph neural networks (GNN), such as MACE 7 , SevenNet 34 , eSEN 22 , and AlphaNet 48 . Second, unlike the discrete token representations in language models, where sparse expert activation is natural, MLIPs model continuous potential energy surfaces. Abrupt expert switching in this setting can introduce numerical instabilities or non-physical discontinuities, potentially violating the law of energy conservation. To address these challenges, the MoLE framework, exemplified by the UMA models, was introduced 46 . MoLE resolves the first issue by employing a linear combination of experts, ensuring compatibility with equivariant representations. However, the absence of nonlinear activation on the expert outputs may limit the expressive capacity of the resulting representations. To address the second issue, MoLE designs expert routing weights to depend only on configuration-level features derived from the chemical species present in the system, with these weights shared across all atoms in a configuration. Consequently, the expert gating remains invariant with respect to atomic positions, preventing abrupt expert switching and ensuring that the predicted PES remains smooth and differentiable. Nevertheless, this global gating assumption precludes the use of distinct expert knowledge for different atomic species within the same configuration, thereby limiting the model’s capacity to model potential energy contributions through element-wise expert routing. In this work, we propose an MoE-integrated DPA3 MLIP architecture. DPA3 is a graph neural network built on a line-graph series and has demonstrated state-of-the-art accuracy across diverse benchmarks 52 , making it a robust baseline for evaluating MoE-based extensions. Crucially, DPA3 employs exclusively invariant node and edge features, which permits valid nonlinear operations on expert outputs and thus enables the seamless incorporation of standard MoE mechanisms within its framework. Since invariant GNNs can be viewed as a special case of equivariant GNNs, the MoLE architecture can also be implemented within DPA3, providing a unified and controlled setting for direct comparisons between MoE and MoLE designs in terms of their effectiveness for scaling MLIP performance. Building on this foundation, we introduce element-wise expert gating, yielding an element-dependent MoE that models potential-energy contributions through species-specific expert routing. In addition, we incorporate a shared-expert mechanism 17 , ensuring that a subset of experts is always activated to capture knowledge common across all element types. The effectiveness and robustness of the proposed MoE design are rigorously validated on the OMol25, OMat24, and OC20M benchmarks, where it achieves substantial improvements over the baseline. Notably, Principal Component Analysis (PCA) of the expert latent representations reveals that the learned expert distributions spontaneously recover trends consistent with the periodic table, suggesting that the model implicitly encodes fundamental chemical identities to achieve enhanced predictive accuracy. 2 Methodology Figure 1: Schematic illustration of the model architectures. (a) The standard MLP framework. (b) The MoE-E framework, featuring a router that dynamically selects 3 active experts from the pool, complemented by 1 fixed shared expert to capture universal features. (c) The MoLE-E framework, where 4 expert weights are linearly combined via the router to modulate the main network weights, including 1 shared expert component. In this work, we use the DPA3 model architecture to illustrate the incorporation of a MoE design into a MLIP. The same approach can be readily extended to other graph neural networks that employ invariant features, without substantial modification. The model takes as input the atomic coordinates and chemical species, and outputs the total potential energy E of the system. DPA3 assumes an atom-wise energy decomposition, E=\sum_{i}E_{i},\hskip 9.24994ptE_{i}=E_{i}\!\left(v_{i}^{1,L}\right), (1) where E_{i} denotes the energy contribution of atom i , which is derived from the final-layer vertex feature v_{i}^{1,L} . This feature represents the state of atom i within the primary graph of the line-graph series after L update layers. The majority of DPA3 model parameters reside in the dense layers used to construct messages during message passing along graph edges, which connect atom pairs whose interatomic distance falls within a predefined cutoff radius. A detailed description of the DPA3 architecture is provided in Ref. 52 . For each input feature x_{i} associated with atom i , a dense layer computes y_{i}=\sigma\!\left(Wx_{i}+b\right), (2) where W and b denote the weight matrix and bias vector, respectively, \sigma is a nonlinear activation function, and y_{i} is the resulting output (see also Fig. 1 (a)). The representational capacity of the model can be increased by enlarging these dense layers, i.e., by increasing the dimensionality of the weight and bias parameters. Instead of a single dense transformation, the MoE introduces multiple parallel expert subnetworks, each composed of a relatively smaller dense layer. The contribution of each expert to the output is modulated by gating weights, which are produced by a router that selects appropriate weights for atom i based on its chemical identity, encoded by the atomic number Z_{i} . The experts can be sparsely activated, such that only a subset of experts receives nonzero gating weights for a given atom. In addition, a subset of experts may always be activated with fixed weights independent of chemical species; these are referred to as shared experts , as they capture knowledge common to all atoms regardless of chemical type. A schematic illustration of the MoE architecture is provided in Fig. 1 (b). Assume that the MoE contains a total of N experts, consisting of I routed experts and N-I shared experts. The MoE transformation is defined as \displaystyle y_{i} \displaystyle=\sum_{j=1}^{I}\alpha_{ij}\,\sigma\!\left(W_{j}x_{i}+b_{j}\right)+\sum_{j=I+1}^{N}\sigma\!\left(W_{j}x_{i}+b_{j}\right), (3) \displaystyle\alpha_{ij} \displaystyle=\begin{cases}s_{ij},&\text{if }s_{ij}\in\mathrm{TopK}\!\left(\{s_{im}\},\,K^{\prime}\right),\\ 0,&\text{otherwise},\end{cases} (4) \displaystyle s_{ij} \displaystyle=\mathrm{softmax}\!\left(W_{e}u_{i}\right), (5) where \sigma(\cdot) denotes a nonlinear activation function, W_{j} and b_{j} are the trainable expert-specific weights and bias, and \alpha_{ij} is the gating weight applied to the j -th routed expert for atom i . The operator \mathrm{TopK}\!\left(\{s_{im}\},K^{\prime}\right) selects the largest K^{\prime}\leq I elements from the set of gating scores \{s_{im}\mid 1\leq m\leq I\} , thereby enforcing sparse expert activation. Including the (N-I) shared experts that are always active, the total number of activated experts is denoted by K , satisfying K=K^{\prime}+(N-I). (6) The gating scores s_{ij} are computed according to equation 5 , where W_{e}\in\mathbb{R}^{I\times M} is a trainable routing matrix and u_{i}\in\mathbb{R}^{M} is a latent representation of dimension M , encoding the chemical identity of atom i . The hidden representation u_{i} is defined as u_{i}=\mathrm{MLP}\!\left(\mathrm{one\_hot}(Z_{i})\right), (7) where the atomic number Z_{i} is first mapped to a one-hot vector and subsequently projected into a continuous embedding space via a multilayer perceptron (MLP). This element-wise routing mechanism enables atom-type–dependent expert selection while preserving smoothness and differentiability, which are essential for stable and physically consistent interatomic potential modeling. In contrast, the UMA model 46 adopts an atom-averaged routing representation that incorporates dataset identifiers along with charge and spin information. In our implementation, we utilize this atom-averaged routing scheme to ensure a fair and consistent comparison with element-wise routing. Specifically, we define a global routing vector through mean pooling: \bar{u}=\frac{1}{N_{\mathrm{atom}}}\sum_{i}u_{i}, (8) where the unified representation \bar{u} is applied across all atoms in the system to determine gating scores. Under this formulation, atomic representations are averaged across the entire configuration, and all atoms share an identical set of gating scores and weights. To distinguish these two routing strategies, we refer to the atom-wise and global (configuration-wise) representations as MoE-E and MoE-G , respectively. In the MoE formulation, input features are first processed by expert subnetworks and then linearly combined using gating weights. In contrast, in the MoLE formulation the features are first linearly transformed by expert-specific weights, after which the expert contributions are aggregated via gating weights, and a nonlinear activation function is applied at the end, as illustrated in Fig. 1 (c). This yields \displaystyle y_{i}=\sigma\!\left(\sum_{j=1}^{I}\alpha_{ij}\left(W_{j}x_{i}+b_{j}\right)+\sum_{j=I+1}^{N}\left(W_{j}x_{i}+b_{j}\right)\right), (9) \displaystyle\alpha_{ij}=\mathrm{softmax}\bigg(W_{e}u_{i}\bigg) (10) where \sigma(\cdot) denotes the activation function. Equation equation 9 is mathematically equivalent to a single dense layer with effective parameters, \displaystyle y_{i} \displaystyle=\sigma\!\left(\bar{W}x_{i}+\bar{b}\right), (11) \displaystyle\bar{W} \displaystyle=\sum_{j=1}^{I}\alpha_{ij}W_{j}+\sum_{j=I+1}^{N}W_{j}, (12) \displaystyle\bar{b} \displaystyle=\sum_{j=1}^{I}\alpha_{ij}b_{j}+\sum_{j=I+1}^{N}b_{j}. (13) Thus, MoLE can be interpreted as a data-dependent linear transformation whose weights and biases are modulated by the gating mechanism prior to the nonlinearity. Analogous to MoE, MoLE admits two variants depending on how chemical identity information is encoded in the router. We denote these as MoLE-E and MoLE-G , corresponding to element-wise and global (configuration-wise) routing representations, respectively. 3 results In this section, we benchmark the proposed MoE-based architectures. To ensure a controlled and consistent comparison, we adopt a standard 6-layer DPA3 model as our baseline, with all hyperparameters detailed in the Supplementary Table 1. Performance is primarily illustrated using normalized Mean Absolute Error (MAE), while absolute MAE values are provided in the Supplementary Note 2 for comprehensive reference. We evaluate 6-layer variants of MoE and MoLE with both element-wise and global routing, and systematically compare their performance against the baseline to quantify the relative gains enabled by different expert designs. The detailed hyperparameters for the different models are outlined in Supplementary Note 1. 3.1 Sparse activation and shared experts Figure 2: Performance benchmarks on the OMol25 dataset. (a) Impact of shared expert allocation. Normalized energy and force MAE for MoE-E are shown as a function of the shared expert ratio, using a total pool of 64 experts. The x-axis denotes the proportion of shared experts (e.g., (2/6) indicates 2 shared experts among 6 total activated experts). Background shading demarcates different K activation settings for MoE-E. The horizontal dashed line provides a performance baseline corresponding to a standard 6-layer DPA3 model. See Supplementary Table 2 for detailed values. (b) Scaling with total expert count: Comparative analysis of normalized energy (left) and force (right) MAE as a function of the total number of experts. Two baselines are provided: the standard 6-layer DPA3 model and a “ 4\times Params” variant achieved by doubling the hidden dimension of the 6-layer DPA3 model. See Supplementary Table 3 for detailed values. Two important design choices in MoE are sparse expert activation and the use of shared experts . Figure 2 (a) systematically investigates the impact of these design choices using the OMol25 dataset as a benchmark 30 . Covering 83 chemical elements, OMol25 displays remarkable chemical diversity, including complex intermolecular interactions, explicit solvation, variable charge and spin states, as well as reactive configurations. In this experiment, the total number of experts is fixed at N=64 , while the number of activated experts K varies from 2 to 8. For each value of K , multiple configurations with different numbers of shared experts are evaluated. Overall, model performance improves with increasing K . In the absence of shared experts, the normalized energy (force) MAEs for K=2,4,6, and 8 are 1.082 (0.965), 0.965 (0.919), 0.923 (0.902), and 0.926 (0.900), respectively. This trend is primarily driven by the effective linear expansion of model capacity, since a configuration with K activated experts corresponds to an approximately K -fold increase in the number of active parameters. However, several nontrivial behaviors are observed. Notably, the K=2 model without shared experts, despite having roughly twice the parameter count of the baseline, exhibits a degraded energy MAE. In addition, performance saturates between K=6 and K=8 , indicating diminishing returns where the additional computational budget no longer translates into measurable accuracy gains. These results suggest that naive scaling of the number of activated experts alone is insufficient. Across nearly all choices of activated experts K , introducing shared experts consistently improves model performance. This effect is particularly pronounced for K=8 , where increasing the number of shared experts from zero to six reduces the normalized MAEs for energy and force by 0.269 and 0.200, respectively. Notably, performance is optimized when the number of shared experts approaches approximately half of the activated experts, as observed for the K=4,6, and 8 configurations. Beyond this regime, further increasing the fraction of shared experts yields diminishing returns or even slight performance degradation, for example, the force MAE increases when the number of shared experts rises from 2 to 3 in the K=4 configuration. Overall, these results indicate that allocating roughly half of the activated experts as shared experts provides an effective trade-off between capturing global chemical knowledge common across elements and preserving sufficient capacity for element-specific specialization. When adopting a configuration in which half of the activated experts are shared, the normalized energy (force) MAEs for K=2,4,6, and 8 are reduced to 0.939 (0.913), 0.771 (0.779), 0.740 (0.747), and 0.673 (0.715), respectively. In stark contrast to the no–shared-expert setting, the accuracy improvement in this regime does not plateau at K=8 , but instead continues to scale favorably with increasing model capacity. These results highlight that shared experts are indispensable for effectively exploiting larger expert pools and for achieving stable, monotonic performance scaling. Importantly, for a fixed number of activated experts K , increasing the number of shared experts does not alter the total parameter count or computational cost. Consequently, the observed accuracy gains from shared experts are achieved at essentially no additional computational overhead. 3.2 MoE-E vs MoLE-E Figure 2 (b) presents a comparison between the MoE-E and MoLE-E architectures, both of which maintain an identical parameter count. The fundamental distinction between these two designs lies in the sequencing of the nonlinear transformation relative to expert aggregation: MoE-E applies the activation function locally within each expert before mixing their outputs, whereas MoLE-E performs a linear aggregation of expert contributions followed by a global activation function, Furthermore, while MoE-E utilizes a sparse activation of K experts, MoLE-E employs a dense approach where all experts are activated. For a controlled comparison, all MoE-E configurations are evaluated with the same number of activated experts ( K=4 ). The normalized energy and force accuracies of the MoE-E and MoLE-E architectures are reported as functions of the total number of experts N . Despite the architectural difference between the sparsely activated MoE-E and the fully activated MoLE-E, both models show comparable performance without shared experts. Across various expert scales, the discrepancies in normalized energy and force MAE are negligible, staying within 0.02 and 0.03 respectively. In both architectures, substantial performance gains are observed when increasing the number of experts from N=4 to N=16 ; however, the improvement saturates beyond N=16 , with no significant additional accuracy gains when further increasing the expert count to N=64 . The integration of shared experts yields markedly different gains for the two architectures. When two shared experts are introduced, the MoLE-E variant exhibits a reduction in normalized energy MAE of approximately 0.11 , accompanied by a decrease in normalized force MAE ranging from 0.06 to 0.09 . Although the force accuracy shows a slight improvement when increasing the total number of experts from N=16 to N=64 , the corresponding energy accuracy displays a reversed trend increasing from 0.846 to 0.865, partially offsetting the overall performance gain. In contrast, MoE-E demonstrates a substantially stronger response to the introduction of shared experts. For the same transition from zero to two shared experts, MoE-E achieves a larger reduction in normalized energy MAE, ranging from 0.18 to 0.23 , and a decrease in normalized force MAE of approximately 0.13 to 0.14 . Moreover, the accuracy of MoE-E continues to improve as the number of experts increases from N=16 to N=64 , indicating that the MoE-E architecture can effectively leverage a larger expert pool to achieve further performance gains. 3.3 Parameter Efficiency of MoE-E Since the MoE-E model with four activated experts ( K=4 ) utilizes four times the activated parameters of the baseline, we established a widened dense baseline for a fair comparison. This “ 4\times Params Baseline” was constructed by doubling the dimensionality of all hidden features to match the parameter scale. This 4\times Params baseline, yields improvements of 0.181 and 0.169 in normalized energy and force accuracy, respectively, relative to the standard model (Fig. 2 (b)). As the total number of experts increases, the MoE-E model with two shared experts consistently outperforms the widened baseline. In particular, with N=64 experts, MoE-E achieves additional gains of 0.052 and 0.048 in energy and force precision, respectively. These results demonstrate that the sparse-activation strategy in MoE-E provides greater expressive capacity than conventional width-based scaling, enabling higher predictive accuracy than dense architectural scaling under comparable computational budgets. 3.4 Element-wise vs Global routing Model Routing Normalized Energy MAE Normalized Force MAE 4 Experts 16 Experts 64 Experts 4 Experts 16 Experts 64 Experts MoE Element-wise (E) 0.814 0.780 0.771 0.809 0.793 0.779 Global (G) N/A N/A N/A N/A N/A N/A MoLE Element-wise (E) 0.913 0.847 0.866 0.914 0.857 0.847 Global (G) 0.981 0.946 0.952 0.965 0.936 0.950 Table 1: Normalized Energy and Force MAE values. The results compare the performance of Global (G) vs. Element-wise (E) routing across different expert counts. All MAEs are normalized against a 6-layer DPA3 baseline; thus, values represent relative error ratios where a lower score signifies superior performance relative to the baseline. See Supplementary Table 4 for detailed values. We perform a comparative analysis of element-wise routing (MoE-E and MoLE-E) and their global counterparts (MoE-G and MoLE-G) on the OMol25 dataset 30 . This breadth provides a stringent benchmark for evaluating the stability and accuracy of different expert-routing strategies across a highly heterogeneous chemical space. Table 1 summarizes the comparative performance of global and element-wise routing architectures across all architectural variants. In these experiments, we vary the total number of experts N while fixing the number of activated experts to K=4 , among which two are shared experts. As shown in Table 1 , the MoE-G architecture suffers from catastrophic training failure: its energy and force MAEs exhibit severe numerical instability and fail to converge. As a result, its accuracy is not reported. For the remaining variants (MoE-E, MoLE-E, and MoLE-G), the energy and force MAEs are reported after normalization by the corresponding MAEs of the baseline standard DPA3 model. Overall, increasing the number of experts from N=4 to N=16 consistently improves both energy and force accuracy across all stable variants. However, further increasing the number of experts from N=16 to N=64 yields diminishing returns: performance gains saturate for MoE-E, while accuracy degrades for MoLE-G and for the energy prediction of MoLE-E. Comparing different mixture designs, MoE-E consistently outperforms both MoLE variants across all expert scales. Moreover, MoLE-G systematically underperforms its element-wise counterpart MoLE-E, with MoLE-E achieving approximately 5\%\text{--}10\% lower MAEs in both energy and force predictions across all values of N . Taken together, these results demonstrate that element-wise routing is structurally superior to global routing and is essential for the stable and effective integration of MoE architectures into MLIPs. 3.5 Performance on Multiple Datasets Figure 3: Performance benchmarks for normalized energy (left) and force (right) predictions across three datasets: OMol25, OMat24, and OC20M. The baseline corresponds to the standard DPA3 model, while the ”6 \times Params” is scaled to six times the total parameters of the baseline via wider hidden layers. The MoLE-E model employs 64 experts with 3 shared experts, and the MoE-E model uses 64 experts with 3 shared experts out of K=6 activated experts. See Supplementary Table 5 for detailed values. To comprehensively evaluate the robustness and generalization capability of the proposed MoE-based interatomic potentials, we benchmark the models on several widely recognized datasets, including OMol25 30 , OMat24 4 , and OC20M 14 . These datasets collectively span a broad range of chemical environments and material systems, including organic molecules, extended solid-state materials, and catalytic surface reactions, thereby enabling a representative assessment across multiple domains of computational chemistry and materials science. The inclusion of these diverse benchmarks allows for a rigorous evaluation of model performance under heterogeneous conditions and highlights the transferability, stability, and cross-domain generalization capability of the MoE architecture in realistic applications. Fig. 3 summarizes model performance across multiple benchmark datasets. For MoE-E, we employ K=6 activated experts, among which three are shared, with the total number of experts fixed at N=64 . Our evaluation strategy is designed to explicitly highlight the advantages of MoE-E by comparing it against two distinct configurations. First, to control for the total parameter count, we evaluate it against MoLE-E, which maintains an identical architecture to MoE-E model, specifically configured with 3 shared experts out of a total of 64 experts. Second, we introduce a “6 \times Params” model to match the activated parameter count of MoE-E. This model is scaled to six times the total size of the baseline via wider hidden layers, allowing us to compare performance under equivalent computational budgets. As shown in Fig. 3 , all enhanced models—MoE-E, MoLE-E, and the 6\times Params model—achieve substantial improvements over the baseline model (indicated by the horizontal line at 1.0) across all three datasets. Notably, MoE-E consistently outperforms MoLE-E across all evaluated benchmarks, demonstrating its robust generalization capabilities. Specifically, we observed a reduction in normalized Energy MAE by 0.10, 0.02, and 0.03, complemented by a decrease in normalized Force MAE of 0.10, 0.03, and 0.04 on the OMol25, OMat24, and OC20M datasets, respectively. These improvements underscore the superiority of nonlinear expert aggregation over linear alternatives in capturing complex interatomic potentials. Furthermore, MoE-E even surpasses a dense baseline with 6 \times the parameter count, yielding additional error reductions of up to 0.07 in Energy and 0.05 in Force. This highlights the exceptional parameter efficiency inherent to the MoE-E architecture. Furthermore, we observe that the performance gains yielded by the enhanced models exhibit significant heterogeneity across different datasets. Specifically, the MoE-E model manifests substantial predictive advantages on the OMol25 (molecular) and OC20M (catalytic) benchmarks. However, its efficacy becomes more marginal on the OMat24 (solid-state) dataset, where only a moderate reduction in MAE is recorded. This divergence in performance is primarily ascribed to the inductive biases inherent in the DPA3 backbone. 3.6 Experts weighting Distribution Patterns Figure 4: Expert weighting distribution analysis via PCA for various elemental groups. (a) Overview of the general distribution for all elements. (b) Distribution patterns for alkali metals, alkaline earth metals, and noble gases. (c) Regional distribution of p-block groups except noble gas, specifically Boron (Group 13) through Halogen (Group 17) elements.(d) Spatial clustering of transition metal elements. To gain deeper insights into the interpretability of the MoE-E model, we investigate the distribution patterns of experts weight as defined by the projection W_{e}u_{i} . This analysis aims to elucidate how the routing mechanism allocates learning responsibilities among experts according to elemental characteristics. We conduct this analysis on the OMat24 dataset, which provides an ideal testbed due to its extensive coverage of 86 elements with a relatively uniform distribution across the periodic table. Principal component analysis (PCA) of the expert-weighting distributions reveals highly efficient dimensionality reduction, with the first two principal components accounting for 93.57\% of the total explained variance. Figure 4 presents the projected expert distributions across chemical species, revealing systematic organization in the PCA space 39 ; 16 . As shown in Fig. 4 (a), which illustrates the global distribution of all elements, clear chemical groupings emerge: lanthanides and actinides cluster on the left side of the PCA manifold, while transition metals predominantly occupy the central region. Figure 4 (b) highlights the alkali metals (Group I), alkaline-earth metals (Group II), and noble gases (Group VIII), while Fig. 4 (c) focuses on elements from Groups III to VII together with hydrogen. A consistent pattern emerges across chemical groups: elements within the same group arrange themselves diagonally from the top-left to the bottom-right of the PCA space. This spatial ordering correlates strongly with increasing atomic number, with lighter elements located in the top-left region and heavier elements positioned toward the bottom-right. Along the period direction, as shown in Fig. 4 (c), elements with pronounced non-metallic character exhibit a systematic shift from the top-right toward the bottom-left of the PCA space with increasing atomic number. In contrast, strongly metallic elements follow the opposite orientation, extending from the bottom-left (Group I) to the top-right (Group II), as visualized in Fig. 4 (b). Elements with intermediate chemical behavior—particularly those in Groups III (Al, Ga, In, Tl) and IV (Si, Ge, Sn, Pb)—display overlapping distributions in the PCA embedding (Fig. 4 (c)). Noble gases, although electronically distinct, constitute an exception: their embeddings partially overlap with those of intermediate elements, which may reflect their relatively limited representation in the OMat24 dataset. As illustrated in Fig. 4 (d), transition metals are predominantly concentrated near the center of the PCA space. Within this cluster, elements belonging to the same period exhibit a clear diagonal alignment: as the group number increases, their distributions shift from the top-right toward the bottom-left. This trend is further reflected by the color gradient from dark to light green, capturing the systematic evolution of their electronic characteristics. These structured distributions demonstrate that the MoE-E router effectively encodes elemental characteristics in a manner consistent with fundamental physicochemical principles. The observed organization—group-wise diagonals, periodic progressions, and the distinct clustering of transition metals and f -block elements—indicates that the model internalizes a chemically meaningful representation of the periodic table. This interpretable, element-wise task specialization provides a clear mechanistic explanation for the superior accuracy and generalization of MoE-E. By explicitly leveraging elemental identity to guide expert routing, the model captures sharper distinctions among chemically diverse species while allocating expert capacity more efficiently across compositional space. Such a design bridges learned representations with established chemical intuition, offering a principled and scalable pathway toward more accurate, transferable, and interpretable MLIPs. 4 Conclusion In this work, we present a systematic investigation of MoE architectures for MLIPs and demonstrate that sparsely activated expert scaling provides an effective pathway for improving predictive accuracy beyond conventional dense model enlargement. By integrating MoE designs into the DPA3 framework, we demonstrate that sparse activation, shared-expert mechanisms, nonlinear expert specialization, and element-wise expert routing collectively constitute the key ingredients enabling stable and scalable performance gains. Across diverse benchmarks—including OMol25, OMat24, and OC20M—the proposed element-wise MoE architecture consistently achieves substantial improvements in both energy and force prediction accuracy, while also exhibiting interpretable expert specialization patterns that align with periodic chemical trends. Our analysis provides several conceptual insights into MoE design choices for scaling atomistic foundation models. First, the introduction of sparse activation together with shared experts enables the learning of transferable representations of common interatomic interactions, allowing efficient utilization of large expert pools. Second, nonlinear expert aggregation delivers greater representational capacity than linear mixture formulations, underscoring the importance of nonlinear specialization for accurately modeling complex potential-energy surfaces. Third, element-wise routing is essential for stable training and effective specialization in chemically heterogeneous systems, consistently outperforming global routing strategies that may introduce optimization instability. Collectively, these findings establish MoE-based conditional computation as a principled alternative to dense parameter scaling for next-generation large atomistic models. Despite these advances, an important limitation of the present study is that the current implementation does not yet exploit a fully distributed training and inference framework specifically optimized for sparse expert parallelism. While MoE architectures are intrinsically well-suited for distributed environments and offer substantial theoretical scalability advantages over dense scaling, realizing these benefits in practice requires dedicated expert-parallel system design, communication scheduling, and load-balancing strategies. Future work will therefore focus on developing large-scale distributed MoE training and inference pipelines to unlock the full computational efficiency and scaling potential of expert-based atomistic foundation models. Overall, this work demonstrates that MoE architectures provide a scalable and physically grounded paradigm for expanding the expressive capacity of MLIP, offering a promising foundation for increasingly accurate and transferable simulations across the chemical and materials sciences. Data Availability All data supporting the findings of this study are available within the Supplementary Information Competing Interests The authors declare no competing interests. Author Contribution Y.L., D.Z., A.P., H.W. conceived the idea of this work, designed the model structure, and implemented the model. The experiments were mainly designed and performed by Y.L. Other authors performed data collection and model tests on different systems. All authors contributed to the discussions and edited the manuscript.