Capsule graph networks for accurate and interpretable crystalline materials property prediction Authors: Eddah K Sure, Xing Wu, Quan Qian Published: 2025 ABSTRACT AbstractAccurate and interpretable modeling of crystalline materials is essential for understanding the structure–property relationships in materials critical in accelerating materials discovery. While recent graph neural networks (GNNs) have achieved high predictive accuracy, they often struggle to provide physical interpretability and fail to explicitly model the hierarchical and symmetrical nature of crystals. In this work, we introduce Capsule Graph Networks with E(3)-Equivariance (CGN-e3), a novel deep learning framework that integrates equivariant message passing with capsule networks to capture both geometric symmetries and hierarchical motif structures. CGN-e3 leverages E(3)-equivariant message passing to learn physically consistent features and organize them into capsule representations that can disentangle local motifs, such as polyhedral environments, and connects them to global properties. We validate the effectiveness of our framework on bandgap and formation energy prediction, as well as material classification using Materials Project and Matbench datasets. Our model achieves a MAE of 0.054 eV/atom and 0.379 eV on formation energy and bandgap prediction, respectively, outperforming CGCNN and matching the performance of MEGNet on the same dataset, while also providing insightful interpretations of the learned capsule representations.Scientific contribution: We present the first integration of E(3)-equivariant graph neural networks with capsule networks for modeling crystalline materials. This unified architecture captures both the fundamental physical symmetries of 3D space; rotation, translation, reflection and the intrinsic hierarchical part-whole relationships e.g., atoms to polyhedra to extended motifs found in crystal structures. The framework provides an unsupervised pathway for interpretable motif discovery. The dynamic routing-by-agreement mechanism identifies and aggregates structurally significant local environments such as the TiO6 octahedra into higher-order graph-level capsules. This process yields human-intelligible insights by explicitly quantifying the contribution of specific structural motifs to target material properties, moving beyond "black-box" predictions. We validate our framework on key property prediction tasks and provide capsule-level interpretation of the results.Supplementary InformationThe online version contains supplementary material available at 10.1186/s13321-025-01139-5.Keywords: Material representation, Capsule networks, Equivariant machine learning, GNNs, Interpretable machine learning Introduction Crystal structures exhibit complex hierarchical patterns. These patterns are governed by their atomic composition and spatial organization. At the lowest level, atoms form local bonding geometries and motifs such as tetrahedra, octahedra, and other coordination polyhedral. These motifs subsequently aggregate into larger units that adhere to well-defined symmetry operations defined by crystallographic space groups. This inherent multi-level organization plays a critical role in determining a material’s properties. For instance, subtle variations in bond angles within a tetrahedron can significantly alter a material’s electronic or mechanical behavior [1, 2]. In perovskite oxides such as SrTiO3, even slight distortions in the TiO6 octahedra significantly influence their electronic and ferroelectric properties [3–6]. Accurately modeling these atomic systems requires respecting these structural patterns and the fundamental symmetries of 3D space, that is, rotations, translations, and reflections which, collectively, are described by the Euclidean group E(3). The landscape of machine learning (ML) for modelling atomic systems is diverse, encompassing a number of distinct modeling paradigms. As reviewed by Butler et al. [7], these broadly include; descriptor-based models that utilize fixed representations for use with traditional ML algorithms, graph-based networks that operate on an explicit atom-bond topology, and grid-based models that treat structures as 3D images for convolutional processing. Recent works have further demonstrated that descriptor-based and physics-informed approaches remain highly competitive, particularly in low-data regimes or when interpretability is paramount. For instance, physics-informed models leveraging quantum-chemical descriptors have been shown to outperform more complex architectures for certain properties, providing feature importance that aligns with physical intuition [8]. Similarly, convolutional networks operating on grid-projected electron densities can approximate complex electronic structure properties while retaining physically meaningful representations [9]. These works underscore that non-graph models continue to offer distinct advantages. Nevertheless, graph-based representations have emerged as a particularly natural and powerful paradigm for crystalline materials [10], as they explicitly encode the relational and topological structure inherent in atomic systems. A common theme in graph-based material models is the use of nodes to represent a crystal’s atom types and edges as connections between these atoms which can be characterized by interatomic distances or bond length. Architectures like CGCNN [10], SchNet [11], GATGNN [12] and iCGCNN [13] construct convolutions over these graphs, where the initial node features are iteratively updated by aggregating the information from neighboring atoms. However, these models predominantly rely on scalar node representations and often fail to fully capture geometric information such as angles and orientation, which are essential for modeling phenomena like anisotropy or phase behavior. To improve local geometric expressiveness, models such as ALIGNN [14], M3GNet [15], and DimNet [16] seek to incorporate way beyond atoms and bond lengths by explicitly incorporating bond angles and directional information into graph construction. Inspite of these advancements, these models fail to explicitly enforce equivariance and learn the hierarchical patterns inherent in materials resulting in limited generalization and interpretability. Recent advances introduce E(3)-equivariant GNNs. E(3)-equivariant GNNs aim at fully incorporating the full symmetry of 3D Euclidean space, that is, translations, rotations and reflections directly into their architecture. This ensures equivariance such that a networks output transforms with the transformations at input point therefore, preserving physical properties while respecting the symmetry of materials. E(3)-equivariant GNNs leverage message passing schema and geometric tensors that respect crystal symmetries. DimeNet++ [17], while it does not enforce strict equivariance, it encodes directional information through Bessel functions and spherical harmonics to capture angular dependencies explicitly. Models such as PaiNN [18], MACE [19], Allegro [20] and NequIP [21], enforce strict equivariance to Euclidean transformations, allowing them to model directional and angular dependencies with high fidelity for interatomic potentials. These models leverage rotation and translation equivariant operations to maintain physical consistency and data efficiency across 3D atomic environments. EGNN [22] and TENN-e3 [23] extend equivariance principles to a wider physics context by including time reversal symmetry. By encoding fundamental symmetries, E(3)-equivariant models reduce the need for data augmentation and have better generalization across chemical and structural spaces. However, these models still lack a mechanism to capture the hierarchical relationships in atomic systems such as how local atomic environments aggregate into larger structural motifs. Capsule networks, introduced by Sabour and Hinton [24], represent features as a group of vectors or matrices to encode both the presence of a feature and the pose of features. The dynamic routing in capsule networks models the part-whole relationships. While capsule networks were originally developed for 2D image data, the use of capsule concepts have been extended to graphs and 3D data. Capsule enhanced GNNs [25, 26] have demonstrated improved representation of local substructures in graph classification tasks. Zhao [27] extends the application of capsules to the 3D point cloud domain. However, not much work has been done on extending capsules to the material chemistry domain and neither has much work been done to incorporate full E(3)-equivariance into capsule networks, a critical property for materials modeling. CGN-e3 integrates the strengths of both E(3)-equivariance and capsule networks. The model uses E(3)-equivariance GNNs to extract physically consistent scalar and vector features that transform correctly under 3D rotations and translations and then use capsule networks to aggregate these equivariant features hierarchically, enabling the network to disentangle complex spatial motifs and symmetries in crystals. This hierarchical capsule representation aligns with recent work [28] emphasizing on symmetry recognition in materials. The dynamic routing mechanism used in our CGN-e3 is inspired by [24] and we have adapted it here to operate on both scalar and vector features. This routing iteratively refines coupling coefficients between lower-level capsules and higher graph-level capsules, effectively learning which local atomic environments contribute to global material properties. This work bridges geometric deep learning with hierarchical representation learning, establishing a new pathway for hierarchical feature learning, symmetry-aware, interpretable, and efficient modeling in computational materials science. The capsule-based readout provides a physically grounded and interpretable mechanism for material representation and allows the model to generalize well on property prediction tasks on limited data. The key contributions of this work are threefold: First, We design an E(3)-Equivariant message passing network that effectively preserves equivariance to 3D Euclidean transformation to ensure that the predictions are consistent with the underlying physical symmetries of atomic systems. The model leverages RBF functions and spherical harmonics to encode interatomic distances and angular dependencies, providing a more principled approach to capturing geometric relationships critical for material properties. Second, by integrating capsule layers into the GNN framework, this model hierarchically captures both invariant and equivariant features enabling richer and more interpretable representation of the atomic system. The capsule layers enable the model to capture not only local atomic environments but also higher-order spatial motifs and part-whole hierarchies. This dual encoding allows the network to preserve the local geometric details present within the structure, enabling a more faithful reconstruction of the crystal’s local symmetry. Third, the routing mechanism dynamically routes information by agreement between lower-level capsules, which represent local atomic motifs, and higher-level capsules that represent larger, more complex structures. The routing-by-agreement mechanism mimics the hierarchical assembly observed in crystallization, wherein capsules that align in pose and activation reinforce one another to build robust higher-order representations. Methods Development of CGN-e3 The CGN-e3 processes a crystal’s atomic graph where nodes represent atoms with atomic features and edges as neighbor pairs with 3D distances under periodic boundaries through a sequence of equivariant graph convolutions, capsule routing layers and an attention mechanism to predict material properties. First, atomic numbers are embedded via one-hot and distances are encoded by a radial basis function (RBF) expansion. In each convolutional layer, for each atom and neighbor, the relative vector is computed and decomposed into its length and direction. The distance is expanded into a vector of Gaussian RBFs and direction expanded in spherical harmonics. A learnable multilayer perceptron (MLP) is applied to the RBF vector to produce radial filter coefficients, which are then multiplied with the spherical harmonics and tensor-producted with the neighbor’s feature tensor. Clebsch–Gordan tensor product aggregates spherical-harmonic order with neighbor features to produce output of order. The summed messages are then assembled back into irreducible feature vectors for atom. After convolutions, each atom has a tuple of scalar and vector features encoding local geometry and chemistry. These features are treated as primary capsules at the node level. The primary node capsules are then aggregated into a smaller set of graph capsules via an routing mechanism. An attention mechanism weighs nodes to balance different graph sizes, and then dynamic routing iteratively refines coupling coefficients between each lower level capsule and higher graph-level capsule. Figure 1 shows the overall model architecture, the first part on input processing is discussed in this results section while the two main components of the model are elaborated in the methodology section. Architecture overview of CGN-e3. The model processes a crystal structure through four key stages: a Graph Construction: The input crystal is converted into a graph where nodes represent atoms and edges connect atoms within a cutoff radius. b E(3)-Equivariant Message Passing: A series of equivariant graph convolution compute and update graph features using a tensor product between RBF-expanded distances and spherical harmonic-projected directions. c Capsule Network and Dynamic Routing: With atom-level features as primary capsules, a dynamic routing-by-agreement mechanism iteratively refines the coupling between these primary capsules and a smaller set of graph-level secondary capsules, hierarchically aggregating local atomic environments into global structural motifs. d Property Prediction: The graph-level capsules are aggregated via an attention mechanism to produce the final prediction for the target properties Encoding distances and angles We represent a crystal as a graph G=VE. The atoms are represented as nodes i∈V and are encoded using learned embedding of the atomic number. The edges ij∈E connect spatially neighboring atom using distance a cutoff condition. Each edge encodes the interatomic spatial distance dij=ri-rj, bond length rij=rij and unit direction. Distance expansion adopts the Radial Basis Function (RBF) method [29]. The RBF layer encodes the positional, scalar, distances between atoms, dij=ri-rj, as smooth continuous vectors,ϕkdij∈Rk, as to make up the edge features. For each distance dij, a vector of length Nrbf is computed as; Where μk are fixed grid of centers evenly spaced between 0 and cutoff, σk are fixed widths. To ensure a smoother decay near cutoff, an envelope function edij=1-dijdcutoff2is applied to give a vector rbfdij=edij∙ϕ1dij,...,ϕNrbfdij. This vector is invariant to both rotation and translation. The direction r^ij is normalized and the spherical harmonics are computed as; Equivariant message computation For equivariant message computation, CGN-e3 adopts the concept of tensor message passing [30], in which each neighboring atom contributes a tensor product of radial and angular components to the target node’s feature representation. This formulation allows messages to transform consistently under rotations and reflections, ensuring full E(3)-equivariance. Node features are combined with spherical harmonics to produce directional messages that preserve geometric information. We compute the scalar node features xi(0), vector node features xi(1), edge index pairs (i, j), edge attributes rbf(dij), and atomic positions ri∈R3. Scalar and vector messages are then computed separately to maintain the distinction between invariant and equivariant feature channels. Scalar message: For each edge (i, j), the scalar message is computed by concatenating the scalar features of the source, target, and edge attributes: Vector message: For atom j with irreducible feature hj(ℓ) of order ℓ, a learnable weight function Wℓ′ℓ,n maps this to a contribution of order ℓ′. For each neighbor j of node i, the equivariant message of order ℓ′ is constructed using Clebsch–Gordan tensor products of radial, angular, and feature terms: Here, ⊗ denotes the Clebsch–Gordan tensor product, which combines the angular and feature representations into an equivariant output. This operation ensures that if the input atomic coordinates are rotated or translated, the resulting messages transform accordingly. Aggregation: Messages from all neighboring atoms are aggregated by summation: The resulting aggregated messages miscalar and mivector are used to update the scalar and vector node features, respectively. Nonlinearities are applied separately: SiLU (or other linear activations) for scalar features, and a gated sigmoid activation for vector features. In summary, each CGN-e3 layer expands edges via radial basis functions (RBFs) and directional information through spherical harmonics, performs equivariant tensor products to couple geometric and feature information, and applies separate nonlinear transformations to scalar and vector components. This design ensures that learned representations remain physically meaningful and consistent under E(3) transformations. Capsule network After a stack of equivariant convolutions, the updated scalar and vector features are passed into a hierarchical capsule routing module. This stage models the part–whole relationships within the crystal by dynamically routing information from local atomic environments to higher-order structural motifs. Each node-level capsule encodes local atomic geometry and chemical context, while higher-level capsules capture larger-scale motifs such as polyhedral units or coordination complexities.These node level encodings are treated as primary capsules at the node level. They are then projected and transformed into capsule spaces, for an input feature h∈Rd, learned transformations are applied separately for the scalars and vectors. Wscalar∈RC×D2×D2; Wvector∈RC×D2×D2. For every primary capsule i and secondary capsule j; The primary node capsules are then aggregated into a smaller set of graph capsules via an attention-and-routing mechanism where attention weights balance the different graph sizes. Routing by agreement then iteratively refines the coupling coefficients between each node’s capsule and each graph-level capsule. Dynamic routing in capsules The routing-by-agreement mechanism iteratively updates coupling coefficients between capsules, reinforcing connections whose predicted “pose” and activation align across layers. In doing so, the network learns to assemble physically coherent and interpretable representations, where specific capsule activations correspond to distinct structural motifs. During dynamic routing, routing logits bij are initialized to 0; that is bij=0, for each routing iterations, coupling coefficients cij are computed using softmax; For each of the secondary capsules, the computed weighted sum of predictions is given by where ⊕ is the concatenation of scalar and vector parts. Normalization is then applied using a squashing function to ensure that the vectors length lie between 0 and 1, represents the strength of the capsule, a nonlinear squashing function is applied; The logits are then updated based on the agreement between the prediction vectors and the activation. Attention Finally, to produce the property prediction, the graph capsule outputs are further routed into final output capsules. We introduce an attention-based aggregation mechanism to weigh the contribution corresponding to the target properties. This attention mechanism serves as an invariant weighting scheme for selectively emphasizing relevant capsules. The attention network operates on the scalar part of each capsule; that is the sm,c=ym,c;D2. A linear layer computes unnormalized attention scores for each capsules and the resulting scores are normalized using SoftMax. For a capsule c in a batch m, the attention weigh αm,c is given as; The final graph level representation is the weighted sum of the secondary capsules. The full dynamic routing algorithm is provided in the supplementary information. Results We first evaluate the CGN-e3 framework on a curated dataset of about 19K unique crystal structures from the Materials Project database.Our dataset curation strategy closely follows the principles established by Xie and Grossman in [10]. Specifically, all structures were sourced from the Materials Project database and subjected to equivalent filtering criteria, that is, inclusion of only DFT converged, ordered, and fully relaxed structures with complete crystallographic and energetic metadata, and selection of the most stable polymorph for each of the composition. This filtering pipeline also ensured data completeness by ensuring availability of all target properties. We limited the datasize to 19k, due to the computational complexity of the model. While the curated 19k subset preserves the essential statistical characteristics of the target properties (Supplementary Figure 7), the sub-sampling used here carries a risk of introducing potential bias. The model achieves a MAE of 0.054 eV/atom for formation energy prediction and 0.379 eV for bandgap prediction, demonstrating high quantitative accuracy across both continuous property tasks. To further assess the framework’s adaptability to discrete properties, the regression head was replaced with a binary classification layer optimized via a cross-entropy loss. On the Matbench benchmark dataset containing 2,083 crystals, CGN-e3 achieved an overall classification accuracy of 87 % in distinguishing metallic from non-metallic systems. The parity plots for formation energy and bandgap prediction, together with the receiver operating characteristic (ROC) curve for the classification task, are illustrated in Fig. 2i–iii while the t-SNE visualizations of the capsule embeddings Fig. 2iv–vi reveal distinct clustering patterns that correlate with material properties. Parity plots for; i Formation energy, ii Bandgap and iii ROC curve for metal classification The quantitative t-SNE visualization of the learned capsule embeddings across the three tasks is given by; iv Formation energy v band gap, and vi metal classification. The embeddings reveal distinct clustering behavior that correlates with target properties. vii is a heatmap that shows the pairwise correlation between the activation magnitudes of each of the capsules across a sampled dataset, the dendrogram in viii shows the clustering of capsules depending on how similar their activation patterns are while the bar plot at the left bottom shows the correlation between capsules activation and the target property To benchmark the predictive performance of CGN-e3, we compared CGN-e3 against two established and widely adopted graph-based baselines, CGCNN [10] and MEGNet [15], on our dataset of 19k crystals. All models were trained and evaluated on an identical splits. As summarized in Table 1, CGN-e3 achieved the lowest MAE for formation energy, surpassing CGCNN and matching the accuracy of the more parameter intensive MEGNet architecture. For bandgap prediction, CGN-e3 consistently outperformed both baselines, exhibiting improved generalization while maintaining architectural simplicity and interpretability. Table 1Mean absolute error (MAE) and coefficient of determination R2 comparison across formation energy and bandgap prediction tasks.ModelFormation energyBandgap (eV)MAE R2 MAE R2 CGCNN0.0620.9800.3990.883MEGNET0.0540.9920.3710.885CGN-E30.0540.9850.3790.887E(3)-GNN0.0680.9720.4010.870 Mean absolute error (MAE) and coefficient of determination R2 comparison across formation energy and bandgap prediction tasks. To quantify the contribution of the capsule network’s hierarchical routing mechanism, we performed an ablation study. We compared our full CGN-e3 model against an E(3)-GNN baseline that uses the same equivariant message-passing layers but replaces the capsule network with a simple invariant mean-pooling operation for graph-level representation. As shown in Table 1, the full CGN-e3 model consistently outperforms the ablated variant across both property prediction tasks. For instance, on formation energy prediction, the capsule routing provides a 26% reduction in MAE. This result quantitatively demonstrates the significance of capsule enhanced aggregation of local motifs into graph-level motifs, which is not only central to the model’s interpretability but also a significant contributor to its predictive accuracy. The heatmap in Fig. 2vii shows the pairwise correlation between the activation magnitudes of each of the six capsules across a sampled dataset. Higher values mean that capsules tend to be activated together for the same samples, suggesting they often detect related that motifs that frequently co-occur or different aspects of the same structural motifs that frequently co-occur in the crystals such as overlapping TiO6 and LiO6 environments. While the low values insinuate that the capsules are rarely or inversely activated together indicating they capture distinct, motifs that do not commonly appear together in the crystal structure or competing features. The dendrogram in Fig. 2viii visualizes how similar the activation patterns of different capsules are, clustering those with similar patterns together. Capsules that merge at lower shorter distance are more similar in their activation profiles across the dataset. The bar plot and scatter plots in Fig. 2ix display the correlation between capsules activation and the target property, linking these motifs to the property of interest and in this case the formation energy, highlighting which motifs are more or less predictive for the task. Capsule 3 at -0.46 has a strong negative correlation and is likely detecting motifs that are stabilizing or lowering the formation energy. Capsules with weak or near zero correlation like capsule 0 at 0.012 is less directly linked to the property detecting common features. Table 2 below highlights CGN-e3’s ability to quantify the motif contributions across diverse material chemistries. The consistent high importance of oxygen-coordinated motifs underscores their dominant role in thermodynamic stability. This quantitative motif importance provides design principles for materials optimization by identifying which structural features most significantly influence formation energy. Table 2Motif assignment and importance scores for prediction of formation energy across different materialsMaterial IDCapsuleImportanceInterpreted MotifDFT calculated(ev/atom)Predicted (ev/atom)mp-55680600.5054Si, 1O -3.0020 -3.0031 BaNa2(SiO3)240.1883Si, 2Ba50.1493O, 2Bamp-101977400.4951Si, 4Ce -3.2603 -3.2411 K2CeAl(SiO3)440.2031Al, 4Ce50.1371O, 4Cemp-55685800.4965P -1.6894 -1.5222 Al2P3H9(CO3)340.2225Al50.1725Omp-3686400.4992Ti, 3O -2.9770 -2.9856 Li2TiO340.2103Li, 2 Ti50.1462Ti, 3O Motif assignment and importance scores for prediction of formation energy across different materials Interpretation and prediction based on capsule representation We provide analyses at capsule levels, offering insight into the model’s internal reasoning and learned representations. To quantify the contribution of each of the atoms to the overall predictions, a saliency based gradient attribution technique identifies which node asserts more influence on a target property. For a case study, we take mp-36864, Li2TiO3, a compound with two in-equivalent Li1+ sites and two other similar compounds shown in Fig. 3. Li2TiO3 crystallizes in the monoclinic C2/c space group characterized by two in-equivalent Li+ sites, both of which are coordinated by six O2 atoms forming the LiO6 octahedra with each octahedron sharing corners with two equivalent TiO6 octahedra as well as with other LiO6 and TiO6 octahedra via edges. The Ti4+ sites are each coordinated by O forming TiO6 units that connect via edges and corners both LiO6 and other TiO6 octahedra. Structural comparison of three chemically related compounds. The crystal structures of mp-36864, mp-760017, and mp-4156 are visualized, highlighting their structural similarity (87.13% and 82.03%, respectively). All three structures are characterized by a network of corner and edge sharing octahedra. In mp-36864 and mp-760017, the framework is built from TiO6 octahedra (blue polyhedra), while in (c), the analogous role is played by ZrO6 octahedra (green polyhedra). The saliency maps from CGN-e3 (Fig. 4) demonstrate that the model consistently identifies these transition-metal-oxygen octahedra as the dominant motif for formation energy prediction across all three crystals, showcasing its ability to generalize structural roles despite the elemental substitution of Ti to Zr The three most important capsules (0, 4, and 5) when predicting the formation energy of Li2TiO3 are highlighted by saliency maps in Fig. 4, each with its motif and relative importance to the formation energy prediction. Each map corresponds to a capsule, a group of neurons, highlighting the most influential atomic motifs and elements for the prediction of formation energy. The saliency maps for Capsules 0 and 5 highlight motifs centered on Titanium and Oxygen atoms, reflecting the dominance and central role of TiO6 octahedra in the stability and formation energy of Li2TiO3. This is strongly supported by both experimental crystallography and DFT calculations, which consistently show that the TiO6 framework is the backbone of the structure and a key contributor to its thermodynamic stability [31, 32, 33]. The vibrational modes and lattice energies are dominated by Ti–O interactions, as observed in DFT calculations [31]. Capsule 4 identifies a motif with 3 Lithium and 2 Titanium atoms, indicating Li’s influence on the TiO6 network. Theoretical studies confirm that Lithium occupies distinct octahedral sites and that its distribution can subtly affect the distortion and connectivity of TiO6 octahedra, impacting the overall formation energy [32]. DFT calculations also distinguish multiple in-equivalent Li sites, each with unique local environments, consistent with our saliency map’s identification of Li-rich motifs. Atom level saliency maps for formation energy for three closely related crystals, mp-36864, mp-760017 and mp-4156 highlighting the patterns in discovered motifs and their contribution and importance towards the prediction of formation energy. For mp-760017 which is 87.13 percent similar to mp-36864, a similar pattern in detected motifs is observed to accurately predict the formation energy as -2.8890ev/atom compared to the DFT calculated at -2.8261ev/atom. Similarly, for mp-4156 which is 82.03 percent similar to mp-36864 has the most important capsule for prediction of formation energy as the motif ZrO6 octahedra which is analogous to the TiO6 units in Li2TiO3 crystals and confirm that Zr-O connectivity is the dominant motif for formation energy. CGN-e3 predicts the formation energy of mp-4156 as -2.9935ev/atom against -3.0402ev/atom as calculated by DFT. In these three crystals, the most important capsules consistently highlight the transition metals Ti or Zr and oxygen atoms forming the XO6 octahedra. This reflects the central role of these polyhedral in determining the structural stability and formation energy. Capsule 4, in both materials, captures motifs with a higher proportion of Li atoms, underscoring Li’s secondary but significant influence on the octahedral network. Li modifies the connectivity and local distortion of the XO6 units, which can affect the overall lattice energies. The spatial arrangement and corner/edge sharing of octahedra are encoded in the specific atoms highlighted by each capsule, indicating that the network is sensitive to subtle differences in motif connectivity, even between highly similar structures. The saliency maps show that the model adapts to the substitution of Ti by Zr, with similar motif types like the XO6 octahedra and Li-rich environments but different central atoms. This demonstrates the model’s ability to generalize structural motifs across related chemistry. The importance values are very close in the three crystals, suggesting that the energetic landscape is governed by similar structural features, with minor shifts due to the change from Ti to Zr. Bandgap prediction Transition metals and electronegative atoms like oxygen or nickel have partially filled d orbitals which can overlap with orbitals from other atoms. This leads to the formation of new molecular orbitals that extend into bands in solids. These orbitals affect the bandgap and electronic conductivity. For instance, in Nickel based compounds such as mp-78118, mp-764496 and mp-770357, Nickle and Oxygen anchored motifs consistently rank higher in importance when predicting the bandgap of these materials. As shown in Fig. 5, the saliency maps for these materials highlight the top 3 capsules capturing elements whose combinations contribute largely to bandgap prediction. This confirms the Ni-O hybridization, common in Ni 3d and O 2p govern the electronic structure hence the projected bandgap properties. Capsule 6’s importance is, repeatedly high, highlighting the model’s learned structural-electronic dependency on this motif. The detailed importance scores and motif assignments for these materials are provided in Table 3. To assess how the model’s performance is affected by slight changes in atomic positions, we introduce disturbance in atomic coordinates of the atoms using zero mean gaussian noise at multiple scales to stimulate the structural distortions. We then observe the changes in the predicted values of the predicted values. CGN-e3 demonstrates robust invariance to slight positional disorders with predictions remaining stable at ±0.05ev for up to 1.0 Angstroms when predicting band gaps. The saliency maps here highlight the top 3 capsules, capturing a set of elements whose combination contribute largely to the prediction of Bandgap by CGN-e3. The graphs on far right of each material gives the variation in predicted bandgap under perturbations of up to 1.0 Angstroms Table 3Motif assignment and importance scores for Bandgap prediction for materials highlighted in Fig. 4Material IDCapsuleImportanceInterpreted MotifDFT calculated(ev)Predicted (ev)mp-78011860.4634Ni, 1O0.86441.0030Na4Ni2O540.3083Na, 2Ni00.1235Nimp-76449660.5401O, 4Ni1.63241.5238Na14Ni2O940.1844Na, 1Ni00.1361O, 4Nimp-77035760.5221O, 4Ni1.88221.5346Li2AlNiO440.2464Ni, 1Al00.1131O, 4Ni Motif assignment and importance scores for Bandgap prediction for materials highlighted in Fig. 4 Discussion While machine learning is transforming the material science and chemistry domain, this transformation is constrained by the challenges in accurate representation of the complex spatial hierarchies and structural motifs intrinsic in crystalline materials as discussed above. Despite traditional GNNs showing promise in modeling crystal graphs, their reliance on scalar node embeddings often fail to capture the hierarchical spatial relationships and symmetry-equivariant features inherent in crystalline materials. Recent advances have sought to address these limitations. Models like CGCNN, SchNet, and GATGNN incorporate local atomic environments, while ALIGNN and M3GNet extend this by explicitly encoding bond angles and directional information. However, these models still fall short in two fundamental areas. First, these GNNs fail to preserve full geometric and symmetry information. By failing to preserve this information, the models do not enforce strict equivariance to 3D transformations limiting their ability to model physical symmetries and ability to generalize across diverse crystal systems. The second challenge is the inability of GNNs to explicitly capture the hierarchical relationships within crystals. Crystal structures are built from repeating units that are arranged in specific orientations and positions relative to each other and can exhibit different chemical characteristics depending on the chemical space captured and therefore, are crucial for determining global properties. Traditional GNNs, however, treat nodes and edges in isolation and lack mechanisms to model how motifs relate. This limitation has been shown to impact predictive performance. For instance, [13] highlight that GNNs often fail to distinguish between materials with similar local structures but different global arrangements such as structural polymorphs. Similarly, [18] and others have noted that scalar message passing in traditional GNNs is insufficient to differentiate between structures of isomers like graphene and carbon nanotubes which a locally similar but globally distinct underscoring the need for models that can capture higher order geometric and symmetry information. E(3)-equivariant GNNs, such as PaiNN, MACE, and Allegro, in an attempt to solve the first mentioned challenge have advanced the field by embedding Euclidean symmetries directly into their architectures. These models ensure that predictions transform accordingly under 3D transformations, thus preserving physical consistency. However, even these approaches lack explicit mechanisms for learning hierarchical and how local motifs aggregate into larger structures which are critical for understanding of the underlying chemistry of materials. The CGN-e3 framework represents a paradigm shift in materials representation learning by unifying geometric symmetry principles with hierarchical feature learning. Our results demonstrate that the model achieves competitive predictive accuracy (MAE of 0.054 eV/atom for formation energy, 0.379 eV for bandgap, and 87% accuracy for metal classification) while fundamentally advancing interpretability in materials informatics. More importantly, the capsule representations reveal how the local atomic environments hierarchically assemble into functional motifs that dictate material properties a capability absent in existing equivariant GNNs. The consistent motif discovery across diverse material systems validates CGN-e3’s ability to identify physically meaningful structural patterns. In Li2TiO3 and its structural analogs, the model quantitatively identifies TiO6/ZrO6 octahedra as the dominant contributors to formation energy (49.9% importance), while simultaneously capturing Li’s secondary influence through Li-rich motifs (21.0% importance). Similarly, in nickel-based compounds, the model independently discovers Ni-O hybridization as the key electronic structure descriptor across different chemical contexts. This unsupervised motif discovery, combined with quantitative importance scoring, provides a powerful tool for understanding structure–property relationships beyond human intuition. In a crystallography, local symmetry means that equivalent atomic arrangements appear in different locations and orientations as characterized by mp-556858 shown in Fig. 6. For mp-556858, Al2P3H9(CO3)3 the Al3+ and P5+ ions exhibit two distinct coordination environments which define the local symmetry equivariant to symmetry operations of rotation translation and reflection. The first Al3+ forms an AlO6 octahedra occupying the Wyckoff 2b position and shares corners with PCO3 tetrahedra. The second Al3+ tetrahedra occupies the 6c Wyckoff position bonded to 4 oxygen atoms forming the AlO4 tetrahedra. The symmetry analysis in Fig. 6 exemplifies CGN-e3’s ability to recognize and preserve crystallographic symmetries while maintaining predictive accuracy. In mp-556858, the model automatically distinguishes between PCO3 tetrahedra and AlO4 tetrahedra while respecting their equivalent positions under space group operations, with these motifs contributing 49.6% and 22.1% to formation energy prediction respectively. This symmetry-aware motif discovery is crucial for materials design, as it ensures that learned representations adhere to crystallographic constraints while maintaining physical interpretability. Local symmetry in mp-556858, Al2P3H9(CO3)3 The hierarchical routing mechanism demonstrates how local atomic features assemble into polyhedral units and eventually into extended structural frameworks, mirroring the natural crystal growth process. This multi-level organization enables the model to capture complex relationships such as how corner-sharing versus edge-sharing of octahedra influence stability. The robust performance under geometric perturbations, +0.05 eV stability up to 1.0 Å displacements, further validates that the learned representations capture fundamental structural principles rather than memorizing specific coordinates. Most significantly, the symmetry aware capsule representations provide a foundation for generative materials discovery. The learned motifs encoded as transferable, symmetry-aware building blocks with quantified property contributions can be reassembled to propose novel crystal structures with desired characteristics. This addresses a critical challenge in the inverse material design, that is, creating physically realistic structures that obey crystallographic principles while optimizing target properties based on validated motif-property relationships. Despite these strengths, the CGN-e3 framework has several limitations that bring forth opportunities for future work. The primary constraint is its computational complexity. The dynamic routing mechanism and equivariant tensor operations introduce significant overhead compared to simpler GNNs, making it less suitable for ultra large-scale screening of millions of structures in its current implementation. Furthermore, the model’s interpretability is inherently tied to the quality of the structural data. It identifies motifs based on geometric proximity and learned patterns, but cannot currently incorporate explicit chemical knowledge, such as known oxidation states or thermodynamic stability rules, which could further refine the discovered motifs. Finally, while the capsule representations are highly interpretable, their ’language’ is still a learned latent space. A more direct mapping between specific capsule activations and standardized crystallographic terminology (e.g., ’corner-sharing octahedra’) would make the insights more immediately actionable for materials scientists. Future work will focus on developing more efficient routing algorithms and integrating external chemical knowledge bases to address these limitations. CGN-e3 opens a number of exciting research directions including; extending the hierarchical framework to dynamic properties and phase transitions, incorporating the quantified motif importance for guided materials design, and developing generative models that leverage the learned capsule vocabulary for de-novo materials discovery. By achieving both competitive predictive performance and unprecedented interpretability, this work establishes a new path toward intelligent materials design systems that combine human crystallographic knowledge with machine learning scalability. Conclusion Our work introduces CGN-e3 framework, a symmetry aware, hierarchical representation for crystalline materials that offer an interpretable approach to materials property prediction. By integrating E(3)-equivariant message passing with capsule networks, CGN-e3 uniquely captures both local and global symmetry reflecting the multilevel organization present in crystals. This architecture uniquely captures both the geometric symmetries and hierarchical organization inherent in crystalline materials, moving beyond black-box prediction to provide mechanistic insights into structure–property relationships. Through quantitative motif importance scoring, we have demonstrated CGN-e3’s ability to discover and rank structural features from TiO6 octahedra governing formation energy to Ni-O hybridization dictating bandgap behavior, providing a new paradigm for interpretable materials informatics. Beyond property prediction, CGN-e3 establishes a foundation for accelerated materials discovery by encoding crystals into a space of symmetry-aware, transferable motifs. This capsule-based representation not only explains the model’s predictions but also opens a new path for inverse design, where learned building blocks can be reassembled to propose novel functional materials. Our results on three structurally similar compounds, mp-36864, mp76017 and mp-4156 demonstrate the models ability to interpret and quantify the effects of elemental substitution. The saliency maps reveal that the same symmetry aware motifs dominate the property landscape across the three crystals, while the model sensitively adjusts attributions to reflect changes in chemistry and connectivity. This gives promise to the potential of CGN-e3 new crystals design by elemental substitution. By bridging the gap between geometric deep learning and hierarchical feature learning, CGN-e3 provides a powerful method for decoding the complex nature of crystal structures and accelerating the design of next-generation materials. Supplementary Information Supplementary material 1. Author contributions Eddah K Sure conceived the idea, wrote the main manuscript and code associated with this work. Xing Wu and Quan Qian did the work flow plan andreviewed the manuscript. Data availability The data files for this work are available at: https://github.com/Eddah-Sure/CrystalCaps Declarations Competing interests The authors declare no Competing interests. Footnotes Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Eddah K. Sure, Xing Wu and Quan Qian have contributed equally to this work. References Associated Data Supplementary Materials Supplementary material 1. Data Availability Statement The data files for this work are available at: https://github.com/Eddah-Sure/CrystalCaps