Chi-Geometry: A Library for Benchmarking Chirality Prediction of GNNs

Rylie Weaver, Massamiliano Lupo Pasini · 2025

Read Chi-Geometry: A Library for Benchmarking Chirality Prediction of GNNs by Rylie Weaver, Massamiliano Lupo Pasini (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-09097.txt.

Introduction

Handed molecules trip up graph neural networks, and Rylie Weaver and Massamiliano Lupo Pasini build a test kit for that skill in a 2025 paper. Their library generates synthetic molecular graphs with set shape traits and shuffled atoms, so stray clues cannot leak the answer. Each graph holds exactly one handed center, marked left or right.

Networks must flag that center as a sorting task. Clean benchmarks showed where top architectures fail, which guided the design of two new ones. Chirality decides drug safety, so a fair yardstick for machine chemists matters before models screen real compounds.

Catalog record

CREATORRylie Weaver, Massamiliano Lupo Pasini
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-09097

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Chi-Geometry: A Library for Benchmarking Chirality Prediction of GNNs Rylie Weaver, Massamiliano Lupo Pasini Abstract We introduce Chi-Geometry – a library that generates graph data for testing and benchmarking GNNs’ ability to predict chirality. Chi-Geometry generates synthetic graph samples with (i) user-specified geometric and topological traits to isolate certain types of samples and (ii) randomized node positions and species to minimize extraneous correlations. Each generated graph contains exactly one chiral center labeled either R or S, while all other nodes are labeled N/A (non-chiral). The generated samples are then combined into a cohesive dataset that can be used to assess a GNN’s ability to predict chirality as a node classification task. Chi-Geometry allows more interpretable and less confounding benchmarking of GNNs for prediction of chirality in the graph samples which can guide the design of new GNN architectures with improved predictive performance. We illustrate Chi-Geometry’s efficacy by using it to generate synthetic datasets for benchmarking various state-of-the-art (SOTA) GNN architectures. The conclusions of these benchmarking results guided our design of two new GNN architectures. The first GNN architecture established all-to-all connections in the graph to accurately predict chirality across all challenging configurations where previously tested SOTA models failed, but at a computational cost (both for training and inference) that grows quadratically with the number of graph nodes. The second GNN architecture avoids all-to-all connections by introducing a virtual node in the original graph structure of the data, which restores the linear scaling of training and inference computational cost with respect to the number of nodes in the graph, while still ensuring competitive accuracy in detecting chirality with respect to SOTA GNN architectures. 1 Introduction Materials and molecular structures naturally map onto graphs, where atoms are represented as nodes and interatomic bonds as edges. Because of this natural mapping, graph neural networks reduce the cumbersome and tedious processes of input feature engineering, which is essential for other types of Deep Learning models, making them the tool of choice for predicting a wide variety of properties at the atomic scale. By predicting scientifically relevant structural and atomic properties, graph neural networks facilitate research and development in chemistry and materials science. As a result, they have important downstream applications in healthcare (drug design), technology (battery materials, semiconductors), sustainability (biodegradable materials), infrastructure (composites, alloys), and agriculture (fertilizers, pesticides). Various material and molecular properties strongly depend on chirality. Three examples with important practical applications are: (i) binding affinity ( Morehead and Cheng 2022 ; Gaiński et al. 2023 ) , (ii) electronic circular dichroism ( Li et al. 2025 ) , and (iii) toxicity ( Cremer et al. 2023 ) . To accurately predict these properties, it is crucial for graph neural networks to correctly capture the chirality of an atomic structure. High-quality benchmarks are critical to guide the development of new, robust graph neural network models because they allow the research community to identify shortcomings of existing graph neural network architectures and propose new graph neural network architectures to overcome them. However, current datasets for benchmarking chirality prediction have serious limitations. Firstly, there is no straightforward way to isolate data samples with geometric and topological traits that are relevant to a graph neural network’s predictive ability. Isolating relevant traits, such as the hop distance between the nodes that determine a structure’s chirality, enables targeted testing and thereby more interpretable results for analyses of graph neural network prediction failures. Secondly, current benchmark datasets contain extraneous correlations that graph neural networks may inadvertently use when making predictions. This may produce misleading results where a graph neural network records high performance on the benchmark without learning the function of chirality. Traits that may have extraneous correlations with the benchmarking data are chemical composition, atomic numbers, and interatomic distances/angles. To address these limitations, we developed Chi-Geometry, a library that generates graphs for testing graph neural networks’ ability to predict chirality. Chi-Geometry enables targeted testing by generating datasets composed exclusively of samples with user-specified geometric and topological traits, while minimizing extraneous correlations by randomizing node species and positions. To avoid the injection of extraneous correlations in the data, Chi-Geometry generates synthetic graph samples without taking chemical constraints into consideration. As a result, the edges of the generated graph samples do not have chemical meaning and therefore cannot be easily deduced from nodes attributes (e.g., atom types and number of valence electrons for each atom). In contrast, material and molecular graphs typically feature chemically meaningful bonds that are ‘‘learnable’’ from atomic properties. This distinction is crucial for benchmarking graph neural network architectures that establish all-to-all connections in the graph (e.g., SE(3)-transformer 1 1 1 Since there is a mathematical equivalence between the self-attention mechanism in Transformers and all-to-all message passing in GNNs ( El et al. 2025 ) , Transformers fall within our broader investigation of GNN architectures. and global E3NN). In fact, when these models are used on chemical data, they can implicitly reconstruct the original connectivity of the graph samples using the chemically meaningful nodal features. However, such retrievability cannot be performed when the graph samples are synthetically generated by Chi-Geometry because the input graph samples do not consider the chemistry. Therefore, graph neural network architectures that establish all-to-all connections in the graph must explicitly keep track of the original graph topology to succeed in benchmarking tests where data is synthetically generated with the Chi-Geometry library. We illustrate the effectiveness of Chi-Geometry by benchmarking the ability of several widely used state-of-the-art graph neural networks to model chirality. In doing so, we find that Chi-Geometry eliminates a previously observed confounding result (Section 4.1 ) and yields more interpretable results, which facilitates root-cause-analyses of why state-of-the-art graph neural networks fail to predict chirality (Section 4 ). Based on the shortcomings identified, we propose two new graph neural network architectures. Both are equivariant to reflections 2 2 2 Although invariance is formally a special case of equivariance (see Appendix C ), we use “equivariant” to concisely refer to models that are equivariant but not invariant. . The first graph neural network architecture proposed uses global (all-to-all) connections and engineered edge features that preserve the original graph topology, which attains high accuracy across challenging configurations where previous models failed. Its ability to learn long-range interactions while including global connections is reasonably expected, since global connections artificially convert all long-range interactions into direct, short-range ones. However, its computational cost scales quadratically with the number of nodes, which is undesirable for large graph structures. To mitigate the computational cost while preserving the capability to model long-range interactions, inspired by existing work ( Okabe et al. 2024 ; Sestak et al. 2024 ; Zhang et al. 2024 ; Xia et al. 2025 ) , we introduce a second graph neural network architecture that replaces global connections with the inclusion of a virtual node in the original graph, which is then connected to all the other nodes to accelerate exchange of information across the graph topology. While incorporating a virtual node partially sacrifices accuracy compared to the globally connected model, it maintains linear computational scaling. Notably, it is the only graph neural network we test that is able to detect long-range chirality with better-than-random accuracy while scaling linearly. To the best of our knowledge, this is

Read the complete text