When Molecular Similarity Works: Property Cliffs Reveal Hidden Errors
2026
Read When Molecular Similarity Works: Property Cliffs Reveal Hidden Errors (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-17265.txt.
Introduction
Hu and colleagues expose a blind spot in molecular property prediction: models can look strong overall yet fail where similar molecules differ sharply in behavior, known as property cliffs. Their CliffSplit protocol builds test cases that stress those cliff zones, while CliffLoss, a training-only fix, shrinks the cliff-to-smooth error gap. Tests show cliff zones carry at least 15 percent higher error, and the fix cuts the gap by up to 30 percent.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54433 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.17265
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
When Molecular Similarity Works: Property Cliffs Reveal Hidden Errors Abstract Accurate prediction of molecular properties underpins drug discovery and material design, yet even state-of-the-art models remain vulnerable to localized failure modes that aggregate metrics cannot detect. The places where molecular similarity should be most helpful are also places where standard evaluation can be most misleading. Property cliffs expose this gap: structurally similar molecules can still differ sharply in target property, so models with competitive overall performance may fail in high-risk local neighborhoods. To expose and mitigate this failure mode, CliffSplit, a cliff-aware evaluation protocol that constructs locally supported, cliff-exposed test cases, and CliffLoss, a model-agnostic train-only mitigation mechanism for cliff-sensitive errors, are introduced. Experiments on three QM9 targets and three MoleculeNet tasks across five backbones show that CliffSplit reveals at least 15% higher error in cliff-heavy QM9 regions, while CliffLoss reduces the cliff-to-smooth error gap by up to 30% on Lipophilicity and improves overall MAE by 9.7%. Together, these results turn molecular similarity failure from a descriptive anomaly into a benchmarked evaluation problem for molecular machine learning. The code is available at https://anonymous.4open.science/r/Cliff_Loss . 1 Introduction Drug discovery [ 23 , 47 , 45 ] and molecular design [ 5 , 24 , 21 ] address a central human need: finding compounds with desired properties under a massive search space and high experimental cost. Molecular representation learning has become a core capability here [ 20 ] , with graph neural networks [ 44 , 43 ] , geometric predictors [ 40 , 25 , 35 ] , and foundation models [ 30 , 48 ] advancing benchmark performance. Yet model quality is still judged mainly through aggregate metrics, so overall accuracy is often treated as a proxy for reliability. In practice, however, reliability-sensitive tasks such as lead optimization demand that predictions remain trustworthy not only on average but also locally [ 22 ] ; a molecule predicted accurately in aggregate may still fail precisely where confidence should be highest [ 19 ] . This tension is sharpened by property cliffs. The similar property principle [ 36 , 15 ] underlies most molecular predictors: nearby structures should yield nearby values. Property cliffs violate this expectation sharply, as structurally similar molecules can differ dramatically in target property [ 27 , 10 , 38 , 13 ] . The challenge is not merely that some molecules are hard to predict; rather, the hardest failures arise inside apparently well-supported local neighborhoods, where standard evaluation still appears reassuring. A model can achieve low overall error while systematically mispredicting at cliff edges, because aggregate metrics smooth over these localized discontinuities. Existing evaluation protocols are not designed to expose this regime. Random split tends to colocate near neighbors across train and test, so cliff-prone regions receive overlapping support and local failure remains hidden: when both ends of a cliff edge are present in training, the model learns a locally smooth fit that appears correct on each molecule individually, and the test set offers no isolated cliff endpoint to expose the discontinuity. Scaffold split tests structural extrapolation at the scaffold level [ 4 , 41 ] , but does not target local property inconsistency under structural support; it separates molecules by chemical skeleton rather than by property gap, so molecules within the same scaffold often share similar property values while true cliff pairs may fall entirely on the same side of the split boundary or into unrelated scaffolds, yielding no explicit cross-split cliff exposure. Figure 1 illustrates this contrast: random split hides cliff errors by placing structurally similar neighbors on both sides of the train/test divide, whereas scaffold split groups molecules by structure rather than by property discontinuity. Neither paradigm constructs cross-boundary cliff exposure as an explicit objective, motivating the need for a dedicated cliff-aware evaluation protocol. Figure 1: Motivation: cliff exposure across split paradigms. Existing splits differ fundamentally in how they treat cross-split cliff exposure, motivating the need for a dedicated cliff-aware evaluation protocol. To address this challenge, CliffSplit and CliffLoss, a benchmark-and-mitigation framework for property-cliff reliability in molecular property prediction, are proposed. CliffSplit constructs locally supported yet cliff-exposed train/test partitions to reveal a reliability axis hidden by aggregate metrics, while CliffLoss provides a model-agnostic train-only mitigation mechanism that reweights regression errors by offline cliff severity with adaptive validation feedback. Experiments on three QM9 targets and three MoleculeNet tasks across five backbones show that cliff-heavy QM9 regions incur at least 15% higher error than smooth regions, and that CliffLoss reduces the cliff-to-smooth error gap by up to 30% on Lipophilicity while improving overall MAE by 9.7%. Overall, our contributions can be summarized: (i) a cliff-aware evaluation protocol, CliffSplit, is proposed to construct cliff-exposed train/test partitions and evaluate models via severity-conditioned single-molecule error, exposing a reliability axis invisible to aggregate metrics; (ii) a model-agnostic loss plugin, CliffLoss, is introduced to reweight regression errors by offline cliff severity with adaptive control, reducing cliff-region degradation without architectural modification; (iii) Experiments on QM9 and MoleculeNet across five backbones show that CliffSplit exposes systematic severity-dependent degradation, with at least 15% higher error in cliff-heavy QM9 regions, while CliffLoss reduces the Lipophilicity gap by up to 30% without sacrificing overall accuracy. 2 Related Work The similar property principle [ 36 ] has long guided chemoinformatics and continues to shape modern molecular predictors, which inherit an implicit smoothness bias [ 15 , 9 ] . Activity cliffs, sharp local discontinuities where structurally similar molecules exhibit large property differences, have been extensively documented in bioactivity landscapes, and ML models have been shown to be least accurate precisely in these regions [ 27 , 10 , 38 , 13 , 7 , 37 ] . That literature established the chemical reality of cliff-prone error, but its main focus remains cliff detection and SAR interpretation rather than benchmark design for general molecular property prediction. This shift makes evaluation protocols central. Modern molecular property prediction spans diverse model families [ 42 , 35 , 40 , 3 , 33 , 30 , 48 ] and standard benchmarks such as QM9 and MoleculeNet have standardized evaluation across quantum-chemical and experimental properties [ 31 , 41 , 4 ] . Yet dominant practice still relies on MAE- or RMSE-style pointwise summaries under random or scaffold splits, which do not directly test whether error concentrates in high-similarity neighborhoods where the similarity principle should be most informative. The missing component is therefore not another model family, but a released protocol that exposes cliff-heavy local neighborhoods across the train/test boundary. Once the problem is framed as reliability rather than average accuracy, it connects to robustness and hard-case learning, including example reweighting, long-tail aware methods, hard-region reweighting, distributionally robust optimization, invariant learning, predictive uncertainty, out-of-distribution (OOD) and graph OOD benchmarks, calibration, and relation-aware objectives [ 32 , 28 , 26 , 34 , 2 , 8 , 18 , 17 , 12 , 11 , 14 , 46 , 24 ] . These directions are relevant, but they largely assume that difficult regions are already identified. What remains underdeveloped is the benchmark layer itself: a protocol that constructs cliff-exposed test regions, induces molecule-level severity from local cliff structure, and evaluates standard single-molecule predictors by severity-conditioned error rather than aggregate accuracy alone. 3 Cliff Benchmark