When should we trust the annotation? Selective prediction for molecular structure retrieval from mass spectra

Mira Jürgens, Gaetan De Waele, Morteza Rakhshaninejad, Willem Waegeman · 2026

Read When should we trust the annotation? Selective prediction for molecular structure retrieval from mass spectra by Mira Jürgens, Gaetan De Waele, Morteza Rakhshaninejad, Willem Waegeman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10950.txt.

Introduction

Mira Jürgens and coauthors ask when a machine's guess should be trusted. The guess is at a molecule's structure from its mass spectrum. They also ask when it should be held back for review. Their selective prediction framework sorts annotations by risk. It compares uncertainty scores at three levels: input, fingerprint, and retrieval. The tests use a standard benchmark. Retrieval-level uncertainty proves the strongest rejection signal. Cheap first-order confidence scores remain solid baselines. Distribution-free risk control lets users set a tolerable error rate in advance.

Catalog record

CREATORMira Jürgens, Gaetan De Waele, Morteza Rakhshaninejad, Willem Waegeman
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51932
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.10950

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

When should we trust the annotation? Selective prediction for molecular structure retrieval from mass spectra Mira Jürgens, Gaetan De Waele, Morteza Rakhshaninejad, Willem Waegeman Abstract Machine learning methods for identifying molecular structures from tandem mass spectra (MS/MS) have advanced rapidly, yet current approaches still exhibit significant error rates. In high-stakes applications such as clinical metabolomics and environmental screening, incorrect annotations can have serious consequences, making it essential to determine when a prediction can be trusted. We introduce a selective prediction framework for molecular structure retrieval from MS/MS spectra, enabling models to abstain from predictions when uncertainty is too high. We formulate the problem within the risk-coverage tradeoff framework and comprehensively evaluate uncertainty quantification strategies at two levels of granularity: fingerprint-level uncertainty over predicted molecular fingerprint bits, and retrieval-level uncertainty over candidate rankings. We compare scoring functions including first-order confidence measures, aleatoric and epistemic uncertainty estimates from second-order distributions, as well as distance-based measures in the latent space. All experiments are conducted on the MassSpecGym benchmark. Our analysis reveals that while fingerprint-level uncertainty scores are poor proxies for retrieval success, computationally inexpensive first-order confidence measures and retrieval-level aleatoric uncertainty achieve strong risk-coverage tradeoffs across evaluation settings. We demonstrate that by applying distribution-free risk control via generalization bounds, practitioners can specify a tolerable error rate and obtain a subset of annotations satisfying that constraint with high probability. Scientific contribution : This work provides the first systematic evaluation of selective prediction for molecular structure retrieval from mass spectra. We demonstrate that computationally inexpensive retrieval-level confidence measures outperform Bayesian epistemic uncertainty estimates for identifying reliable annotations, and that distribution-free risk control yields provable guarantees on annotation quality, transforming molecular identification into an uncertainty-aware decision process. keywords metabolomics, molecular retrieval, uncertainty quantification, selective prediction, risk control † † footnotetext: Preprint. Under review. 1 Introduction Figure 1 : Overview of the selective prediction framework for molecular structure retrieval from tandem mass spectra. (a) Fingerprint-based molecular retrieval. A tandem mass spectrum \boldsymbol{x} is mapped by a model f to a vector of bitwise probabilities \boldsymbol{\theta}\in[0,1]^{4096} . Each candidate fingerprint \boldsymbol{c}_{j} in the instance-specific candidate set \mathcal{C}_{i} is scored by cosine similarity s_{j}=\mathrm{sim}(\boldsymbol{\theta},\mathbf{c}_{j}) , and candidates are ranked by descending score. (b) Sources of uncertainty in the learned representation space. Aleatoric uncertainty is due to inherent noise and ambiguity in the data, and can arise for structurally similar molecules. Epistemic uncertainty reflects a general lack of knowledge, and arises in regions of the representation space that are far from training data. (c) Selective prediction. Left: test spectra are sorted by confidence in descending order. A threshold \tau partitions spectra into accepted and rejected predictions. Right: sweeping the threshold \tau over its full range traces the risk-coverage curve, where different points on the curve correspond to different risk-coverage trade-offs. Untargeted metabolomics generates vast numbers of tandem mass spectra (MS/MS), yet only approximately 10\% of detected features can be annotated with molecular structures 7 ; 23 , a deficit also known as dark matter of metabolomics 12 ; 6 . This difficulty stems from different sources, including a fundamental mismatch between the vast chemical space and the limited coverage of spectral reference libraries, the inherent complexity of molecular fragmentation, and the variability of spectra across instrument settings and ionization conditions. The correct identification of the molecular structure from a mass spectrum is hence considered a very challenging task, while being of fundamental importance with applications in drug discovery 2 ; 37 ; 46 , environmental biochemistry 39 ; 27 and clinical diagnostics 38 . A growing body of literature uses machine learning methods to tackle this problem, where two main paradigms are formed by molecular retrieval 17 ; 51 ; 22 ; 31 and de novo structure elucidation 47 ; 10 ; 8 ; 24 . The retrieval paradigm, which is the focus of this work, predicts a molecular representation from the query spectrum and ranks candidate structures from a molecular database via a similarity function. The classical approach within this paradigm is molecular fingerprint prediction 17 ; 51 ; 22 , where a model learns to map spectra to binary fingerprint vectors encoding the presence or absence of molecular substructures 42 ; 4 . More recently, joint embedding approaches avoid explicit fingerprint prediction and instead learn a shared latent space in which spectra and molecules can be compared directly via cosine similarity 31 ; 52 . The pace of progress in retrieval-based identification has been substantial. CSI:FingerID 17 trains an array of support vector machines, one per molecular property, using multiple kernel learning over fragmentation tree similarities to predict molecular fingerprints from MS/MS spectra. It was the first method to substantially outperform in silico fragmentation tools such as MetFrag 43 and CFM-ID 3 , and became the core of the widely adopted SIRIUS framework 16 . MIST 22 subsequently introduced a transformer architecture that encodes MS/MS peaks as chemical formula vectors rather than binned m/z values, thereby implicitly featurizing pairwise neutral losses between fragments. The introduction of the MassSpecGym benchmark 9 , comprising 231,104 labeled spectra with structure-disjoint evaluation splits based on molecular similarity, provided a standardized basis for comparing retrieval methods. Joint embedding approaches, which embed molecules and spectra in a shared latent space and rank candidates by cosine similarity rather than by comparing explicit fingerprint reconstructions, have since demonstrated substantial improvements. JESTR 31 treats molecules and their spectra as two views of the same underlying object and trains molecular and spectral encoders jointly using contrastive multiview coding 49 . GMLR 55 initializes the molecular encoder from ChemFormer 29 and jointly finetunes it during contrastive training. Despite this rapid progress, a significant error rate at the current state of the art remains consequential for downstream applications. In clinical metabolomics, incorrect identifications can delay or misdirect diagnosis of inborn errors of the metabolism 38 . In environmental nontarget screening, annotations directly inform regulatory decisions on chemical contamination 27 . These examples illustrate that predictive accuracy alone, without a mechanism to express the reliability of individual predictions, is insufficient for trustworthy metabolomics. To avoid making unreliable predictions, correct uncertainty quantification can help to build a trustworthy deployment of machine learning pipelines. Selective prediction 40 ; 11 ; 18 provides a powerful tool to reduce the error rate by abstaining from making a prediction when the uncertainty is too high. A selective classifier augments a prediction function with a selection function that determines, for each input, whether to predict or to abstain, thereby trading off coverage , i.e. the fraction of inputs on which the model commits, for lower risk on accepted predictions 18 ; 21 . The effectiveness of selective prediction depends on the scoring function that decides which predictions to trust. A natural source of such scores is predictive uncertainty, which can be decomposed into aleatoric uncertainty, arising from irreducible noise or ambiguity in the data, and epistemic uncertainty, reflecting the model’s lack of knowledge due to limited training data 28 ; 32 . In mass spectrometry, aleatoric uncertainty is inherent to the measurement process: structural isomers can produce nearly indistinguishable fragmentation patterns 53 , and spectra of the same compound vary with collision energy, instrument type, and acquisition conditions. Epistemic uncertainty on the other hand reflects limitations of

Read the complete text