AutoSciDACT: Automated Scientific Discovery through Contrastive Embedding and Hypothesis Testing
Samuel Bright-Thonney, Christina Reissel, Gaia Grosso, Nathaniel Woodward, Katya Govorkova, Andrzej Novak, Sang Eon Park, Eric Moreno, Philip Harris · 2025
Read AutoSciDACT: Automated Scientific Discovery through Contrastive Embedding and Hypothesis Testing by Samuel Bright-Thonney, Christina Reissel, Gaia Grosso, Nathaniel Woodward, Katya Govorkova, Andrzej Novak, Sang Eon Park, Eric Moreno, Philip Harris (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21935.txt.
Introduction
Samuel Bright-Thonney leads a nine-author team building a pipeline that hunts for genuinely new phenomena in scientific data without crying wolf. Experiments produce noisy, high-dimensional data, and most anomaly hunters flag oddities without the statistical backing a discovery claim demands.
Their system, AutoSciDACT, first compresses data into compact representations using contrastive training, leaning on the plentiful simulations many fields already have and on smart transformations guided by expert knowledge. A sensitive two-sample test, built on machine learning, then judges whether real data departs from the expected pattern strongly enough to matter. The authors present the pipeline as a first step toward novelty detection whose outputs can support quantified discovery statements rather than hunches, across scientific domains.
Catalog record
| CREATOR | Samuel Bright-Thonney, Christina Reissel, Gaia Grosso, Nathaniel Woodward, Katya Govorkova, Andrzej Novak, Sang Eon Park, Eric Moreno, Philip Harris |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-21935
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AutoSciDACT: Automated Scientific Discovery through Contrastive Embedding and Hypothesis Testing Samuel Bright-Thonney, Christina Reissel, Gaia Grosso, Nathaniel Woodward, Katya Govorkova, Andrzej Novak, Sang Eon Park, Eric Moreno, Philip Harris Abstract Novelty detection in large scientific datasets faces two key challenges: the noisy and high-dimensional nature of experimental data, and the necessity of making statistically robust statements about any observed outliers. While there is a wealth of literature on anomaly detection via dimensionality reduction, most methods do not produce outputs compatible with quantifiable claims of scientific discovery. In this work we directly address these challenges, presenting the first step towards a unified pipeline for novelty detection adapted for the rigorous statistical demands of science. We introduce AutoSciDACT (Automated Scientific Discovery with Anomalous Contrastive Testing), a general-purpose pipeline for detecting novelty in scientific data. AutoSciDACT begins by creating expressive low-dimensional data representations using a contrastive pre-training, leveraging the abundance of high-quality simulated data in many scientific domains alongside expertise that can guide principled data augmentation strategies. These compact embeddings then enable an extremely sensitive machine learning-based two-sample test using the New Physics Learning Machine (NPLM) framework, which identifies and statistically quantifies deviations in observed data relative to a reference distribution (null hypothesis). We perform experiments across a range of astronomical, physical, biological, image, and synthetic datasets, demonstrating strong sensitivity to small injections of anomalous data across all domains. 1 Introduction Scientific discovery is often characterized by serendipity: an unexpected observation turns out to have a profound impact on a field, leading to rapid progress or discovery. Today’s data-rich scientific landscape is potentially brimming with curious or unexplained observations, but the scale and complexity of available data increasingly obscures genuine novelties behind statistical noise or incidental fluctuations. As scientific datasets continue to grow, so too grows the challenge of uncovering meaningful unexplained phenomena. The scientific method traditionally provides a structured framework for discovery, encompassing observation, inquiry, research, hypothesis formulation, experimentation, and conclusion (top row of Fig. 1 ). Effective implementation of this method relies on human intuition and domain expertise to identify relevant observables and devise meaningful experiments. However, given the magnitude of modern datasets, the process would significantly benefit from automated tools to efficiently identify the most promising regions for discovery. To address this challenge, it is essential to develop methods that intelligently prioritize informative regions—areas where genuine scientific surprises are most likely to emerge. Traditional feature-engineering approaches are human-driven and domain-specific, limiting scalability and generalizability. Recent advances utilizing agentic AI systems and large language models can partially automate aspects of scientific inquiry [ 1 , 2 , 3 ] , yet still lack integrated frameworks capable of rigorous, automated hypothesis testing and validation. We introduce AutoSciDACT (Automated Scientific Discovery with Anomalous Contrastive Testing), a pipeline that parallels the scientific method and streamlines scientific inquiry by automating key steps of the scientific discovery process. AutoSciDACT streamlines the phases of data reduction, hypothesis formulation, and statistical testing (bottom row of Fig. 1 ) by deploying contrastive learning together with the New Physics Learning Machine (NPLM) [ 4 ] . Contrastive learning is used to reduce raw, high-dimensional datasets into expressive low-dimensional feature embeddings while NPLM provides a statistically robust mechanism for identifying and quantifying novel structures within this learned embedding space. A key insight with AutoSciDACT is the effective, automated incorporation of domain expertise as a tool to reduce the dimensionality to a small number of well-behaved features, making it possible to construct a robust statistical model. Using NPLM, our pipeline systematically compares incoming data with reference distributions of known (background) data, finding the most anomalous regions and quantifying their statistical significance. We validate our approach using synthetic benchmarks and real-world datasets from astronomy, physics and biomedical domains. AutoSciDACT reliably detects meaningful novelties while remaining robust against spurious variations. Our results demonstrate the promise of combining structured contrastive learning methods with automated statistical hypothesis testing to accelerate scientific discoveries. Contributions Our main contributions can be summarized as follows: • An end-to-end pipeline for novelty discovery in scientific datasets that is readily transferable across domains. • A principled procedure for incorporating scientific simulations, hand-labeled data, and expert knowledge into a contrastive dimensionality reduction pipeline. • A statistically rigorous framework for quantifying the significance of observed anomalies, beyond simply flagging anomalous datapoints. • A realistic demonstration of novelty detection in four disparate scientific domains. Figure 1 : Illustration of the scientific method (top row) and the AutoSciDACT pipeline (bottom row), emphasizing the corresponding methodological steps implemented within AutoSciDACT. 2 Related Work Contrastive Learning Contrastive learning is a powerful tool for learning expressive, low-dimensional data representations. Self-supervised methods such as SimCLR [ 5 ] , MoCo [ 6 ] , VICReg [ 7 ] , and Barlow Twins [ 8 ] use data augmentations (e.g., blurring, cropping) to promote semantically meaningful and well-separated embeddings. Supervised contrastive learning [ 9 ] uses class labels to define positive pairs, more efficiently capturing semantic relationships between datapoints but requiring labeled datasets to train. Its applications have expanded to structured data [ 10 ] , multimodal inputs [ 11 ] , and domain-specific tasks [ 12 ] , demonstrating broad utility. Contrastive Anomaly Detection Several existing methods leverage contrastive embeddings to search for out-of-distribution (OOD) data, but primarily focus on identifying individual anomalous instances for e.g. industrial applications. In contrast, AutoSciDACT (our method) is tailored for scientific contexts and makes statistical statements about the presence of anomalous data, identifying distribution-level deviations relative to a baseline expectation (null hypothesis). Existing approaches include Refs. [ 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ] , all of which rely primarily on the AUROC for identifying OOD points as a figure of merit. They do not attempt to statistically quantify observations of OOD data, as is required in the scientific context. CADet [ 13 ] is nearest to our setup in using a two-sample test, but focuses again on AUROC. In scientific contexts, various combinations of contrastive learning and anomaly detection have been used for domain-specific applications – e.g. in astronomy [ 22 , 23 ] , histology, [ 24 , 25 ] and particle physics [ 26 , 27 ] – but, to our knowledge, no unified approaches have been proposed. Hypothesis Testing for Anomaly Detection Traditional goodness-of-fit (GOF) tests are powerful in univariate settings but struggle beyond that due to the curse of dimensionality. Simple multivariate extensions have limiting factors. The Mahalanobis distance-based test [ 28 ] , for instance, assumes Gaussianity and is sensitive to the choice of covariance estimation, limiting its applicability in complex data regimes. Recent machine learning-based methods have introduced model-agnostic, data-driven alternatives to enable non-parametric, highly adaptable high-dimensional tests. To quantify distributional differences, Maximum Mean Discrepancy (MMD) [ 29 , 30 , 31 ] embeds distributions into a reproducing kernel Hilbert space, while the Classifier Two-Sample Test (C2ST) [ 32 ] uses a trained classifier’s accuracy. Other methods include density-ratio estimation [ 33 ] and generative modeling frameworks that assess sample likelihoods [ 34 ] . In this work, we draw from statistical anomaly detection tests developed for high-energy physics, where sensitivity to subtle deviations is crucial. Autoencoders [ 35 , 36 , 37 , 38 ,