Self-Supervised Learning Strategies for Jet Physics

Patrick Rieck, Kyle Cranmer, Etienne Dreyer, Eilam Gross, Nilotpal Kakati, Dmitrii Kobylanskii, Garrett W. Merz, Nathalie Soybelman · 2025

Read Self-Supervised Learning Strategies for Jet Physics by Patrick Rieck, Kyle Cranmer, Etienne Dreyer, Eilam Gross, Nilotpal Kakati, Dmitrii Kobylanskii, Garrett W. Merz, Nathalie Soybelman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-11632.txt.

Introduction

Patrick Rieck and co-authors teach jet representations without labels in this 2025 paper. Jets are sprays of particles born in collider crashes, and labeled data for them is costly. Their trick reruns the standard simulation chain with a shared start and split endings.

Two sim runs share the hard crash and the first shower steps. Later stages, such as particle forming and detector hits, run apart for each. A contrast score then pulls the paired jets' codes close. That forces the code to hold the physics of the shared early stages. The team grades the codes on later tasks to test their worth.

Catalog record

CREATORPatrick Rieck, Kyle Cranmer, Etienne Dreyer, Eilam Gross, Nilotpal Kakati, Dmitrii Kobylanskii, Garrett W. Merz, Nathalie Soybelman
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-11632

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Self-Supervised Learning Strategies for Jet Physics Patrick Rieck, Kyle Cranmer, Etienne Dreyer, Eilam Gross, Nilotpal Kakati, Dmitrii Kobylanskii, Garrett W. Merz, Nathalie Soybelman Abstract We extend the re-simulation-based self-supervised learning approach to learning representations of hadronic jets in colliders by exploiting the Markov property of the standard simulation chain. Instead of masking, cropping, or other forms of data augmentation, this approach simulates pairs of events where the initial portion of the simulation is shared, but the subsequent stages of the simulation evolve independently. When paired with a contrastive loss function, this naturally leads to representations that capture the physics in the initial stages of the simulation. In particular, we force the hard scattering and parton shower to be shared and let the hadronization and interaction with the detector evolve independently. We then evaluate the utility of these representations on downstream tasks. March 2025 1 Motivation The bulk of machine learning in high-energy physics (HEP) is based on a supervised learning paradigm and large simulated datasets where labels are derived from the knowledge of the underlying simulation. Although this approach is successful for a wide variety of tasks, unsupervised and self-supervised learning paradigms offer promising ways to leverage large, unlabeled datasets and create versatile representations that can be used for multiple downstream tasks. Self-supervised learning (SSL) approaches, have seen success in a wide variety of applications including computer vision and natural language processing [ 1 ] . In science, the SSL approach has proved useful in several fields, including chemistry [ 2 ] , climate science [ 3 ] , and astrophysics [ 4 ] . In HEP, the first explorations of SSL have aimed to derive generic representations of jets that serve as a starting point for various downstream tasks [ 5 ] . In this paper, we provide new insights into the benefits of SSL for the study of jets, moving this approach closer to applications for particle collider data. In particular, we extend prior work in re-simulation-based self-supervised learning (RS3L) [ 6 ] . This work moves us closer towards the goal of a foundation model of jet physics. Most approaches to SSL utilize some notion of data augmentation. Often a single training example is converted into multiple examples that are considered to be intrinsically the same, and the differences between these examples are effectively irrelevant. This notion of ’sameness’ may either be an ad hoc heuristic or motivated by some prior knowledge. Examples include randomly cropping an image or transforming a training example according to a symmetry group that is assumed to be present. In the case of contrastive learning, two data points comprise a “positive pair” if they share a common origin and differ only with respect to a randomized augmentation. In the re-simulation-based approach introduced in [ 6 ] , and extended in this paper, the Markov structure of a stochastic simulator provides the notion of sameness. The loss functions encourage the network to find representations such that positive pairs of data points are mapped to nearby points in the representation space. Negative pairs of jets, on the other hand, are mapped to points in representation space far apart from each other. In consequence, the resulting neural network, called the backbone or foundation model, which serves as the basis of further unsupervised or supervised downstream tasks, encodes the physics of the jets as it is defined by the underlying notion of sameness. The Markov structure of the simulation chain used in HEP, provides a well-motivated and powerful notion of sameness for SSL. The simulation chain is factorized into stages that are based on different phenomena separated by the appropriate energy scales; evolving from high-energy (TeV) to nuclear (GeV), and atomic (MeV) energy scales. Each step in this simulation chain offers a way to define the sameness of a pair of jets, requiring equality of the evolution up to the step of interest, while later steps of the evolution develop independently for the two jets. A first approach along these lines was implemented in [ 6 ] where the sameness of jets is defined based on the underlying, primary partons. This paper explores an alternative notion of sameness, requiring the sameness of jets down to and including the parton shower step. In this work, we extend and explore re-simulation-based SSL strategies for HEP and aim to increase similarity of these studies to real physics analyses occurring at collider experiments such as ATLAS and CMS. Our new developments are: • A new definition of the sameness of jets underlying the SSL approach, such that the physics of parton showers is incorporated into the learned representation • A realistic detector simulation to study jets emerging from proton-proton collisions. Using state-of-the-art simulations of interactions between particles and detector material, we take into account the details of the detector response to jets which make jet physics challenging. • An extended phase space spanning the full coverage of the detector and a range of transverse momenta around 100 GeV, which covers the bulk of jets under investigation at CERN’s Large Hadron Collider (LHC) • A transformer architecture representing the state-of-the-art models used in HEP experiments [ 7 , 8 ] • Two downstream, supervised learning tasks, namely the classification of quark jets as opposed to gluon jets, and the identification of tau leptons decaying into hadrons • A comparison with a self-supervised pre-training approach other than contrastive learning, namely an autoencoder • A demonstration of how to leverage a large dataset by means of self-supervised learning to detect anomalous jets 2 Related Work Early work on the applications of SSL for high-energy physics focused on the incorporation of physical symmetries into representations of jets [ 5 ] . These representations were derived based on contrastive learning, using the NT-Xent loss function [ 1 ] , which we also adopt. In [ 5 ] , the usefulness of the jet representations based on SSL was demonstrated in terms of classification tasks, discriminating between jets emerging from pure strong interactions on the one hand, and top-quark jets on the other hand. In Refs. [ 9 , 10 ] , techniques to identify signatures of physics beyond the Standard Model (SM) of elementary particles were explored, covering a broad range of the new physics parameter space. In Ref. [ 11 ] , large, unlabelled datasets were exploited for means of jet classification. In Ref. [ 6 ] , the authors introduced re-simulation-based self-supervised learning (RS3L), which we were independently investigating. In this approach, the pairs of training examples used in the contrastive learning objective are based on a notion of sameness that is motivated by the structure of the simulation chain. In particular, they create pairs of events where the parton shower and detector simulation vary, but the parton-level description of the event is the same. We build on this approach by intervening in the simulation after the parton shower and at the hadronization stage, and we use a more realistic Geant4 -based detector simulation. This leads to representations that capture more details of the jet substructure that are useful for a wide range of downstream tasks. 3 Dataset \left\{\begin{array}[]{c}\\ \end{array}\right. \left\{\begin{array}[]{c}\\ \end{array}\right. \left\{\begin{array}[]{c}\\ \end{array}\right. \left\{\begin{array}[]{c}\\ \end{array}\right. \hskip-34.44434pt\begin{array}[]{ccc}z_{\mathrm{Hard\,Scatter}}&&z_{\mathrm{Parton}}\end{array}\hskip 34.44434pt\begin{array}[]{ccc}z_{\mathrm{Hadron}}^{1}&&x_{\mathrm{Detector}}^{1}\\[4.30554pt] z_{\mathrm{Hadron}}^{2}&&x_{\mathrm{Detector}}^{2}\end{array} p(x_{\mathrm{Detector}})=\int\mathrm{d}z_{\mathrm{Hard\,Scatter}}\mathrm{d}z_{\mathrm{Parton}}\mathrm{d}z_{\mathrm{Hadron}}\,p(z_{\mathrm{Hard\,Scatter}},z_{\mathrm{Parton}},z_{\mathrm{Hadron}},x_{\mathrm{Detector}})\ , p(z_{\mathrm{Hard\,Scatter}},z_{\mathrm{Parton}},z_{\mathrm{Hadron}},x_{\mathrm{Detector}})= p(x_{\mathrm{Detector}}|z_{\mathrm{Hadron}})p(z_{\mathrm{Hadron}}|z_{\mathrm{Parton}})p(z_{\mathrm{Parton}}|z_{\mathrm{Hard\,Scatter}})p(z_{\mathrm{Hard\,Scatter}})

Read the complete text