Vision Transformers for End-to-End Quark-Gluon Jet Classification from Calorimeter Images
Md Abrar Jahin, Shahriar Soudeep, Arian Rahman Aditta, M. F. Mridha 0, Nafiz Fahad, Md. Jakir Hossen 0 · 2025
Read Vision Transformers for End-to-End Quark-Gluon Jet Classification from Calorimeter Images by Md Abrar Jahin, Shahriar Soudeep, Arian Rahman Aditta, M. F. Mridha 0, Nafiz Fahad, Md. Jakir Hossen 0 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-14934.txt.
Introduction
Md Abrar Jahin and five co-authors test whether vision transformers can tell quark jets from gluon jets in collider data better than the convolutional networks usually used. The distinction matters for hunting new physics at the Large Hadron Collider, but the two jet types look frustratingly alike. The team built multi-channel images from simulated detector data, combining energy deposits in two calorimeter layers with reconstructed particle tracks.
Across their benchmarks, transformer models and hybrids that marry transformers to modern convolutional blocks beat the established baselines on accuracy, F1 score, and ranking quality. The advantage, they argue, comes from attention's ability to link distant parts of a jet's spray, which local filters miss. The tests used public 2012 CMS open data under realistic detector and pileup conditions. The authors offer the study as the first systematic baseline for transformer jet tagging, with a structured dataset for others to build on.
Catalog record
| CREATOR | Md Abrar Jahin, Shahriar Soudeep, Arian Rahman Aditta, M. F. Mridha 0, Nafiz Fahad, Md. Jakir Hossen 0 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-14934
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Vision Transformers for End-to-End Quark-Gluon Jet Classification from Calorimeter Images Md Abrar Jahin, Shahriar Soudeep, Arian Rahman Aditta, M. F. Mridha 0, Nafiz Fahad, Md. Jakir Hossen 0 Abstract Distinguishing between quark- and gluon-initiated jets is a critical and challenging task in high-energy physics, pivotal for improving new physics searches and precision measurements at the Large Hadron Collider. While deep learning, particularly Convolutional Neural Networks (CNNs), has advanced jet tagging using image-based representations, the potential of Vision Transformer (ViT) architectures, renowned for modeling global contextual information, remains largely underexplored for direct calorimeter image analysis, especially under realistic detector and pileup conditions. This paper presents a systematic evaluation of ViTs and ViT-CNN hybrid models for quark-gluon jet classification using simulated 2012 CMS Open Data . We construct multi-channel jet-view images from detector-level energy deposits (ECAL, HCAL) and reconstructed tracks, enabling an end-to-end learning approach. Our comprehensive benchmarking demonstrates that ViT-based models, notably ViT+MaxViT and ViT+ConvNeXt hybrids, consistently outperform established CNN baselines in F1-score, ROC-AUC, and accuracy, highlighting the advantage of capturing long-range spatial correlations within jet substructure. This work establishes the first systematic framework and robust performance baselines for applying ViT architectures to calorimeter image-based jet classification using public collider data, alongside a structured dataset suitable for further deep learning research in this domain. The implementation of our code is available at: https://github.com/Abrar2652/particle_reconstruction/ . 1 Introduction The analysis of hadronic jets, collimated sprays of particles produced by the hadronization of quarks and gluons, is central to precision measurements and new physics searches at the CERN Large Hadron Collider (LHC) [ 23 ] . Among the various jet classification problems, distinguishing between quark- and gluon-initiated jets remains particularly important and challenging. Many signatures of Standard Model (SM) processes and beyond-the-SM scenarios involve final states with quark or gluon jets, and accurate identification of their origin can directly improve background rejection and signal sensitivity in collider analyses [ 23 ] . Quark-gluon discrimination is difficult because both types of jets emerge from QCD interactions and share overlapping kinematic and topological features. The difference lies in subtle variations in their radiation patterns, color factors, and particle multiplicities. Gluon jets typically produce higher particle multiplicity, broader radiation patterns, and softer transverse momentum spectra due to their larger color charge [ 23 ] . Capturing these subtle differences requires classifiers capable of learning fine-grained spatial and structural patterns in the detector data, often in the presence of pileup contamination and detector noise. Recent advances in machine learning (ML), particularly deep learning (DL), have transformed jet tagging by enabling algorithms to learn directly from low-level detector or reconstructed particle data, bypassing hand-engineered observables [ 29 ] . At the CMS experiment, conventional approaches for particle and jet identification have historically relied on complex reconstruction algorithms that combine information from various sub-detectors [ 30 ] . However, recent studies have demonstrated that applying convolutional neural networks (CNNs) directly to detector-level images can yield competitive classification performance, without relying on the full reconstruction chain, through an end-to-end learning paradigm [ 2 , 1 ] . While CNNs operating on jet images and particle cloud networks have achieved remarkable success, they often rely on local receptive fields and predefined hierarchical structures, limiting their ability to capture long-range correlations within a jet’s spatial energy deposit profile. These global contextual dependencies can be crucial for separating quark and gluon jets, especially in high-pileup environments. In contrast, transformer-based architectures, which have demonstrated state-of-the-art performance in natural language processing [ 34 ] and computer vision [ 11 , 24 ] , offer a compelling alternative. Vision Transformers (ViTs) [ 11 ] and their hierarchical variants, such as Swin Transformer [ 24 ] and MaxViT [ 33 ] , have shown strong capabilities in modeling global image dependencies without relying on inductive biases inherent to CNNs. Their attention-based mechanism allows the model to dynamically focus on important spatial regions in the input, which is highly desirable for interpreting sparse and irregular energy patterns in jet images. Despite their success in other domains, the application of ViT-based models to calorimeter image data for jet classification remains largely unexplored. Existing studies in high-energy physics (HEP) predominantly focus on particle cloud representations [ 29 , 17 ] , leaving a gap in evaluating transformer-based models directly on image-based representations of jet substructure. Furthermore, publicly available, realistic datasets like the CMS Open Data provide an ideal testbed to assess these models under conditions closely matching those of operational collider experiments. In this study, we present a systematic evaluation of ViTs and ViT-CNN hybrid architectures for quark-gluon jet classification using simulated 2012 CMS Open Data [ 1 ] . We construct jet-view images incorporating energy deposits from the ECAL, HCAL, and reconstructed tracking systems, forming multi-channel images that preserve the spatial distribution of detector signals within a fixed angular region. This end-to-end approach, free from particle reconstruction algorithms, provides the classifier direct access to rich detector-level data in all its complexity [ 2 , 1 ] , offering an opportunity to test ViTs as an alternative to CNNs for calorimeter image analysis at the LHC. Our work benchmarks the performance of ViT-based models against competitive CNN baselines and investigates their sensitivity to key training hyperparameters through an extensive ablation analysis. Our key contributions are as follows: (1) We propose the first systematic framework applying ViTs and ViT-CNN hybrid models to calorimeter image-based quark-gluon jet classification using CMS Open Data. (2) We construct a structured calorimeter image dataset from simulated 2012 CMS Open Data , incorporating realistic detector and pileup conditions, suitable for evaluating deep learning models for jet classification tasks. (3) We conduct an extensive performance evaluation and hyperparameter sensitivity analysis of ViT-based and CNN-based models, establishing new baselines for transformer architectures in end-to-end detector image-based jet classification tasks. 2 Related Work 2.1 Deep Learning for Jet Tagging DL has transformed jet tagging at the LHC, achieving unprecedented accuracy by leveraging various data representations. Early approaches included 2D CNNs for calorimeter images [ 9 ] , sequential models for particle lists [ 16 ] , recursive networks [ 26 ] , and graph neural networks [ 18 ] . The field has since shifted toward particle cloud representations inspired by computer vision point cloud methods [ 29 ] , notably through the Energy Flow Network (EFN) [ 21 ] and ParticleNet [ 29 ] , both built on the Dynamic Graph CNN (DGCNN) framework [ 35 ] . While subsequent works explored GAPNet-like [ 3 ] and transformer-based point cloud models [ 27 ] , none have decisively surpassed ParticleNet on public benchmarks. Parallel efforts introduced physically motivated inductive biases, including the Lund jet plane [ 12 ] , Lorentz-equivariant networks [ 19 , 15 ] , and rotationally invariant models [ 10 ] . These advances improved robustness, data efficiency, and interpretability, with a notable impact on LHC analyses. The CMS DeepAK8 algorithm [ 31 ] and ParticleNet have enabled key results, including the first observation of Z\rightarrow c\bar{c} decays [ 7 ] and rare Higgs process searches [ 6 ] . 2.2 Transformer Architectures in Scientific Applications Transformers, introduced in NLP by Vaswani et al. [ 34 ] and extended to vision via ViT [ 11 ] and