IAFormer: Interaction-Aware Transformer network for collider data analysis

W. Esmail, A. Hammad, M. Nojiri · 2025

Read IAFormer: Interaction-Aware Transformer network for collider data analysis by W. Esmail, A. Hammad, M. Nojiri (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-03258.txt.

Introduction

W Esmail, A Hammad, and M Nojiri present a 2025 paper on IAFormer, a leaner transformer network for sorting particle collision data. Detectors record each collision as a spray of particles, and finding the interesting events means modelling how particles relate. Earlier particle transformers do this well but carry heavy costs in size and compute.

IAFormer's attention is steered by fixed, boost-proof measures of each particle pair, which slashes the parameter count. A second trick, sparse differential attention, lets the network spend its effort on the informative particles and skip the rest. The result matches the best known accuracy while running more than ten times cheaper, the authors report.

Catalog record

CREATORW. Esmail, A. Hammad, M. Nojiri
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-03258

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

IAFormer: Interaction-Aware Transformer network for collider data analysis W. Esmail, A. Hammad, M. Nojiri Abstract In this paper, we introduce IAFormer, a novel Transformer-based architecture that efficiently integrates pairwise particle interactions through a dynamic sparse attention mechanism. The IAformer has two new mechanisms within the model. First, the attention matrix depends on predefined boost invariant pairwise quantities, reducing the network parameter significantly from the original particle transformer models. Second, IAformer incorporate the sparse attention mechanism by utilizing the “differential attention”, so that it can dynamically prioritizes relevant particle tokens while reducing computational overhead associated with less informative ones. This approach significantly lowers the model complexity without compromising performance. Despite being computationally efficient by more than an order of magnitude than the Particle Transformer network, IAFormer achieves state-of-the-art performance in classification tasks on the Top and quark-gluon datasets. Furthermore, we employ AI interpretability techniques, verifying that the model effectively captures physically meaningful information layer by layer through its sparse attention mechanism, building an efficient network output that is resistant to statistical fluctuations. IAformer highlights the need to sparse attention in any Transformer analysis to reduce the network size while improving its performance. 1 Introduction Analyzing LHC collision data to uncover intriguing physics phenomena, selecting an appropriate jet reconstruction approach and tagging algorithm is essential, as they must align with the expected characteristics of the final state. For scenarios involving the production of a high momentum, hadronically decaying heavy particle such as a W, Z, or a top quark, a large radius jet is often used to capture entire particles arising from the heavy particle decay. The recognition of fat jets with the multi-prong structure was first noted in Ref. [ 1 ] , where jet substructure techniques emerged as a powerful method for suppressing QCD backgrounds, thereby advancing Beyond Standard Model (BSM) searches at the LHC. The pronged structure associated with different heavy resonant particles is dictated by the final state products of their decays. Over time, jet substructure methodologies have evolved, leading to Deep Learning (DL) based classification techniques for tagging heavy particles [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ] . These approaches are now widely applied in LHC analyses conducted by ATLAS and CMS [ 22 , 23 , 24 , 25 , 26 , 27 ] . They are also employed in scenarios with multiple heavy particle final states [ 28 , 29 , 30 ] and in extracting fundamental parameters of the theory [ 31 , 32 , 33 , 34 , 35 ] . In contrast, processes that generate spatially isolated quarks or gluons are typically reconstructed using one or more small radius jets. This is followed by flavor tagging, a technique that identifies the type of initiating parton, whether an up, down, strange, charm, or bottom quark, or a gluon. Bottom and charm hadrons (primarily B and D hadrons) have relatively long lifetimes, allowing them to travel a short distance within the detector before decaying. Their decays produce displaced tracks originating from a secondary vertex, distinct from the primary collision point. By leveraging machine-learning techniques, this vertex displacement information, combined with subsequent decay properties, significantly improves tagging efficiency [ 36 , 37 , 38 , 39 ] . Additionally, studies have explored the potential for strange-quark identification using ML techniques [ 40 , 41 , 42 ] . Since gluons belong to the octet representation of QCD SU(3) symmetry, their jet characteristics differ from those of quarks, which are in the triplet representation. Consequently, ML-based methods have also been developed to distinguish gluon-initiated jets from quark-initiated ones [ 43 , 44 , 45 , 46 , 47 , 48 ] . Over the past few decades, significant advancements have been made in jet identification techniques at hadron colliders. In particular, DL methods have transformed how we analyze and utilize jet-related observables. Treating a jet as a single entity provides limited information about its origin, whereas a more detailed analysis of its internal structure and constituent particles offers valuable insights into the nature of the initiating particle. This has led to the development of various techniques collectively referred to as jet tagging, which aim to classify jets based on their underlying physics. Recently, Transformer networks have demonstrated significant improvements in jet tagging tasks [ 49 , 50 , 21 ] . In their standard architecture, Transformers process jet constituents along with their corresponding features, which include kinematic information for each individual hadron inside the jet. In this setup, the network assigns weights to each particle based on its feature importance in the overall classification decision. However, incorporating features derived from particle-pair interactions can further enhance network performance, as these features play a crucial role in capturing the radiation pattern inside the jet [ 10 ] . Efforts to integrate pairwise interactions into the attention mechanism have involved either using an interaction matrix as a bias for the computed attention scores [ 49 ] or replacing the Query and Key matrices with the interaction matrix itself [ 50 ] . In both approaches, the number of attention heads is constrained by the number of features in the interaction matrix. Additionally, the interaction matrix remains fixed across attention heads and is injected into each attention layer without updates. This rigid structure can disrupt the learning of attention patterns across different layers, potentially limiting the network overall effectiveness. IAFormer addresses these challenges by allowing the interaction matrix to be updated across different attention heads and Transformer layers. Furthermore, the interaction matrix is refined through skip connections and propagated to later attention layers. Another key innovation in IAFormer lies in its ability to dynamically focus on the most important particles while suppressing attention to less relevant ones. This is achieved through a dynamic sparse attention mechanism, implemented as the difference between two copies of the learned interaction matrix [ 51 ] . By encouraging the network to learn distinct representations for these copies, IAFormer prioritizes essential hadrons while de-emphasizing soft radiation, which often exhibits similar patterns in both signal and background. The IAFormer architecture demonstrates superior performance compared to previously introduced Transformer-based models when evaluated on top tagging and quark-gluon tagging tasks, achieving this with a significantly reduced network size. Although the network is validated against state-of-the-art jet tagging models, IAFormer is designed as a general framework that can be adapted for various classification tasks. The implementation is publicly available at IAFormer GitHub , and detailed usage instructions are provided in the appendix. This paper is organized as follows; Section 2 discusses various Transformer network architectures, including the structure of IAFormer. Section 3 presents validation results, comparing IAFormer with existing networks for top tagging and quark-gluon tagging tasks. In Section 4, we employ different interpretability methods to analyze the learned attention patterns in IAFormer during the top tagging task. Section 5 provides a discussion and conclusion. 2 Transformers for jet tagging Transformers were originally introduced for natural language processing, particularly for sequence-to-sequence tasks like machine translation [ 52 ] , which were later modified for collider

Read the complete text