Anja Butter

1,2 · 2024

Read Anja Butter by 1,2 (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-12074.txt.

Introduction

Anja Butter and five co-authors teach a transformer to improvise extra particles in a simulated particle jet, beyond the numbers it was trained on, in this 2024 paper. Simulating collisions at the Large Hadron Collider is a computing bottleneck: every event must be generated particle by particle. Generative networks promise speed, but fixed-size generators clash with physics, since real jets contain any number of particles.

Autoregressive transformers solve the count problem by emitting particles one at a time, like words in a sentence, stopping when the jet is complete. The team shows such a model learns the factorized probability structure of jet radiation, the physics rulebook for how jets sprout particles. With bootstrapped training data and a reshaped loss, the model extrapolates, generating higher particle counts than it ever saw. Fast, faithful, and flexible simulation is worth real computing budgets at CERN.

Catalog record

CREATOR1,2
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-12074

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Anja Butter 1,2 Abstract Extrapolating Jet Radiation with Autoregressive Transformers 1 Institut für Theoretische Physik, Universität Heidelberg, Germany 2 LPNHE, Sorbonne Université, Université Paris Cité, CNRS/IN2P3, Paris, France 3 Meta FAIR, CERMICS - Ecole des Ponts 4 Interdisciplinary Center for Scientific Computing (IWR), Universität Heidelberg, Germany August 24, 2026 Abstract Generative networks are an exciting tool for fast LHC event fixed number of particles. Autoregressive transformers allow us to generate events containing variable numbers of particles, very much in line with the physics of QCD jet radiation, and offer the possibility to generalize to higher multiplicities. We show how transformers can learn a factorized likelihood for jet radiation and extrapolate in terms of the number of generated jets. For this extrapolation, bootstrapping training data and training with modifications of the likelihood loss can be used. Contents 1 Introduction Modern LHC physics is defined by precision tests of the fundamental properties of particles and their interactions, both in and beyond the current Standard Model. The level of precision is continuously improved by experimental and theoretical progress, accompanied by but not limited to the rapidly increased LHC luminosity towards the high-luminosity LHC. In view of this precision program, experimental and theoretical LHC physics are being transformed through modern machine learning (ML) [ 15 , 97 ] . On the theory and simulation side, a range of neural network applications are improving every step of our first-principles simulation chain. This includes phase-space sampling [ 31 , 68 , 67 , 52 , 77 , 29 , 74 , 54 , 76 ] , scattering amplitude surrogates [ 27 , 13 , 10 , 90 , 109 , 14 , 88 , 104 , 89 , 33 , 34 ] , end-to-end event generation [ 73 , 55 , 45 , 3 , 44 ] , and detector simulators trained on full simulations [ 95 , 96 , 61 , 16 , 38 , 83 , 9 , 82 , 39 , 50 , 92 , 51 , 56 , 72 , 110 , 37 , 35 , 58 , 62 , 63 , 40 , 99 , 6 ] . The main workhorses behind this transformation are generative networks. They learn phase space densities or Jacobians from simple distributions given sets of events and can reproduce these densities through fast sampling. Modern generative network architectures are normalizing flows, diffusion networks, and autoregressive transformers. Because they are fast, differentiable, and flexible, generative networks can enable new simulation and analysis strategies. These networks are reaching new levels of accuracy and can significantly amplify simulated training data [ 41 , 23 ] and speed up the generation. Given the LHC requirements, they have to be controlled and precise in encoding kinematic patterns over an, essentially, interpretable phase space [ 42 , 108 , 93 , 86 , 53 ] . Conditional versions of the forward-generative networks allow for probabilistic unfolding [ 18 , 17 , 11 , 85 , 100 , 102 , 57 , 79 ] or inference through posterior sampling [ 24 , 43 , 75 ] . In this paper we tackle the physics problem of using generative networks to describe jets radiated from a hard scattering process. In fundamental QCD, jet radiation is described by successive probabilistic parton splittings. It is an integral part of QCD predictions for hadron colliders, where final states with a fixed number of jets are not in line with parton densities and collinear factorization [ 59 , 98 , 48 ] . The corresponding splitting kernels and the generated phase space correlations are approximately universal [ 105 ] . The generated number of jets follows well-defined patterns, also predicted by QCD. Autoregressive generative networks can, just like with language, generate open-end sequences of particles, or events with a variable number of particles. An autoregressive generation requires a factorized phase space probability [ 8 , 7 ] . This structure matches the QCD aspects of universal splittings and well-defined jet numbers. Our generative architecture of choice is an autoregressive transformer [ 64 , 44 ] . An attractive benefit of this approach is the possibility of exploiting universal structures across jet multiplicities, which could allow for a single generative network to be deployed instead of a collection of specialized models. In this work we establish the fundamental idea by studying extrapolation to higher multiplicities, highlighting both the challenges and the opportunities. The goal of this paper is to show, for the first time, that a generative transformer can extrapolate in the number of jets and generate approximately universal jet radiation for higher jet numbers than seen during the training. In Sec. 2 we describe the QCD structures motivating an approximately factorized phase space likelihood and its ML-realization, leading to our autoregressive generative transformer. We then present extrapolated predictions in the number of jets in Sec. 3 , using bootstrapped training data in Sec. 3.2 , a truncated loss without fixed stopping condition in Sec. 3.3 , and a loss that overrides the stop condition in Sec. 3.4 . In Appendix A we provide additional information on how to improve the accuracy of the generative transformer through including a classifier in the training, in the spirit of a GAN. 2 Autoregressive jet radiation Given that jet radiation in QCD is described by universal splitting kernels and well-defined scalings in the number of jets, we will train an autoregressive transformer with a factorized likelihood loss to generate QCD jet radiation. The ultimate goal is to show that the transformer not only describes jet radiation to a number of jets represented in the training data, but that it can extrapolate to larger jet counts than seen during training. We first remind ourselves of universal splittings in QCD and the typical scaling in the number of produced jets. We will then motivate our Z +jets dataset, exhibiting the universal so-called staircase scaling. To train a generative network we first derive a factorized phase space probability and then encode it in a loss function for an autoregressive transformer. 2.1 QCD jet radiation Collinear parton splittings in the initial or final states are the backbone of QCD predictions for hadron colliders. Their universal nature is the basis of parton densities, parton showers, and jet radiation, and it defines the structure of LHC events [ 59 , 98 , 48 ] . A challenging consequence of collinear splittings is that any hard scattering process is accompanied by a variable number of jets in the final state, as described by jet radiation and parton showers in the multi-purpose event generators [ 20 , 103 , 4 , 32 ] . Combining parton shower and hard matrix element predictions is the theory basis for the entire precision physics program at the LHC [ 49 , 91 , 65 , 30 ] . Universal autoregressive structure The physics background of our paper is the universal nature of jet radiation from collinear splittings, reflecting the collinear factorization of the matrix element and the phase space. It allows us to generate events with n+1 final-state jets from events with n final-state jets. For final state radiation this factorization is schematically written as \displaystyle\sigma_{n+1}\sim\int\frac{dp^{2}}{p^{2}}dz\ \frac{\alpha_{s}}{2\pi}P(z)\sigma_{n}\;, (1) where p^{2} is the invariant

Read the complete text