Madgraph on GPUs and vector CPUs: towards production

\firstname Andrea \lastname Valassi \fnsep, \firstname Taylor \lastname Childers, \firstname Stephan \lastname Hageböck, \firstname Daniele \lastname Massaro, \firstname Olivier \lastname Mattelaer, \firstname Nathan \lastname Nichols, \firstname Filip \lastname Optolowicz, \firstname Stefan \lastna · 2025

Read Madgraph on GPUs and vector CPUs: towards production by \firstname Andrea \lastname Valassi \fnsep, \firstname Taylor \lastname Childers, \firstname Stephan \lastname Hageböck, \firstname Daniele \lastname Massaro, \firstname Olivier \lastname Mattelaer, \firstname Nathan \lastname Nichols, \firstname Filip \lastname Optolowicz, \firstname Stefan \lastna (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21935.txt.

Introduction

Andrea Valassi and nine co-authors report on a five-year push to run a major particle physics program on graphics chips and modern processors. Their paper appeared in two thousand twenty-five. The program, MadGraph, simulates collisions for labs like the one behind the Large Hadron Collider. Faster runs mean more events can be simulated.

Work began in two thousand twenty and brought many new features, tests, and fixes. Early feedback from the CMS experiment helped shape the code. The first full release for the simplest class of processes came out in the autumn of the year before the paper. The authors describe the state of that release as of a major computing meeting. They also set out what remains for fuller production use. Speed matters because collider data keeps growing.

Catalog record

CREATOR\firstname Andrea \lastname Valassi \fnsep, \firstname Taylor \lastname Childers, \firstname Stephan \lastname Hageböck, \firstname Daniele \lastname Massaro, \firstname Olivier \lastname Mattelaer, \firstname Nathan \lastname Nichols, \firstname Filip \lastname Optolowicz, \firstname Stefan \lastna
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-21935

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Madgraph on GPUs and vector CPUs: towards production \firstname Andrea \lastname Valassi \fnsep, \firstname Taylor \lastname Childers, \firstname Stephan \lastname Hageböck, \firstname Daniele \lastname Massaro, \firstname Olivier \lastname Mattelaer, \firstname Nathan \lastname Nichols, \firstname Filip \lastname Optolowicz, \firstname Stefan \lastname Roiser, \firstname Jørgen \lastname Teig, \firstname Zenny \lastname Wettersten Abstract The effort to speed up the Madgraph5_aMC@NLO generator by exploiting CPU vectorization and GPUs, which started at the beginning of 2020, has delivered the first production release of the code for leading-order (LO) processes in October 2024. To achieve this goal, many new features, tests and fixes have been implemented in recent months. This process benefitted also from the early feedback of the CMS experiment. In this contribution, we report on these activities and on the status of the LO software at the time of CHEP2024. The 5-year journey to the first LO release CUDACPP v1.00.00 1 Introduction MadGraph5_aMC@NLO bib:mg5 ; bib:mg5amc (hereafter, MG5aMC) is a physics event generator used in the data processing workflows of many High Energy Physics (HEP) experiments, notably those at CERN’s Large Hadron Collider (LHC). The computational cost of event generation is a non-negligible fraction of the total computing costs of the LHC experiments and is predicted to further increase as the high-luminosity LHC programme (HL-LHC) gets underway bib:csbs . The MG5aMC software has been developed over more than two decades and initially targeted CPUs only, mostly using sequential processing paradigms. Over time, new computing architectures designed for parallel processing, such as multi-core CPUs with wide vector registers and graphical processing units (GPUs) have become widely available in the Worldwide LHC Computing Grid (WLCG) infrastructure. This represents a challenge, because porting legacy software to efficiently support these new architectures require their radical rethink and redesign, and failure to do so may lead to large under-utilization of the computing power of existing hardware resources; but it also poses an opportunity, as the efficient exploitation of these resources using better software can significantly reduce the computing costs of the LHC experiments. This was precisely the motivation behind the development effort described in this paper, which started in February 2020 with the initial goal of porting MG5aMC to GPUs, but was very soon extended to also optimize the code for vector CPUs, using the same data parallel approach and the same code base for both architectures. After almost five years of development, the first production release of the code was delivered in October 2024, just before the CHEP2024 conference where we have presented it for the first time bib:chep2024slides . MG5aMC is a code generating framework, largely written in Python, which allows users to generate and run physics code to perform Monte Carlo (MC) event generation for any physics process of their choice. By default, physics code in MG5aMC is generated in Fortran and can only run on CPUs, with no support for vectorization: this is the production backend that has been used so far by the LHC experiments in their event generation campaigns. The main deliverable of the work described in this paper is a new code-generating plugin that we named “CUDACPP”, because the generated physics code uses C++ instead of Fortran to execute the most computationally intensive sections of the code on vector CPUs, and also includes CUDA extensions to run the code on NVidia GPUs. Our contribution includes not only the first release v1.00.00 of the CUDACPP plugin, but also the addition of many changes that were needed in the overall framework into a new production release MG5aMC v3.6.0. In this paper, we focus on the description of the additional work that was necessary to achieve the release, with respect to what has been presented at previous conferences bib:vchep2021 ; bib:ichep2022 ; bib:acat2022 ; bib:chep2023 . This includes new features such as the support for Beyond-Standard-Model (BSM) processes, the support for AMD GPUs via HIP language extensions to our previous CUDA/C++ codebase, a mechanism to distribute our software as a new plugin of the MG5aMC framework, and especially the extensive testing, debugging and optimization of the full MG5aMC workflow and of these new functionalities. Work is also in progress to support Intel GPUs via SYCL language extensions, integrating into the CUDACPP plugin the prototypes we presented at previous conferences bib:ichep2022 ; bib:acat2022 ; bib:chep2023 , but we will not cover any of this work in this paper. Our work on the first CUDACCP release has also benefitted from the early feedback of our colleagues from the CMS experiment, whose extensive tests of the functionality and performance speedup achievable using our software have also been presented to this conference bib:jin . Currently, CUDACPP only supports Leading-Order (LO) physics processes: The techniques and software developed for LO are largely reusable also for Next-to-Leading-Order (NLO) processes. Extending CUDACPP to also support NLO is one of the main further directions of ongoing developments and is described in a separate contribution to this conference bib:zenny . This is the outline of this paper: in Sec. 2 we describe the architecture and the evolving focus of developments over time; in Sec. 3 we give performance results and an outlook. 2 Overview of architecture and developments From a computational point of view, MG5aMC is a complex framework with many different software layers. What follows is only a high-level summary; a more complete description can be found in the MG5aMC paper bib:mg5amc . Essentially, a user who wants to compute a cross section or generate some events mainly needs to specify the desired physics process, the required precision or sample size and the desired backend, via an interactive prompt or a batch script. The MG5aMC framework (largely written in Python) then takes care of everything else: it determines the subprocesses that must be computed to handle the desired physics process; it generates physics code for the required backend (by default, using Fortran for CPUs) and builds it into one “madevent” executable for each subprocess; it optimizes configuration files via test runs and optionally creates a “gridpack” for Grid distribution; it executes one or more copies of all relevant madevent executables; finally, it combines the results of the different program executions to provide the overall output cross section and/or an output LHE file aggregating all MC events that have been generated. Since the first software commits for our project in February 2020, achieving the delivery of the first production release took almost five years because we had to adapt and test all of these layers. We did this incrementally in successive steps, as described in the following. This is schematically represented in Fig. 1 . Figure 1: Schematic representation of the architectural evolution of MG5aMC. The main difference between the old Fortran-only version (top left, pink background) and those based on CUDACPP (light blue background) is that the former uses a sequential single-event API for the calculation of matrix elements, while the latter uses a data-parallel multi-event API. Additional details on the evolution of the work on the CUDACPP plugin between 2020 and 2024 are provided in the text. These plots, presented at CHEP2024 bib:chep2024slides , are derived from those presented in a 2020 talk to the HSF generator WG bib:hsf2020 . Internally, each madevent executable has itself a complex structure. A similar data

Read the complete text