DPHEP-2025-01 Data Preservation in High Energy Physics DPHEP Collaboration

Alexandre Arbey, Jamie Boyd, Daniel Britzger, Concetta Cartaro, Gang Chen, Gabor David, Dmitri Denisov, Cristinel Diaconu, Dirk Duellmann, Marcus Ebert, Eckhard Elsen, Jacopo Fanini, Dillon S. Fitzgerald, Benjamin Fuks, Gerardo Ganis, Achim Geiser, Takanori Hara, Lukas Heinrich, Michael D. Hildreth, · 2025

Read DPHEP-2025-01 Data Preservation in High Energy Physics DPHEP Collaboration by Alexandre Arbey, Jamie Boyd, Daniel Britzger, Concetta Cartaro, Gang Chen, Gabor David, Dmitri Denisov, Cristinel Diaconu, Dirk Duellmann, Marcus Ebert, Eckhard Elsen, Jacopo Fanini, Dillon S. Fitzgerald, Benjamin Fuks, Gerardo Ganis, Achim Geiser, Takanori Hara, Lukas Heinrich, Michael D. Hildreth, (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23619.txt.

Introduction

The DPHEP collaboration reports on fifteen years of preserving data from high-energy physics experiments. Saved data keep producing science long after a detector shuts down, and they seed the design of future machines. The report speaks to Europe's strategy planning for particle physics.

It draws lessons from past colliders, such as reusing HERA data for a planned electron-ion collider, and from keeping LHC data usable far past the run. Careful preservation planned into new experiments boosts computing efficiency and open science. The authors list recommendations for getting full value from costly experiments.

Catalog record

CREATORAlexandre Arbey, Jamie Boyd, Daniel Britzger, Concetta Cartaro, Gang Chen, Gabor David, Dmitri Denisov, Cristinel Diaconu, Dirk Duellmann, Marcus Ebert, Eckhard Elsen, Jacopo Fanini, Dillon S. Fitzgerald, Benjamin Fuks, Gerardo Ganis, Achim Geiser, Takanori Hara, Lukas Heinrich, Michael D. Hildreth,
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-23619

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DPHEP-2025-01 Data Preservation in High Energy Physics DPHEP Collaboration Alexandre Arbey, Jamie Boyd, Daniel Britzger, Concetta Cartaro, Gang Chen, Gabor David, Dmitri Denisov, Cristinel Diaconu, Dirk Duellmann, Marcus Ebert, Eckhard Elsen, Jacopo Fanini, Dillon S. Fitzgerald, Benjamin Fuks, Gerardo Ganis, Achim Geiser, Takanori Hara, Lukas Heinrich, Michael D. Hildreth, Julie M. Hogan, Henry Klest, Sabine Kraml, Eric Lançon, Clemens Lange, Kati Lassila-Perini, Sergey Levonian, Dietrich Liko, Chiara Mariotti, Zach Marshall, Thomas McCauley, François Le Diberder, Jean-Yves Le Meur, Gerald Myatt, Maxim Potekhin, Michael Roney, Pablo Saiz, Heidi Schellman, Jose Benito Gonzalez Lopez, Matthias Schröder, Ulrich Schwickerath, Tim Smith, David South, Giordon Stark, Tibor Šimko, Jan Timmermans, Andrii Verbytskyi, Arne Wiebalck, Zhiqing Zhang Abstract Data preservation significantly increases the scientific output of high-energy physics experiments during and after data acquisition. For new and ongoing experiments, the careful consideration of long-term data preservation in the experimental design contributes to improving computational efficiency and strengthening the scientific activity in HEP through Open Science methodologies. This contribution is based on 15 years of experience of the DPHEP collaboration in the field of data preservation and focuses on aspects relevant for the strategic programming of particle physics in Europe: the preparation of future programs using data sets preserved from previous similar experiments (e.g. HERA for EIC), and the use of LHC data long after the end of the data taking. The lessons learned from past collider experiments and recent developments open the way to a number of recommendations for the full exploitation of the investments made in large HEP experiments. 1 1 1 This document has been submitted to the European Strategy for Particle Physics 2026. 1 Introduction and context The need for data preservation demands long-term collaboration to support the continuing analysis program of complex data produced by large experiments. This was highlighted at the conclusion of operation of several large collider experiments at the beginning of this century, in the years before the LHC began. A study group was convened and produced in 2009 a report [ 1 ] that proposed a coordinated initiative in HEP, further detailed in a 2012 blueprint [ 2 ] . The document outlined initial data preservation projects and solidified the collaborative framework. The DPHEP collaboration, launched in 2013 by about ten national funding agencies and CERN, partnered with ICFA to facilitate discussion, consensus, and knowledge sharing in HEP data preservation and governance. It is a unique forum to discuss R&D for shared preservation tools, align projects, and develop sustainable, cost-effective long-term strategies. Additionally, DPHEP promotes data preservation within the HEP community and among key stakeholders. DPHEP issues global reports [ 3 , 4 ] and holds regular collaboration workshops. It should however be noted that, beyond the success stories described below and surveyed by DPHEP, the overall observation is that the heritage of complex high-energy physics experiments is in general lost 2 2 2 It is estimated that only a few percent of the experiments in HEP are connected and active on data preservation issues. One can base this estimate on the CERN experiments grey-book when compared to active experiments involved in DPHEP. . Beyond the missed opportunities and given the large investments behind these data sets, this situation does not shed the best light on the place of research in society and may suggest that HEP scientists do not recognize or fully appreciate the value of preserved data as a key research product. 2 Data preservation concepts Across the scientific data landscape, HEP data acquisition and processing presents several outstanding features: size, complexity, distributed and experiment-optimised computing models. It is structured by “events”, that are time stamped and represent the smallest unit of processing. One of the central computing optimization goals is to extract as much information as possible from that provided by the detector within an event. To make it meaningful, the information needs to be reconstructed and interpreted. Those enhanced events are selected, then grouped in statistical ensembles (histograms), and compared to the simulation and theory. Finally, the systematic uncertainties are estimated and the physics measurements are extracted and summarized in publications. Although the full information is needed to qualify and reproduce the results, only a small fraction of the data sample constitutes the final analysis. Collider experiments have broad physics programs, and the experimental collaborations can only publish a fraction of the possible physics analyses during their lifetimes. This leaves a lot of room to explore and it is not uncommon for HEP experiments that analyses appear that go beyond initial physics analysis plans. It is important to note that the term data , is not restricted to files on disk, but comprises all digitally encoded information which has been created as a result of the planning, running and exploiting of an experiment. It includes therefore software at all levels (analysis scripts as well as the minimally requested reprocessing programs), documentation in all forms, indexing and meta-data, calibration databases etc.. Similarly, preservation is not restricted to securely storing data, but comprises all it takes for keeping the data alive, accessible and usable, across the ever changing IT infrastructure and technologies, including data and software quality procedures, computing frameworks etc. Therefore, the term “Data Preservation” is inclusive and covers all necessary ingredients to ensure extraction of original results from experimental data well beyond the period of data taking. 3 Data Preservation benefits The benefits of Data Preservation (DP) were predicted in the initial DPHEP document [ 1 ] . Long-term continuation and extension of scientific programs maximise existing data’s potential. The JADE data rescue, leading to a significantly improved measurement of the strong coupling, is an iconic example 3 3 3 S. Bethke and A. Wagner, Eur. Phys. J. H 47 (2022), 16 [ arXiv:2208.11076 [hep-ex]]. . Reusing past data can lead to advancements through new theories or techniques 4 4 4 The case for data reinterpretation and its impact on the preservation is addressed in a community paper submitted to ESPP. . These sophisticated analyses benefit from full statistical power and enable unique cross-collaboration opportunities to reduce uncertainties, refine measurements, and make new discoveries. Preserving data also supports education, training, and outreach, engaging students in high-energy physics research and attracting new talents. With a perspective of 15 years, the benefits of data preservation have been explored in the past few years through workshops and a general report [ 4 ] . A short account is given below. Figure 1 : Examples of experiments’ scientific output. The period after the end of the data taking is indicated by the shaded area. The periods where the collaborations relied on computing projects newly designed for DP is show by an arrow. The period of scientific production is comparable with the data acquisition duration. 3.1 Enhanced scientific outcome The main indicator for a successful DP remains the publication record. The experience acquired with large collider experiments demonstrates that the HEP data is exploited in the long term: systematically for at least five years after the end of data taking, and very often for 15 years (e.g. large parts of the LEP data are still alive today, 25 years after the end of the data taking). The reasons are multiple: measurements take longer than initially planned, results are

Read the complete text