An AI-ready, Polarized Electron-Positron Collision Dataset

Chi Lung Cheng, Simon Corrodi, T. J. Hobbs, Alaettin Serhan Mete, Benjamin Nachman · 2026

Read An AI-ready, Polarized Electron-Positron Collision Dataset by Chi Lung Cheng, Simon Corrodi, T. J. Hobbs, Alaettin Serhan Mete, Benjamin Nachman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00224.txt.

Introduction

Cheng and co-authors free a trove of old clash data in a form AI tools can use. The data come from the SLD test at the SLAC site. That test ran in the late years of the last century. Its beam of light-charged bits was strongly spun.

The set holds near 660,000 rebuilt clash events. They were caught at an energy near the Z mass peak. Old store forms are now turned into fresh, wide-used forms. AI agents helped with the shift. A stack of in-house notes was also scanned into text. The team shows checks that prove the data are sound. They sketch uses for both clash science and machine learn work.

Catalog record

CREATORChi Lung Cheng, Simon Corrodi, T. J. Hobbs, Alaettin Serhan Mete, Benjamin Nachman
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.00224

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

An AI-ready, Polarized Electron-Positron Collision Dataset Chi Lung Cheng, Simon Corrodi, T. J. Hobbs, Alaettin Serhan Mete, Benjamin Nachman Abstract We present a modernized, AI-ready release of reconstructed data from the SLD experiment at the SLAC Linear Collider (SLC). The dataset comprises approximately 660,000 reconstructed events collected at s ≈ 91.2 \sqrt{s}\approx 91.2 GeV with a highly polarized electron beam from 1996–1998. The data have been translated from legacy formats into modern, widely-used file formats with the help of AI agents. The release also includes a corpus of newly digitized SLD internal documentation. We describe the contents of both components and provide physics validation demonstrations along with illustrations of their utility for physics and machine learning research in particle physics. I Introduction The SLAC Linear Collider (SLC) SLC:1980report was the first and only high-energy linear e^{+}e^{-} collider, operating at the Z pole ( \sqrt{s}\approx 91.2 GeV) from 1989 to 1998. Its unique feature was a highly polarized electron beam, achieving longitudinal polarizations of \sim 73–77% SLD:2000leq , which enabled precision electroweak measurements not accessible at unpolarized colliders. The SLD experiment SLD:1984design ran from 1992 to 1998 at the Z pole. The high-polarization 1996–1998 portion, covered by the present release, alone comprises approximately 660,000 reconstructed events. SLD made pioneering contributions to heavy flavor physics and electroweak precision tests, including the world’s most precise single measurement of the weak mixing angle \sin^{2}\theta_{W}^{\text{eff}} via the left-right asymmetry A_{LR} SLD:2000leq . The detector featured an innovative CCD-based vertex detector (VXD) SLD:1996vxd3 , which provided exceptional impact parameter resolution and enabled high-purity flavor tagging of b - and c -quark jets. A representative reconstructed Z decay in the SLD detector is shown in Figure 1 . Despite the lasting physics impact of SLD, the reconstructed data have remained in legacy formats that are inaccessible to modern analysis tools. As interest in applying machine learning (ML) to particle physics continues to grow Carleo:2019ptp , there is significant value in making diverse experimental datasets available in standardized, ML-ready formats. Electron-positron collision data are particularly attractive for ML development because the initial state is precisely known, the events are relatively clean compared to hadron collisions, and the well-understood theoretical predictions provide reliable ground truth for benchmarking. Equally important is the institutional knowledge that enables analysts to interpret legacy datasets, including data format conventions, calibration choices, and selection rationales. For older experiments, much of this lives only in internal notes that were never born digital. Figure 1 : An event display of a collision in the SLD detector. Reproduced from this website . The diversity of public datasets available for training and benchmarking is itself a first-order bottleneck for ML in particle physics. In collider physics, the vast majority of currently public datasets come from a single type of collision—proton-proton ( pp ) collisions at the LHC—leaving the e^{+}e^{-} and ep regimes underrepresented. Reviving high-quality legacy data from these other regimes broadens the range of physics conditions available for downstream analyses and applications, and the SLD is a uniquely valuable target because the longitudinal polarization of its initial state is a configuration that has not been realized at any other collider. SLD therefore remains the only Z -factory dataset where the helicity of the initial state is known event by event. In this paper, we describe the modernization and public release of the SLD reconstructed dataset, including its translation into modern, AI-ready columnar formats. The SLD Collaboration software, written in an extension of the Fortran programming language (Mortran), relied on a proprietary and poorly documented ecosystem that is now largely unavailable, posing a significant challenge for accessing and interpreting the data. In this context, AI-assisted tools were used to aid in the interpretation and recovery of previously undocumented or partially documented data structures. This provides a representative example of how legacy datasets with incomplete software or documentation can be made accessible using modern tools. The release is accompanied by digitized internal documentation previously available only in physical form, along with text extracted by several state-of-the-art document-processing tools to make the corpus directly usable in modern AI workflows. We also showcase the utility of the dataset for modern ML research by passing it through a particle physics foundation model, and demonstrate the AI-readiness of the documentation release through an agentic question-answering system. II Dataset The released dataset is hosted on Zenodo Cheng:2026zenodo and consists of two components: the reconstructed collision data covering the 1996–1998 SLD runs, and the digitized internal SLD documentation. The 1992–1995 data have not yet been recovered and are not part of this release. II.1 Reconstructed Data After data taking concluded, the analysis software that decoded the binary banks fell into disuse. The original developers had dispersed, no public specification of the SLD data binary format (Jazelle) bank layout existed outside of internal notes (some recoverable only through the Internet Archive), and none of the original Mortran reader code remains operational on modern systems. We were fortunate to acquire a partial Java reader written in 2021 by the former SLD software coordinator. While highly useful, this reader covers only a subset of the reconstructed bank families and does not interoperate with contemporary columnar formats or array libraries. For example, it does not expose the PHBM beam-information bank, which records the per-event electron-beam polarization and its associated uncertainty. Without this bank, none of SLD’s polarized observables, including the left-right asymmetry A_{LR} , can be reconstructed. The reconstructed binaries themselves were preserved on archival storage, but without a complete and modern reader the dataset was inaccessible to current analysis tools. We reconstructed the missing bank layouts by reverse-engineering the binary format with AI assistance (Anthropic’s Claude). Two sources of information guided the recovery. The first is partial documentation of bank contents preserved in the recovered SLD internal notes, which specifies which fields each bank should carry but is typically silent on, or outdated about, how those fields are serialized and compressed on disk. The second is physics-based ground truth: the kinematics of well-understood processes such as Z\to q\bar{q} fix the numerical values that must appear in each event, providing a strict test for any proposed layout. Working from this combination, we identified candidate field positions and data types from recurring byte patterns and alignments across event dumps. Candidates that failed to reproduce the expected values across additional events were refined in the next iteration, with the cycle typically converging for each bank family after several rounds. In this way we decoded the principal MiniDST bank families, including several not covered by any surviving implementation, and packaged the result into the jazelle reader described below. A small number of bank families remain incompletely decoded. These are well-defined residual problems expected to be tractable with the same approach given additional test events or documentation. Beyond making SLD data usable again, this exercise illustrates a broader pattern: data files that outlive the software written to decode them are now recoverable by combining surviving documentation with physics-based constraints and modern AI assistance. The translation pipeline is released as the open-source Python package jazelle Cheng:jazelle_reader . It reads the legacy Jazelle binary format and emits Awkward Pivarski:awkward record arrays that can be serialized to several AI-friendly columnar formats including HDF5, Parquet, and Feather. The

Read the complete text