Tools for Unbinned Unfolding

Ryan Milton, Vinicius Mikuni, Trevin Lee, Miguel Arratia, Tanvi Wamorkar, and Benjamin Nachman · 2025

Read Tools for Unbinned Unfolding by Ryan Milton, Vinicius Mikuni, Trevin Lee, Miguel Arratia, Tanvi Wamorkar, and Benjamin Nachman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-09720.txt.

Introduction

Milton and co-authors build tools for a histogram-free way of correcting particle physics data, in this 2025 paper. Detectors distort measurements, and unfolding removes that distortion. New machine learning methods skip the classic binning step, but labs need shared standards before adopting them.

The team show that boosted decision trees can unfold data well with a few summary features, easing the load on heavy deep models. They extend a benchmark data set with acceptance effects that real detectors face. A head-to-head test isolates the effect of binning itself. Two software packages wrap the leading method for lab use, one inside a common framework and one standalone.

Catalog record

CREATORRyan Milton, Vinicius Mikuni, Trevin Lee, Miguel Arratia, Tanvi Wamorkar, and Benjamin Nachman
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-09720

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tools for Unbinned Unfolding Ryan Milton, Vinicius Mikuni, Trevin Lee, Miguel Arratia, Tanvi Wamorkar, and Benjamin Nachman Abstract Machine learning has enabled differential cross section measurements that are not discretized. Going beyond the traditional histogram-based paradigm, these unbinned unfolding methods are rapidly being integrated into experimental workflows. In order to enable widespread adaptation and standardization, we develop methods, benchmarks, and software for unbinned unfolding. For methodology, we demonstrate the utility of boosted decision trees for unfolding with a relatively small number of high-level features. This complements state-of-the-art deep learning models capable of unfolding the full phase space. To benchmark unbinned unfolding methods, we develop an extension of existing dataset to include acceptance effects, a necessary challenge for real measurements. Additionally, we directly compare binned and unbinned methods using discretized inputs for the latter in order to control for the binning itself. Lastly, we have assembled two software packages for the OmniFold unbinned unfolding method that should serve as the starting point for any future analyses using this technique. One package is based on the widely-used RooUnfold framework and the other is a standalone package available through the Python Package Index (PyPI). 1 Introduction Differential cross sections are the currency of scientific exchange in particle, nuclear, and astroparticle physics. These quantities connect the underlying theory to observations. A critical step of any differential cross section measurement is the correction of detector distortions, also known as unfolding or deconvolution . Traditional unfolding methods act on discretized data in the form of histograms. These methods invert the folding equation m=R\,t , where t is the true histogram before detector distortions, m is the observed histogram after detector distortions, and R is the response matrix, which gives the probability of a measurement being in bin i given the truth value is in bin j : R_{ij}=\text{Pr(measure in bin $i$ $|$ truth in bin $j$)}. (1) The matrix R is estimated using detector simulations. However, the solution \hat{t}=R^{-1}m is not a good estimate because it amplifies statistical fluctuations, is not guaranteed to be non-negative definite, and may not exist if R is not square. To circumvent this, various forms of regularized matrix inversion have been proposed over the years to perform unfolding. See Refs. [ 1 , 2 , 3 , 4 ] for reviews and Refs. [ 5 , 6 , 7 ] for the most widely-used unfolding algorithms. While histograms are a useful data representation for a specific analysis, they are fundamentally limited in scope. For example, bin boundaries cannot be changed after the measurement is published, thus inhibiting comparisons between experiments. Most importantly, while the data are intrinsically high-dimensional, published results are low-dimensional. This means that the histograms must be designed for a specific analysis, integrating over the correlations that are thought to be irrelevant for the particular case. For measurements of other observables, such as those motivated by theoretical insight originating long after the initial research, the analysis may have to be repeated from scratch. This motivates machine learning (ML) based unbinned unfolding methods [ 8 , 9 , 10 ] . Such techniques are based on discriminative [ 11 , 12 , 13 ] or generative models, either for distribution mapping [ 14 , 15 , 16 , 17 ] or for probabilistic, conditional generation [ 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 ] . These approaches do not require binning and can readily integrate many features to extend the phase space of the measurement and to improve the precision by accounting for any relevant correlations with the detector response. In the last couple of years, there have been a number of ML-based unfolding measurements performed by various experiments, including H1 [ 27 , 28 , 29 , 30 ] , LHCb [ 31 ] , ATLAS [ 32 , 33 ] , CMS [ 34 ] , and STAR [ 35 , 36 ] . While there has been some discussion of standard tools for presenting the results of ML-based differential cross section measurements [ 8 ] , there are currently no standard tools for doing the measurements themselves. For classical unfolding, RooUnfold [ 37 , 38 ] is the community standard, used by a majority of all measurements in recent years across experiments. The goal of this paper is to describe easy to use software for unbinned unfolding. We have prepared two sets of tools. In order to benefit from the extensive user base of RooUnfold, we have prepared a RooUnfold-inspired version of the OmniFold [ 11 , 12 ] ML-based unfolding algorithm. This currently exists in a fork of RooUnfold but it will be further developed for the inclusion of OmniFold into RooUnfold in the future. With this RooUnfold-inspired OmniFold, users of RooUnfold can now easily use ML by swapping out methods without changing any of the inputs or outputs of their analysis. To complement this, we have also prepared a standalone package in the Python Package Index (pip) that implements OmniFold. The pip version is more light weight, since it does not rely on ROOT [ 39 ] . Best practices for the two approaches are discussed in more detail below. We focus on OmniFold because it is the tool that has been used for all unbinned measurements so far. The tools developed here are able to accommodate other methods, which may be added in the future. This paper is organized as follows. Section 2 reviews unfolding methodology and the implementation of the machine learning methods. To the best of our knowledge, there has never been a direct comparison of OmniFold with classical techniques using the same inputs/outputs and also no comparison of OmniFold with decision trees versus neural networks. Numerical results of the latter comparisons in the context of Gaussian and collider physics examples are presented in Sec. 3 , while comparisons using histograms as input to OmniFold and binned unfolding are shown in Appendix A . The paper ends with conclusions and outlook in Sec. 4 . We additionally provide example usage of the unbinned unfolding methods in Appendix B . 2 Methodology As a classical baseline, we will compare unbinned unfolding with Lucy-Richardson Deconvolution [ 40 , 41 ] , also known as Iterative Bayesian Unfolding (IBU) [ 5 ] in particle physics. OmniFold generalizes IBU and so we start by reviewing IBU in Sec. 2.1 . Section 2.2 then describes how pieces of IBU can be replaced with machine learning in order to process unbinned inputs and accommodate high-dimensional inputs/outputs. 2.1 Iterative Unfolding To begin, assume that there is no background to subtract and every event that passes the particle-level event selection passes the detector-level event selection (and vice versa). Non-trivial acceptance effects are revisited at the end of this section. For an initialized result t^{(0)}_{j}=\Pr_{0}(\text{truth is in bin $j$}) , IBU proceeds as follows: \displaystyle t_{j}^{(n+1)} \displaystyle=\sum_{i}\text{Pr}_{n}(\text{truth is $j$}\,|\,\text{measure $i$})\Pr(\text{measure $i$}) \displaystyle=\sum_{i}\frac{R_{ij}t_{j}^{(n)}}{\sum_{k}R_{ik}t_{k}^{(n)}}\times m_{i}, (2) where n is the iteration number. The result is \hat{t}_{\text{IBU}}=t^{(N)} for some N>0 . As N\rightarrow\infty , the unfolded result converges to the maximum likelihood estimate [ 42 ] . In practice, the algorithm is truncated after a few iterations in order to regularize the result. For

Read the complete text