COWs and their Hybrids: A Statistical View of Custom Orthogonal Weights
Chad Schafer, Larry Wasserman, Mikael Kuusela · 2025
Read COWs and their Hybrids: A Statistical View of Custom Orthogonal Weights by Chad Schafer, Larry Wasserman, Mikael Kuusela (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-16174.txt.
Introduction
Chad Schafer and two co-authors give a statistical foundation to a weighting trick used across high-energy physics. Experiments often record mixtures of signal and background events, and a discriminating variable can assign each event a signal weight, letting analysts study other variables for the signal alone.
Custom orthogonal weights, introduced by Dembinski and collaborators in 2022 as a generalization of the older sPlot technique, tailor those weights to the task. The authors formalize the assumptions the method silently needs, establish its statistical properties, and examine extensions and alternatives. Their treatment clarifies when the weights can be trusted, a question of practical weight for any analysis built on separated signal samples.
Catalog record
| CREATOR | Chad Schafer, Larry Wasserman, Mikael Kuusela |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-16174
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
COWs and their Hybrids: A Statistical View of Custom Orthogonal Weights Chad Schafer, Larry Wasserman, Mikael Kuusela Abstract A recurring challenge in high energy physics is inference of the signal component from a distribution for which observations are assumed to be a mixture of signal and background events. A standard assumption is that there exists information encoded in a discriminant variable that is effective at separating signal and background. This can be used to assign a signal weight to each event, with these weights used in subsequent analyses of one or more control variables of interest. The custom orthogonal weights (COWs) approach of Dembinski et al. 2022 , a generalization of the sPlot approach of Barlow 1987 and Pivk and Le Diberder 2005 , is tailored to address this objective. The problem, and this method, present interesting and novel statistical issues. Here we formalize the assumptions needed and the statistical properties, while also considering extensions and alternative approaches. , and Department of Statistics & Data Science and STatistical Methods for the Physical Sciences (STAMPS) Research Center, Carnegie Mellon University 1 Introduction Detectors used in particle physics experiments collect data on events , often resulting from particle collisions within an accelerator, such as the Large Hadron Collider (LHC) at CERN. Some events comprise the signal of interest, while others are contaminating background. For example, Aaij et al. 2025 reported on seminal results from the LHCb Experiment ( Augusto Alves Jr et al. 2008 ) at the LHC in which key properties of the \Xi_{c} baryon were estimated, including its spin-parity , characterizing its total angular momentum and intrinsic symmetry under spatial inversion. A central challenge to achieving these results is the statistically rigorous separation of the \Xi_{c} baryon (signal) events from the background. A common approach to achieving this separation (including that used in the aforementioned study) is the sPlot (or sWeights ) method ( Barlow 1987 ; Pivk and Le Diberder 2005 ) . Here, the joint distribution of a pair of random variables (M,T) is modelled as p(m,t)=zg_{1}(m)h_{1}(t)+(1-z)g_{2}(m)h_{2}(t),\>\>\>0\leq z\leq 1, (1) where the first term represents the signal distribution and the second represents the background. In the sPlot formulation, the functions g_{1},g_{2},h_{1},h_{2} are each densities, but this could be relaxed. M is referred to as the discriminant variable (often a mass measurement) and T is called the control variable (often an angular variable or some other kinematic property of the event). In this model, M and T are independent in the signal and background. That is, M and T are conditionally independent given S , where S\in\{1,2\} denotes signal or background. The goal is to estimate the signal density for T , \mathsf{s}(t)=h_{1}(t) . In the absence of additional assumptions, the model ( 1 ) is an example of a nonparametric mixture model with conditional independence constraints. Such models have been well-studied in the statistics literature. See, for example, Hall et al. 2005 ; Hall and Zhou 2003 ; Chauveau et al. 2015 ; Allman et al. 2009 ; Zheng and Wu 2020 ; Bonhomme et al. 2016 . A surprising result, due to Hall and Zhou 2003 and Allman et al. 2009 , is that the mixture ( 1 ) is nonparametrically identified if a third conditionally independent variable is added. But with only two variables, and lacking further assumptions, the model ( 1 ) is non-identified and we may find other functions \widetilde{g}_{1},\widetilde{h}_{1},\widetilde{g}_{2},\widetilde{h}_{2} and \widetilde{z} such that p(m,t)=\widetilde{z}\,\widetilde{g}_{1}(m)\widetilde{h}_{1}(t)+(1-\widetilde{z})\widetilde{g}_{2}(m)\widetilde{h}_{2}(t). Thus, recovering the component functions is impossible even if the joint distribution p(m,t) were known. The sPlot approach avoids this non-identifiability by restricting the form of g_{1} and g_{2} . In the extreme case, these functions can be taken as fully known, based on theoretical understanding of the underlying physics. A more realistic assumption, often employed, is that a sample is available from the discriminant variable, and assumptions are made on the parametric forms of g_{1} and g_{2} . The parameters of these distributions (often called shape parameters in particle physics) can hence be estimated provided that the chosen parametric forms result in an identified model. This sets up the main heuristic of this approach: Important information is assumed to be known regarding the nature of the signal and background in the discriminant variable M and, from this knowledge, events can be assigned weights that quantify the likelihood the event is from signal. These weights can then be used in analyses applied to the control variable T of interest. But, this only works in the presence of the aforementioned conditional independence assumptions. Dembinski et al. 2022 introduced a more general model, called COWs (custom orthogonal weights), that relaxes this assumption as follows: p(m,t)=\underbrace{\sum_{k=1}^{s}z_{k}\,g_{k}(m)h_{k}(t)}_{\mathrm{signal},\>\mathsf{s}(m,t)}+\underbrace{\sum_{k=s+1}^{b+s}z_{k}\,g_{k}(m)h_{k}(t)}_{\mathrm{background},\>\mathsf{b}(m,t)},\quad\sum_{k=1}^{b+s}z_{k}=1. (2) The main target of interest is the signal density for T , denoted \mathsf{s}(t) 1 1 1 Throughout this work, to ease notation, we will use argument(s) to a density to specify the corresponding random variables, i.e., we will write \mathsf{s}(t) instead of \mathsf{s}_{T}(t) , unless there is a chance of ambiguity. , for which \mathsf{s}(t)\propto\sum_{k=1}^{s}z_{k}h_{k}(t). This model reduces to sPlot when s=b=1 . But when s,b>1 , the conditional independence assumption of sPlot is relaxed and a broader range of distributions can be fit. Critically, the COWs method assumes that the densities {\cal G}_{1}=\{g_{1},\ldots,g_{s}\} and {\cal G}_{2}=\{g_{s+1},\ldots,g_{s+b}\} are specified, up to at least a parametric form for each g_{k} , and that the resulting shape parameters, along with the z_{k} , are identified and can be estimated consistently. Note that the marginals for the background and signal for M can be written as, respectively, \displaystyle\mathsf{s}(m) \displaystyle\propto\sum_{k=1}^{s}z_{k}\,g_{k}(m)\>\>\>\mbox{and}\>\>\>\mathsf{b}(m)\propto\sum_{k=s+1}^{s+b}z_{k}\,g_{k}(m), (3) and, if \mathsf{s}(m) and \mathsf{b}(m) were taken as known, there would be a constraint placed on the g_{k} ’s. When the g_{k} ’s are given, \mathsf{s}(t) is identified. But, following the results of Hall and Zhou 2003 , there are many choices of g_{k} that satisfy ( 3 ) and give the same joint p(m,t) . So, in fact, the COWs model is not, in general, identified. However, that does not mean that \mathsf{s}(t) is non-identified; indeed, whether it is identified remains an open question. That is: if we change the functions g_{k} while preserving p(m,t) and preserving the constraint ( 3 ), does \mathsf{s}(t) remain the same? If yes, then \mathsf{s}(t) is identified. If no, then \mathsf{s}(t) is not identified. The purpose of this paper is to review the COWs method, to study its statistical properties as well as the identification questions above and to consider some extensions. In particular, we consider an alternative way to relax the conditional independence assumption, namely, a mixture of copulas. This approach avoids some of the identifiability issues. In problems where there are more than two variables, sPlot is fully nonparametrically identifiable. Estimating such a model can be tricky but COWs suggests a new method for this problem as we explain in Section 6.1 . 2 Background In this section, we explain further the COWs approach. Starting with the model given in Equation ( 1 ), the insight of the sPlot method was to note that a weighting function w_{\mathsf{s}} with properties \int w_{\mathsf{s}}(m)g_{1}(m)\>dm=1\>\>\>\>\mbox{and}\>\>\>\int w_{\mathsf{s}}(m)g_{2}(m)\>dm=0 would have the effect of isolating the signal component of the control variable, i.e., \int w_{\mathsf{s}}(m)p(m,t)\>dm=zh_{1}(t). A weighting function for background, w_{\mathsf{b}} , could be