Quasiprobabilistic Density Ratio Estimation with a Reverse Engineered Classification Loss Function
Matthew Drnevich*, Stephen Jiggins, Kyle Cranmer · 2025
Read Quasiprobabilistic Density Ratio Estimation with a Reverse Engineered Classification Loss Function by Matthew Drnevich*, Stephen Jiggins, Kyle Cranmer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-19913.txt.
Introduction
Matthew Drnevich, Stephen Jiggins, and Kyle Cranmer extend a workhorse task to weights that can go negative. The task is density ratio estimation. It trains a classifier whose output turns into the ratio of two densities. Quasiprobabilistic cases, where densities take negative values, show up in physics work. Standard loss functions tie the best classifier to the target ratio in broken ways. The link can jump, or fail to reach some values.
The authors give a convex loss built to keep that link clean. It works for plain and for quasiprobabilistic ratios alike. To score the results, they extend the sliced Wasserstein distance to such distributions. A real case from particle physics tests the method. It is di-Higgs production with jets, through gluon fusion. The method hit state-of-the-art results there.
Catalog record
| CREATOR | Matthew Drnevich*, Stephen Jiggins, Kyle Cranmer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-19913
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Quasiprobabilistic Density Ratio Estimation with a Reverse Engineered Classification Loss Function Matthew Drnevich*, Stephen Jiggins, Kyle Cranmer Abstract We consider a generalization of the classifier-based density-ratio estimation task to a quasiprobabilistic setting where probability densities can be negative. The problem with most loss functions used for this task is that they implicitly define a relationship between the optimal classifier and the target quasiprobabilistic density ratio which is discontinuous or not surjective. We address these problems by introducing a convex loss function that is well-suited for both probabilistic and quasiprobabilistic density ratio estimation. To quantify performance, an extended version of the Sliced-Wasserstein distance is introduced which is compatible with quasiprobability distributions. We demonstrate our approach on a real-world example from particle physics, of di-Higgs production in association with jets via gluon-gluon fusion, and achieve state-of-the-art results. * * footnotetext: Denotes primary contributor August 10, 2026 1 Introduction In many domains of science, density ratios—also known as likelihood ratios—play an important role. They are central to hypothesis tests [ 22 ] , importance sampling [ 18 ] , information theory [ 28 ] , and domain adaptation [ 32 ] . In the context of machine learning, the probability densities associated to data distributions are typically unknown and in scientific settings probability densities are often defined implicitly through a complex simulator [ 9 ] . For example, in High Energy Physics (HEP), Monte Carlo based simulators are heavily utilised to generate synthetic data associated to various theoretical scenarios [ 29 ] . Consequently, neural density ratio estimation (NDRE) has seen wide spread adoption in an array of tasks from simulation-based inference [ 6 , 5 ] to inverse unfolding problems [ 31 , 2 ] . Perhaps the most common approach to NDRE is to train a (probabilistic) binary classifier, and then transform its output so that it serves as an estimate for the density ratio, through what is colloquially referred to as the “likelihood ratio trick” [ 7 ] . The aforementioned transformation is based on the relationship between the Bayes optimal classifier and the targeted density ratio. In this work, we will pursue a generalization of this strategy, in which the function that optimizes a modified loss function is related to the target density ratio through some other transformation. Our motivation for pursuing this alternative strategy is the need to model quasiprobabilistic density ratios, which can be negative. The quasiprobabilistic setting, in which the first Kolmogorov axiom is violated [ 17 ] , arises naturally when modeling quantum mechanical systems, either due to inherently non-classical phenomena such as quantum interference, or as a consequence of approximation techniques employed to describe particle collisions [ 12 , 8 , 21 , 13 ] . The presence of negative densities poses two immediate problems for the vanilla (probabilistic) likelihood ratio trick. First, the output of standard binary classifiers is constrained to be positive, typically as a result of activation functions such as the sigmoid. Second, the family of loss functions used to train binary classifiers [ 27 ] , such as binary cross-entropy, are incompatible with negative densities. Extending neural density ratio estimation to a quasiprobabilistic setting has been demonstrated via the work of Ref. [ 10 ] , in which signed probabilistic measures [ 16 ] , and a quasiprobabilistic-safe loss function, were introduced. In this work, we focus on introducing an analogue of the “likelihood ratio trick” for quasiprobabilsitic classifier-based density ratio estimation through the introduction of a new convex loss function. This approach addresses some of the shortcomings of the PARE loss function introduced in Ref. [ 10 ] and allows one to train a more unrestricted class of models than was previously possible. The paper is organized as follows. In Section 2 , the challenge of applying classifier-based neural density ratio estimation, when faced with learning quasiprobabilistic density ratios, is outlined. A solution to this problem is presented in the form of a new quasiprobabilistic ratio trick, culminating in a novel loss function referred to as the REV erse E ngineered R atio T rick (REVERT) loss. In Section 3 , we demonstrate this new approach on a real world example found in particle physics, by comparing the application of the new loss function to a simple Multilayer Perceptron (dense feed-forward neural network) classifier, in addition to the optimization problem of training the Ratio of Signed Mixtures Models (RoSMM) found in Ref. [ 10 ] . Finally, we conclude with closing remarks in Section 4 . 2 Density Ratio Estimation The problem addressed by this paper is that of learning the density ratio between two quasiprobabilistic distributions using neural classifier-based methods given by: \displaystyle r^{*}(\mathbf{x})\equiv\frac{q(\mathbf{x}|Y=1)}{q(\mathbf{x}|Y=0)}, (1) where the random variable Y is a class label designating from which distribution a dataset is sampled, taking values in \{0,1\} . The distribution q(\cdot) is used to differentiate the quasiprobabilistic density function from a probabilistic density function p(\cdot) . As such, it is allowed that q(\mathbf{x}|y)<0 for some \mathbf{x} . The relationship between the optimal classifier s^{*} for a given loss function \mathcal{L}(s) and the density ratio r^{*} defines the aforementioned “ratio trick”. For the class of loss functions considered herein, the resulting optimised classifier: s^{*}=\argmin_{s}\,\mathcal{L}(s), (2) is directly related to the density ratio r^{*} through some transformation T , such that r^{*}=T(s^{*}) 1 1 1 Note that T here is the inverse of the convention used in Ref. [ 10 ] , but this is the more natural form to use in this setup. . Figure 1 : Examples of transformations between the output space of a classifier s which outputs values in the interval (0,1) and the transformed output of the classifier using “ratio trick” transformations. T_{\rm{BCE}} is the transformation associated with the binary cross-entropy loss function and T_{q} is an example of a transformation suitable for quasiprobabilistic settings. T_{q} can transform the classifier outputs to any real number, while T_{\rm{BCE}} transforms all classifier outputs to positive numbers only. A key characteristic of these transformations is that they should be homeomorphisms (continuous bijective functions) between the codomain of the optimizable function, s (the space of the model outputs), and the codomain of the density ratio function, r (the set of possible density ratio values). In the probabilistic setting, the density ratio is always nonnegative, while a quasiprobabilistic density ratio can take on any real value. Accordingly, standard probabilistic density ratio estimation loss functions are constructed such that their associated transformations are homeomorphisms between the space of classifier outputs and the positive real line \mathbb{R}_{>0} . As a result, these transformations are unable to map any of the classifier outputs to negative density ratio values, even if the data is quasiprobabilistic; as depicted by Figure 1 . This motivates the construction of a new loss function for quasiprobabilistic settings where the associated ratio trick transformation is a homeomorphism between the space of model outputs and all real numbers. 2.1 Deriving a Loss Function One way to construct a suitable loss function is by starting with a ratio trick transformation, and then reverse engineering a loss function that gives rise to said transformation. To achieve this, we first consider how the ratio trick [ 7 ] process works for binary classification. For