An Interpretable Measure for Quantifying Predictive Dependence between Continuous Random Variables
Renato Assunção, Flávio Figueiredo, Francisco N. Tinoco Junior, Leo M. de Sá-Freire, Fábio Silva · 2025
Read An Interpretable Measure for Quantifying Predictive Dependence between Continuous Random Variables by Renato Assunção, Flávio Figueiredo, Francisco N. Tinoco Junior, Leo M. de Sá-Freire, Fábio Silva (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-10815.txt.
Introduction
Renato Assunção and four co-authors propose a new way to measure how strongly two continuous variables depend on each other, in this 2025 paper. Classic correlation misses curved or tangled relationships. Their measure is fully nonparametric, so it assumes no shape for the link.
The number has a plain meaning: the expected relative loss in prediction accuracy for one variable when the other's distribution is ignored. It runs from zero to one, hitting zero exactly when the variables are independent. The team tested it on over ninety thousand real and synthetic data sets against leading alternatives. It revealed dependencies that rivals missed, especially non functional ones where one input maps to many outputs. The authors pitch interpretability, not just sensitivity, as the advance.
Catalog record
| CREATOR | Renato Assunção, Flávio Figueiredo, Francisco N. Tinoco Junior, Leo M. de Sá-Freire, Fábio Silva |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-10815
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
An Interpretable Measure for Quantifying Predictive Dependence between Continuous Random Variables Renato Assunção, Flávio Figueiredo, Francisco N. Tinoco Junior, Leo M. de Sá-Freire, Fábio Silva Abstract A fundamental task in statistical learning is quantifying the joint dependence or association between two continuous random variables. We introduce a novel, fully non-parametric measure that assesses the degree of association between continuous variables X X and Y Y , capable of capturing a wide range of relationships, including non-functional ones. A key advantage of this measure is its interpretability: it quantifies the expected relative loss in predictive accuracy when the distribution of X X is ignored in predicting Y Y . This measure is bounded within the interval [0,1] and is equal to zero if and only if X X and Y Y are independent. We evaluate the performance of our measure on over 90,000 real and synthetic datasets, benchmarking it against leading alternatives. Our results demonstrate that the proposed measure provides valuable insights into underlying relationships, particularly in cases where existing methods fail to capture important dependencies. 1 Introduction One of the foundational tasks in data science involves quantifying the degree of dependence or association between two or more random variables, denoted as X and Y [ 1 , 2 ] . We are inspired by the association measure \tau_{b} by Goodman and Kruskal [ 3 ] for two categorical random variables. This measure is calculated based on discrete data organized in a contingency table (see Figure 1 ). It calculates the reduction in the probability of incorrectly predicting the category of the Y variable when information about the category of X is available. \tau_{b} is one of the few dependence measures with a clear empirical interpretation [ 2 ] . This clarity of interpretation is the main appeal for us when generalizing the measure to continuous random variables. Rows Columns (Variable X ) (Variable Y ) 1 2 \cdots c Total 1 n_{11} n_{12} \cdots n_{1c} n_{1.} 2 n_{21} n_{22} \cdots n_{2c} n_{2.} \vdots \vdots \vdots \ddots \vdots \vdots r n_{r1} n_{r2} \cdots n_{rc} n_{r.} Total n_{.1} n_{.2} \cdots n_{.c} n_{\cdot\cdot} Figure 1: Left: Illustrative contingency table. Right: Regular grid on top of a scatterplot of a random sample (x_{k},y_{k}) , k=1,\ldots,N of the random vector (X,Y) with joint density f_{XY}(x,y) . We begin with a regular grid superimposed on a scatterplot of independent and identically distributed random points drawn from a continuous random vector distribution. This grid will be taken to the limit as the grid spacing approaches zero in our measure and, consequently, will not appear in the final measure definition. We made two fundamental changes from the original proposal in [ 3 ] to ensure meaningful results. As detailed in Section 3 , Goodman and Kruskal considered random assignments of the data into table categories and counted the number of instances where the true data categories matched the randomly assigned ones. In contrast, we consider the continuous case by using the expected rate of correct random assignments, and taking the asymptotic limit as the sample size approaches infinity and the regular grid spacing approaches zero. We find that a modified version of Goodman and Kruskal’s measure can be expressed in terms of the continuous marginal and conditional densities of X and Y . The resulting measure is linked to the quadratic Rényi entropy [ 4 , 5 ] , a measure that has not been frequently used with continuous data. More importantly, we can provide a probabilistic interpretation of our measure’s values. It captures the expected relative prediction loss incurred when predicting Y while ignoring the value of X . Due to this interpretation as a measure of predictive ability, we named our measure PREDEP, an acronym standing for PREdictive DEPendence measure. The PREDEP range is the [0,1] interval. If PREDEP has a value of, for example, 0.5, this means that knowing X can reduce the predictive loss of Y by 50%. The two extreme values in this [0,1] range characterize extreme probabilistic behavior. PREDEP is zero if, and only if, X and Y are independent, whereas PREDEP equal to 1 means that Y is perfectly predicted by X . Among other properties of PREDEP, in specific special cases such as the bivariate Gaussian distribution, we found a functional relationship between PREDEP, the correlation coefficient, and the mutual information measure. Another contribution of our work is the development of a bootstrap method to estimate PREDEP with empirical data. Specifically, in the conditional distribution component of PREDEP, we need a method to handle the small number of data resulting from conditioning X to a specific value. We can interpret the measure components as a certain convolution density function evaluated at zero, and we explore this interpretation to establish an estimator based on a bootstrap method. To evaluate our measure empirically, we compute PREDEP across more than 90 thousand real datasets in addition to other synthetic datasets. PREDEP’s performance was compared with several leading available alternatives (Pearson correlation coefficient [ 6 ] , mutual information measure [ 5 ] , Maximal information criterion [ 7 ] , the distance correlation [ 8 , 9 ] , the Hilbert-Schmidt Independence Criterion [ 10 ] , and Maximum Mean Discrepancy [ 11 ] ). This comparison demonstrated that our measure provides additional insights relative to these alternative association measures. In summary, the main contributions of this paper are: • PREDEP, a new association measure to evaluate the ability of X to predict Y . The measure is not specialized for any specific type of association (such as linearity or monotonicity). It can capture arbitrarily complex non-linear relationships and it has a probabilistic interpretation as the percentage loss in prediction ability of Y when we ignore X . • Proofs of several theoretical properties of the measure. • A proposed method to estimate PREDEP with empirical data based on a bootstrap method coupled with a convolution density evaluation interpretation of the measure. • An extensive empirical evaluation of the measure is based on more than 90 thousand real datasets and numerous synthetic datasets. We compared PREDEP with the six most relevant alternative association measures. 2 Related work Figure 2 shows examples of different types of association between random variables X and Y . The first row of plots are cases in which the conditional distribution of Y is of the form (Y|X=x)=h(x)+\varepsilon , where the function h represents a hypothesis and \varepsilon has an independent distribution of X , such as N(0,\sigma^{2}) . However, this simple functional relationship between X and Y is not the only way they can be dependent. In the second row of plots, we see examples where X and Y are associated but do not have a functional relationship akin to (Y|X=x)\approx h(x)+\varepsilon . Figure 2: Visualization of functional and non-functional relationships used for benchmarking. The Pearson correlation coefficient is one of the oldest and most popular measures of dependence between two random variables. While it captures well the linear association between X and Y , it performs poorly in non-linear relationships. Spearman’s rank correlation [ 12 ] and Kendall’s tau [ 13 ] were more flexible measures as they aimed to detect the presence of monotonic relationships but