A Sample-Efficient Conditional Independence Test in the Presence of Discretization

Boyang Sun, Yu Yao, Xinshuai Dong, Zongfang Liu, Tongliang Liu, Yumou Qiu, Kun Zhang · 2025

Read A Sample-Efficient Conditional Independence Test in the Presence of Discretization by Boyang Sun, Yu Yao, Xinshuai Dong, Zongfang Liu, Tongliang Liu, Yumou Qiu, Kun Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08747.txt.

Introduction

Boyang Sun and six co-authors proposed a sample-efficient conditional independence test in twenty twenty-five for data that arrive discretized. Many real variables are recorded in bins because of measurement limits, and running standard independence tests on the bins can mislead. Earlier fixes binarize the data to reason about the latent continuous variables, but binarization throws away information and weakens the test.

The new test skips binarization. The authors show the latent independence question can be recast as an over-identifying restrictions problem solvable by the generalized method of moments. They derive a test statistic with the correct asymptotic distribution using nodewise regression. Theory and experiments across data sets show the test beats binarization-based competitors, especially when samples are scarce.

Catalog record

CREATORBoyang Sun, Yu Yao, Xinshuai Dong, Zongfang Liu, Tongliang Liu, Yumou Qiu, Kun Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-08747

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Sample-Efficient Conditional Independence Test in the Presence of Discretization Boyang Sun, Yu Yao, Xinshuai Dong, Zongfang Liu, Tongliang Liu, Yumou Qiu, Kun Zhang Abstract In many real-world scenarios, interested variables are often represented as discretized values due to measurement limitations. Applying Conditional Independence (CI) tests directly to such discretized data, however, can lead to incorrect conclusions. To address this, recent advancements have sought to infer the correct CI relationship between the latent variables through binarizing observed data. However, this process inevitably results in a loss of information, which degrades the test’s performance. Motivated by this, this paper introduces a sample-efficient CI test that does not rely on the binarization process. We find that the independence relationships of latent continuous variables can be established by addressing an over-identifying restriction problem with Generalized Method of Moments (GMM). Based on this insight, we derive an appropriate test statistic and establish its asymptotic distribution correctly reflecting CI by leveraging nodewise regression. Theoretical findings and Empirical results across various datasets demonstrate that the superiority and effectiveness of our proposed test. Our code implementation is provided in https://github.com/boyangaaaaa/DCT . 1 Introduction Conditional independence tests for discrete variables are fundamental in statistical analysis and widely applied across various disciplines. Traditional methods including the chi-squared test ( F.R.S. 2009 ) , the G-test (likelihood ratio test) ( McDonald 2009 ) , and measures based on conditional mutual information ( Kubkowski et al. 2021 ) are well established and extensively used. Figure 1 : Illustration of data generative processes using causal graphical models: (a) fork, (b) and (c) chain. The discretization process maps latent continuous variables (white nodes) to observable discrete variables (gray nodes), denoted with a tilde ( \sim ). However, a critical yet often overlooked issue is whether the analyzed variables are truly discrete or if they are inherently continuous but appear discrete due to measurement limitations . In many real-world applications, data collection methods impose unavoidable discretization. That is, continuous variables are artificially binned into categories because of constraints in measurement precision. This phenomenon across various fields, including finance ( Changsheng & Yongfeng 2012 ; Damodaran 2012 ) , psychology ( Mossman et al. 2017 ; Johnson et al. 2019 ) , and recommendation systems ( Sparling & Sen 2011 ; Dooms et al. 2013 ) . In these domains, inherently continuous variables—such as stock prices, cognitive ability scores, and user preferences—are frequently transformed into discrete scales, often leading to biases in statistical inference. When discretization occurs, traditional CI tests can fail to capture the true CI relationship. As shown in Figure 1 where we illustrate the discretization process with a causal graphic model ( Pearl 2000 ) , for all CI tests unaware of discretization, the intent is to test the CI of latent continuous variables X_{1},X_{3} given X_{2} , what is actually being tested is their discretized counterparts \tilde{X}_{1},\tilde{X}_{3} given \tilde{X}_{2} . According to the faithfulness assumption ( Spirtes et al. 2000 ) , we can infer that X_{1}\perp\!\!\!\perp X_{3}|\{X_{2}\} while \tilde{X}_{1}\not\perp\!\!\!\perp\tilde{X}_{3}|\{\tilde{X}_{2}\} . This mismatch between the latent continuous variables and their discretized counterparts causes traditional CI tests, when applied to discretized observations, to draw incorrect conclusions about the true CI relationships of interested variables. Recent work Discretization-Aware CI test (DCT) ( Sun et al. 2024 ) has attempted to address this issue by establishing the correct relationship between discretized data and latent variables through binarization of the observed data. While this approach facilitates more accurate CI testing by simplifying the data structure, it inherently leads to a loss of information. The reduction of data to binary form can significantly degrade the performance of CI tests, especially in settings with small sample sizes where the preservation of information is crucial for reliable statistical inference. Motivated by the limitations of existing methodologies, this paper aims to introduce a sample-efficient CI test that circumvents the need for binarization, thereby preserving the full richness of the data. Our approach leverages the Generalized Method of Moments (GMM) to address the over-identifying restrictions problem, enabling the estimation of covariance of latent continuous variables without sacrificing information. We then adopt the strategy of DCT to derive test statistics and their asymptotic distribution for CI testing, utilizing nodewise regression ( Callot et al. 2019 ) . The paper seeks to contribute as follows: • We propose Discretization-Aware CI Test with GMM (DCT-GMM), a novel CI test tailored for discretization. • We provide a theoretical analysis proving that DCT-GMM is consistent and has lower variance than DCT, making it more sample efficient. ( Ziegel 2002 ) . • We empirically demonstrate DCT-GMM’s effectiveness and superiority over state-of-the-art CI tests, particularly in small-sample regimes. 2 Related Work Conditional Independence Test Testing for CI is a fundamental concept in statistics, with linear Gaussian models traditionally dominating due to their simplicity and interperability. These models assume linear dependencies and Gaussian noise, providing closed-form solutions for testing through metrics like partial correlation ( Yuan & Lin 2007 ; Peterson et al. 2015 ; Mohan et al. 2012 ; Ren et al. 2015 ) . However, the linear Gaussian assumption restricts the generality. Recent CI testing advancements leverage kernel methods for nonlinear continuous relationships ( Fukumizu et al. 2004 ) . Methods like KCI ( Zhang et al. 2012 ) and RCI ( Strobl et al. 2019 ) analyze partial associations, while KCIP ( Doran et al. 2014 ) employs sample permutations to simulate CI. For discrete variables, G^{2} ( Aliferis et al. 2010 ) and conditional mutual information ( Zhang et al. 2010 ) are standard tests. A recent advance on permutation-based rank test, MPRT ( Dong et al. 2025 ) , can also be used to test vanishing partial correlation in the presence of discretization. Prior work DCT ( Sun et al. 2024 ) moves the first step towards the CI test specifically for the discretization scenario. Their approach can be decomposed into three steps: 1. The calculation of estimated covariance \hat{\bm{\Sigma}} , based on the property that the proportion of both observed variables exceeding their means reflects the underlying covariance, solved using a single equation . 2. The deviation of covariance matrix \hat{\bm{\Sigma}}-\bm{\Sigma}^{*} follows a multivariate normal distribution utilizing Z-estimator ( Vaart 1998 ) . 3. The deviation of precision matrix \hat{\bm{\Omega}}-\bm{\Omega}^{*} also follows a multivariate normal distribution utilizing node-wise regression ( Callot et al. 2019 ) . However, despite having multiple solvable equations from the discretized observations, only one parameter of interest exists per variable pair, leading to an over-identification issue. Efficiently utilizing all available information is the key challenge, motivating us to explore the usage of GMM. Generalized Method of Moments GMM ( Newey 2007 ; Hansen 1982 ) is a statistical estimation technique offering a principal solution to the over-identification problem. Suppose \bm{\theta} denotes a p\times 1 vector, x^{i} denotes the observation of a data sample where i\in(1,\dots,n) is the index. f_{i}(\bm{\theta})=f(x^{i},\bm{\theta}) be a m\times 1 vector of functions. For the true parameter \bm{\theta}^{*} , we will have \mathbb{E}[f_{i}(\bm{\theta}^{*})]=\mathbf{0}. Using \hat{g}(\bm{\theta})=\frac{1}{n}\sum_{i=1}^{n}f_{i}(\bm{\theta}) denote the sample average of the f_{i}(\bm{\theta}) . The \mathbf{A} is a m\times m positive semi-definite matrix. The GMM estimator is given by \hat{\bm{\theta}}=\text{arg}\displaystyle\min_{\bm{\theta}}\quad\hat{g}(\bm{\theta})^{T}\mathbf{A}\hat{g}(\bm{\theta}), providing a framework for valid inference ( Newey 2007 ) . In our case, by

Read the complete text