Addressing Correlated Latent Exogenous Variables in Debiased Recommender Systems Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 3–7, 2025; Toronto, ON, Canada. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data
Shuqiang Zhang, Yuchao Zhang, Jinkun Chen, Haochen Sui · 2025
Read Addressing Correlated Latent Exogenous Variables in Debiased Recommender Systems Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 3–7, 2025; Toronto, ON, Canada. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data by Shuqiang Zhang, Yuchao Zhang, Jinkun Chen, Haochen Sui (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07517.txt.
Introduction
Recommenders learn from what users chose to click, so their picture of taste is skewed from the start. Shuqiang Zhang and three co-authors attack a hidden part of that skew in a 2025 paper. The unseen outside factors behind the data may correlate with each other, an assumption earlier debiasing methods quietly deny. They rebuild the data-generating story with those linked hidden factors included.
Their method fits the model by maximizing likelihood, estimating the hard integrals with a Monte Carlo routine. On synthetic data and three real datasets, it beat earlier approaches. The authors distinguish their setting from plain unmeasured confounding and release their code.
Catalog record
| CREATOR | Shuqiang Zhang, Yuchao Zhang, Jinkun Chen, Haochen Sui |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-07517
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Addressing Correlated Latent Exogenous Variables in Debiased Recommender Systems Conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 3–7, 2025; Toronto, ON, Canada. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD ’25), August 3–7, 2025, Toronto, ON, Canada ISBN: 979-8-4007-1454-2/25/08 DOI: 10.1145/3711896.3736832 CCS: Information systems Recommender systems Shuqiang Zhang, Yuchao Zhang, Jinkun Chen, Haochen Sui Abstract . Recommendation systems (RS) aim to provide personalized content, but they face a challenge in unbiased learning due to selection bias, where users only interact with items they prefer. This bias leads to a distorted representation of user preferences, which hinders the accuracy and fairness of recommendations. To address the issue, various methods such as error imputation based, inverse propensity scoring, and doubly robust techniques have been developed. Despite the progress, from the structural causal model perspective, previous debiasing methods in RS assume the independence of the exogenous variables. In this paper, we release this assumption and propose a learning algorithm based on likelihood maximization to learn a prediction model. We first discuss the correlation and difference between unmeasured confounding and our scenario, then we propose a unified method that effectively handles latent exogenous variables. Specifically, our method models the data generation process with latent exogenous variables under mild normality assumptions. We then develop a Monte Carlo algorithm to numerically estimate the likelihood function. Extensive experiments on synthetic datasets and three real-world datasets demonstrate the effectiveness of our proposed method. The code is at https://github.com/WallaceSUI/kdd25-background-variable . 1. Introduction Recommendation systems (RS) have become popular tools for providing personalized content across various domains, including education, e-commerce, and entertainment, as they effectively alleviate the information overload ( Marlin and Zemel 2009 ; Liu et al. 2024 ; Wang et al. 2024b ; Li and Sui 2025 ) . However, a significant issue arises from biases present in user interaction data, particularly selection bias ( Chen et al. 2023 ; Wang et al. 2023b ; Wu et al. 2022 ; Zheng et al. 2025 ) . This bias stems from the fact that users tend to engage only with items that align with their preferences, leading to a distorted representation of their true interests. Such biases undermine the accuracy and fairness of recommendation models, posing a challenge to unbiased learning ( Liu et al. 2022 ; Saito and Nomura 2022 ; Wang et al. 2019 ; Li et al. 2023c ; Li et al. 2023b ) . In response to these challenges, several strategies have been developed to mitigate biases and enhance the quality of recommendations. One prominent approach is the Error Imputation Based (EIB) method ( Marlin and Zemel 2009 ; Steck 2010 ) , which treats the selection bias as a data missing problem, filling in the gaps with pseudo labels for prediction model training. Furthermore, causality-based techniques ( Huang et al. 2025b ; Huang et al. 2025a ; Zhou et al. 2025 ; Wang et al. 2025a ) provide a promising direction for addressing these issues ( Wang et al. 2023a ; Wang et al. 2024a ; Chen et al. 2024 ; Wang et al. 2025c ) . Inverse Propensity Scoring (IPS) ( Luo et al. 2021 ; Schnabel et al. 2016 ) , addresses selection bias by estimating the propensity score, which refers to the probability of a unit being observed and adjusting the weights of observed interactions to better reflect the true underlying distribution. Further advances have led to the development of the Doubly Robust (DR) method ( Wang et al. 2019 ; Saito 2020 ; Wang 2024 ) , which combines both error imputation and inverse propensity weighting. This approach not only reduces bias but also enhances the robustness of the model, making it more reliable across a range of data scenarios. Building upon these foundational methods, recent research has proposed several improvements, including the integration of advanced propensity score learning strategy ( Wang et al. 2021 ; Chen et al. 2021 ) , the use of information-theoretic principles ( Wang et al. 2020 ; Cao et al. 2022 ) , and balancing of bias and variance ( Guo et al. 2021 ; Li et al. 2023e ; Wu et al. 2024 ) . However, when modeling the treatment and outcome using observed features, previous methods assume the independence of exogenous variables. Specifically, the Structural Causal Model (SCM) of previous methods is defined by a triple (U,V,F) , where V are observable endogenous variables (covariate X , the outcome observation indicator O , and outcome R ) and U are unobserved exogenous variables that cannot be caused by any variable in V , including U_{X},U_{O}~\text{and}~U_{R} . Previous methods assume that all the exogenous U are required to be jointly independent. In this paper, we aim to relax the above independence assumption between exogenous variables, as shown in Figure 1 (right). Before introducing our method, we want to first discuss the correlation and difference of previous graphical models under traditional unmeasured confounding (Figure 1 (left)) and scenario in our paper (Figure 1 (right)). Specifically, for the traditional unmeasured confounding scenario, U is an unmeasured endogenous variable, not an exogenous variable. In other words, the data generation process is O=f_{O}(X,U,U_{O}) and R=f_{R}(X,U,U_{R}) with unmeasured endogenous variable U , leading to a biased prediction if we only use variable X . For dependent exogenous variables scenario, U_{O} and U_{R} are correlated exogenous variables of endogenous variables O and R , i.e., O=f_{O}(X,U_{O}) and R=f_{R}(X,U_{R}) but with the correlated background variables U_{O} and U_{R} . Figure 1. Comparison between traditional unmeasured confounding (left) and latent exogenous variables (right). Dashed lines represent variables that are unobserved or latent. In this paper, to train an unbiased prediction model in a correlated latent exogenous variables scenario, we propose a framework that incorporates exogenous variables into recommendation modeling, mitigating selection bias and improving robustness. Specifically, our method models the data generation process with latent exogenous variables under mild normality assumptions. We then develop a Monte Carlo algorithm to numerically estimate the likelihood function. Our contributions can be summarized below: • This paper considers a different unmeasured confounding case and discusses the correlation and difference with traditional unmeasured confounding scenario. • We develop a Monte Carlo algorithm to numerically estimate the likelihood function based on the symmetry of the endogenous variables to handle latent exogenous variables. • Extensive experiments on both synthetic and real-world datasets demonstrate that our method successfully addresses the latent exogenous variable correlation problem. 2. Related Work 2.1. Debiased Recommendation 2.1.1. Debiasing Methods without Unmeasured confounding Recommendation systems (RS) provide personalized recommendations for each user and have been widely applied in various fields such as education, commerce, and entertainment ( Marlin and Zemel 2009 ) , often with time series data ( Lai et al. 2024 ) , in order to address the information overload problem. However, the user interaction data inevitably contain biases, such as selection bias, where users interact only with items they prefer, resulting in a biased selection that challenges the unbiased learning ( Chen et al. 2023 ; Wang et al. 2023b ; Wu et al. 2022 ) . To address these issues, a line of methods are proposed