Correcting for Position Bias in Learning to Rank: A Control Function Approach DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06
Md Aminul Islam, Kathryn Vasilaky, Elena Zheleva · 2025
Read Correcting for Position Bias in Learning to Rank: A Control Function Approach DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 by Md Aminul Islam, Kathryn Vasilaky, Elena Zheleva (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06989.txt.
Introduction
Md Aminul Islam, Kathryn Vasilaky, and Elena Zheleva fix position bias in search ranking in a 2025 paper. Ranking systems learn from clicks, but users click top results more regardless of merit, which bends the training data. Standard fixes estimate click chances per slot, a brittle step.
Their control function framework works in two stages. First it captures leftover variation from the ranking process itself. Then that signal enters the click model to strip out slot effects, with no per-slot propensity scores needed. The method fits nonlinear rankers and plugs into standard algorithms. A companion trick cleans validation data for tuning. On benchmark and industry data, the approach beat leading bias-correction methods.
Catalog record
| CREATOR | Md Aminul Islam, Kathryn Vasilaky, Elena Zheleva |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-06989
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Correcting for Position Bias in Learning to Rank: A Control Function Approach DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 Thanks: This is a preprint version. CCS: Information systems Learning to rank Md Aminul Islam, Kathryn Vasilaky, Elena Zheleva Abstract . Implicit feedback data, such as user clicks, is commonly used in learning-to-rank (LTR) systems because it is easy to collect and it often reflects user preferences. However, this data is prone to various biases, and training an LTR system directly on biased data can result in suboptimal ranking performance. One of the most prominent and well-studied biases in implicit feedback data is position bias, which occurs because users are more likely to interact with higher-ranked documents regardless of their true relevance. In this paper, we propose a novel control function-based method that accounts for position bias in a two-stage process. The first stage uses exogenous variation from the residuals of the ranking process to correct for position bias in the second stage click equation. Unlike previous position bias correction methods, our method does not require knowledge of the click or propensity model and allows for nonlinearity in the underlying ranking model. Moreover, our method is general and allows for debiasing any state-of-the-art ranking algorithm by plugging it into the second stage. We also introduce a technique to debias validation clicks for hyperparameter tuning to select the optimal model in the absence of unbiased validation data. Experimental results demonstrate that our method outperforms state-of-the-art approaches in correcting for position bias. 1. Introduction Learning-to-rank (LTR) systems have become a core component in many online applications, from search engines to recommender systems. These systems typically use implicit feedback data, such as user clicks, as a proxy for user preferences due to its ease of collection compared to costly manual human annotations. While valuable, implicit feedback data is prone to biases ( Ai et al. 2018 ; Yue et al. 2010 ) , with position bias being particularly pervasive. Position bias is the tendency of users to interact more with higher-ranked documents, regardless of their true relevance ( Craswell et al. 2008 ; Joachims et al. 2007 ) . This skews click data, causing LTR models to overestimate higher-ranked documents and underestimate lower ones. Training on such biased data degrades ranking performance and reinforces a feedback loop that negatively affects user experience. State-of-the-art methods that address position bias can be broadly categorized as click models, propensity-based approaches, and econometrics-based models. Click models ( Chapelle and Zhang 2009 ; Craswell et al. 2008 ; Dupret and Piwowarski 2008 ; Wang et al. 2013 ) estimate true relevance of documents by hypothesizing how users browse and maximizing the likelihood of observed clicks. Click models rely on assumptions about complex user click behavior, which sometimes may not hold in practice. Two-tower models ( Guo et al. 2019 ; Yan et al. 2022 ) , following the position-based click model (PBM) ( Richardson et al. 2007 ) , attempt to disentangle relevance and bias from clicks by modeling them separately, but may fail when confounding exists between the two. Propensity-based methods treat clicks as counterfactual events and re-weight them using inverse propensity weighting (IPW) to achieve relevance-equivalent training loss ( Joachims et al. 2017 ; Wang et al. 2016 ) . Propensity estimation often relies on result randomization ( Joachims et al. 2017 ; Oosterhuis and de Rijke 2020 ; Wang et al. 2016 ) , which can harm user experience ( Ai et al. 2018 ; Wang et al. 2018 ) . Some methods ( Ai et al. 2018 ; Hu et al. 2019 ; Luo et al. 2024 ; Vardasbi et al. 2020 ; Zhu et al. 2020 ) jointly learn propensities and rankers, but coupling the two raises concerns about the unbiasedness if either model is misspecified ( Agarwal et al. 2019 ; Fang et al. 2019 ; Tian et al. 2020 ) . Doubly robust (DR) IPW-based bias correction methods ( Luo et al. 2023 ; Oosterhuis 2023 ) have also been proposed recently, where the DR estimator combines an IPW estimate with predictions from a regression model and becomes unbiased if either the estimated propensity or the regression model is accurate. Most click modeling and propensity-based methods require modifying the ranker’s loss function, which makes them tightly coupled with a specific LTR algorithm and limits their generalization to other LTR algorithms. Alternative approaches ( Ovaisi et al. 2020 ; Ovaisi et al. 2021 ) , based on Heckman’s two-stage econometric model ( Heckman 1979 ) , avoid propensity estimation but assume linear ranking models. However, linear models often struggle to capture complex feature interactions and handle the sparse click data typical in LTR systems. Addressing data bias has a long tradition in the econometrics literature. A prominent class of approaches for addressing various types of biases is control functions ( Wooldridge 2015 ) which includes a number of related well-known econometric techniques, such as Heckman selection ( Heckman 1979 ) and endogenous switching models ( Maddala and Nelson 1975 ) ; shares the same conceptual foundations with instrumental variables ( Reiersøl 1941 ) ; and exploits identification via orthogonal moments akin to, the more recent, double machine learning ( Chernozhukov et al. 2018 ) . Control functions are two-stage methods which model bias through the endogeneity in the error term. In this paper, we introduce a control function for addressing position bias. In the first stage, we model the system’s previous ranking process, which generated the user feedback data based on observable features, to determine document positions in a ranked list for a query. In the second stage, we leverage the exogenous variation in first-stage residuals as a control function within the ranking model to adjust for position bias. We introduce residual transformation as a hyperparameter in the second stage and suggest several carefully designed transformation options. A control function approach for addressing position bias in LTR systems offers a number of advantages over existing approaches. Unlike Heckman-based methods ( Ovaisi et al. 2020 ; Ovaisi et al. 2021 ) , which require linear models in both stages, our control function method allows for any linear or non-linear models in both stages. Additionally, by regularizing the feature space during the modeling of the document ranking process, our method addresses multicollinearity issues with high-dimensional feature spaces that Heckman-based methods do not. While propensity-based methods ( Ai et al. 2018 ; Hu et al. 2019 ; Joachims et al. 2017 ; Luo et al. 2024 ; Luo et al. 2023 ; Oosterhuis and de Rijke 2020 ; Vardasbi et al. 2020 ; Wang et al. 2016 ; Wang et al. 2018 ) reweight click data using IPW to account for bias, our method circumvents the need for propensity estimation. We estimate query-document-specific residuals by modeling the system’s previous document ranking process to account for position bias. Our method also does not require result randomization ( Joachims et al. 2017 ; Oosterhuis and de Rijke 2020 ; Wang et al. 2016 ) . Unlike click modeling methods ( Chapelle and Zhang 2009 ; Craswell et al. 2008 ; Dupret and