Double Machine Learning for Causal Inference under Shared-State Interference

Chris Hays, Manish Raghavan · 2025

Read Double Machine Learning for Causal Inference under Shared-State Interference by Chris Hays, Manish Raghavan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-08836.txt.

Introduction

Chris Hays and Manish Raghavan present a 2025 statistics paper on measuring cause and effect when people influence each other through shared channels. In markets and recommendation systems, one person's outcome depends on shared signals like prices or suggested items, which breaks standard trial logic. The pair formalize this setting and call it shared-state interference.

Their key assumption is that, once the shared state is known, people's possible outcomes are independent. On that basis they extend a workhorse method, double machine learning, to give sound estimates in such settings. They work out the details for several concrete models, showing how to measure both the direct effect of a treatment and its total effect across the whole system.

Catalog record

CREATORChris Hays, Manish Raghavan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE128 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-08836

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Double Machine Learning for Causal Inference under Shared-State Interference Chris Hays, Manish Raghavan Abstract Researchers and practitioners often wish to measure treatment effects in settings where units interact via markets and recommendation systems. In these settings, units are affected by certain shared states , like prices, algorithmic recommendations or social signals. We formalize this structure, calling it shared-state interference, and argue that our formulation captures many relevant applied settings. Our key modeling assumption is that individuals’ potential outcomes are independent conditional on the shared state. We then prove an extension of a double machine learning (DML) theorem providing conditions for achieving efficient inference under shared-state interference. We also instantiate our general theorem in several models of interest where it is possible to efficiently estimate the average direct effect (ADE) or global average treatment effect (GATE). \svgsetup inkscapelatex=false, 1 Introduction In causal inference, interference — where individuals’ treatment assignments, outcomes or other characteristics impact others’ outcomes — is everywhere: On short-form video platforms, the consumption of one individual is used as an input to recommendation algorithms that are subsequently used to serve content to others. In ride-sharing or housing rental services, providing a discount to some people can cause them to increase their usage of the service and increase wait times or prices. In such settings, failure to account for interference may lead to biased estimates of causal effects, even in randomized controlled trials. Across many settings, like those of recommender systems and markets, interference between units often follow a common pattern: the outcomes of individuals depend on others through some shared state . In recommender systems, the shared state might be the outputs of recommender systems that are used to generate users’ feeds. In marketplaces, the shared state may be prices, wait times, or availability of inventory. In each of these settings, individuals both influence and are influenced by the shared state. Assumptions are required to perform causal inference in the presence of interference. Prior work has studied this kind of interference in particular applications ( Johari et al., 2021 ; Basse et al., 2016 ; Wager, 2024 ; Munro, 2024 ; Simchi-Levi and Wang, 2023 ; Dhaouadi et al., 2023 ; Farias et al., 2023 ; Farias et al., 2022 ; Li et al., 2024 ) . In some cases, they propose either experimental designs or analysis methods that account for interference and allow for valid inference. However, these approaches typically assume a parametric model of the system or domain-specific assumptions. Our contributions. We provide a general framework to reason about causal inference under shared-state interference. We define a formal model in Section 2 , where individuals arrive sequentially. Their covariates (i.e., unit characteristics), treatment assignments, and outcomes may influence the outcomes of future individuals through some shared state. In Section 3 , we provide conditions under which efficient inference is possible in the framework. We extend methods from double machine learning (DML) ( Chernozhukov et al., 2018 ) , which has shown that it is possible to achieve efficient inference using expressive machine learning methods without parametric functional form assumptions. A key assumption we rely on is that the shared state progresses according to a Markov chain with particular properties. We also provide a consistent variance estimator, which is necessary for constructing valid confidence intervals and running hypothesis tests. Next, we instantiate our framework with two applications: first, we show how our framework can be used to estimate average direct effects (ADE) in observational settings; then, we provide a variance reduction strategy for estimation of the global average treatment effects (GATE) in switchback experiments. Intuitively, the ADE measures the expected difference between treatment and control for a unit drawn uniformly at random, keeping the treatment assignment distributions of other units fixed. The GATE measures the mean difference between outcomes when all units are assigned to treatment versus when all are assigned to control. Key to our approach in these sections is constructing structural models that capture settings of interest and estimators for the treatment effects which satisfy the properties necessary to apply the DML theorem from Section 3 . In each instantiation, we provide simulations validating that our method produces estimators that concentrate around the true treatment effects in finite samples. We also show our consistent variance estimators can be used to construct confidence intervals with the desired coverage probability. 1.1 Related work. Our work sits at the intersection of the study of inference under interference and double machine learning (DML) methods. Shared-state interference. Several prior works have explored causal inference in settings captured by or similar to our formulation of shared-state interference ( Johari et al., 2021 ; Basse et al., 2016 ; Wager, 2024 ; Munro, 2024 ; Holtz et al., 2024 ; Simchi-Levi and Wang, 2023 ; Dhaouadi et al., 2023 ; Farias et al., 2023 ; Farias et al., 2022 ; Li et al., 2024 ; Brennan et al., 2024 ; Bajari et al., 2023 ; Bright et al., 2023 ) . Some of these papers have analyzed bias resulting from naive estimators or experimental designs and proposed less biased alternatives ( Johari et al., 2021 ; Farias et al., 2022 ; Brennan et al., 2024 ) . Other works have proposed parametric models of settings that allow for inference in the presence of interference ( Wager and Xu, 2021 ; Li et al., 2024 ) . Several works consider a Markov model of interference similar to ours in an experimental setting and explore less biased estimation methods ( Farias et al., 2023 ; Farias et al., 2022 ; Glynn et al., 2022 ) . Double machine learning. The double machine learning framework was introduced in Chernozhukov et al. (2018) , drawing on a rich literature in semiparametric statistics (see, e.g., Kennedy (2023) for an overview). DML methods have been extended to several other settings, such as those with continuous treatments ( Kennedy et al., 2017 ) , estimation of quantile effects ( Kallus et al., 2022 ) and settings with limited unobserved confounding ( Rambachan et al., 2024 ) . A few works, like ours, have developed semiparametric methods for settings with interference. Several works have explored interference channeled through individuals’ social networks ( Emmenegger et al., 2023 ; Ogburn et al., 2024 ) . Our setting is different because each unit may be dependent on each of the units before it (whereas their network-based approaches require the network to be sparse), and interference in our setting is anonymous (i.e., independent of the identities of the units). Munro (2024) develops a DML theorem for settings like auctions where interference is channeled through a centralized allocation mechanism. Zhan et al. (2024) develops a DML theorem for a discrete choice model of content consumption under a neural network-based recommender system. In contrast, our work relaxes these assumptions and instead requires a general Markovian assumption. Ballinari and Wehrli (2024) develops a DML theorem for a time-series model where a single unit is observed over time, but where observations across time obey a mixing condition. Their method allows for estimating impulse response functions, which measure the effect of an intervention at a particular time on the outcome at a future time. Our work also relies on mixing

Read the complete text