Design-Based Inference under Random Potential Outcomes
Yukai Yang · 2025
Read Design-Based Inference under Random Potential Outcomes by Yukai Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01324.txt.
Introduction
Yukai Yang asked in 2025 whether one randomised trial can reveal the true average effect of a treatment. That average is taken over hidden random settings, yet one trial shows only one draw of those settings. A sound design alone does not promise the truth can be recovered.
Yang shows that when units depend only weakly on near neighbours, a helpful averaging property appears. Averaging across many units then recovers the average over hidden settings. The usual trial estimates stay accurate, their spread can be pinned down from the single trial, and the estimates settle toward a bell shape. That spread result breaks with classic practice, where only cautious upper bounds were possible.
Catalog record
| CREATOR | Yukai Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 99 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-01324
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Design-Based Inference under Random Potential Outcomes Yukai Yang Abstract We study whether mechanism-level causal estimands, defined as expectations over latent stochastic environments, can be consistently recovered from a single realised randomised experiment. Identification alone does not guarantee recoverability. The target estimand averages over latent environments, whereas a single experiment provides only one realisation of such an environment. We show that suitably sparse local dependence induces an ergodic-type property under which cross-sectional averaging consistently recovers expectations over the latent outcome-generating mechanism. Under this structure, aggregate design-based estimators are consistent and asymptotically normal, and the variance becomes consistently estimable from a single experiment. Unlike classical finite-population inference, where Neyman-type variance estimators are structurally limited to conservative upper bounds, the proposed framework permits consistent variance estimation through the shift from fixed potential outcome schedules to stochastic mechanisms. Keywords: Recoverability; Mechanism-level estimands; Local dependence; Variance estimation; Dependency graphs. 1 Introduction In randomised experiments, researchers are typically interested in whether an observed treatment effect reflects a stable underlying mechanism rather than a favourable realisation of a particular experimental environment. This question arises naturally in applications such as randomised controlled trials, A/B testing, industrial experimentation, and policy evaluation, where the objective is not merely to describe one realised experiment but to understand the broader mechanism generating such effects across hypothetical repetitions of the experiment. The classical fixed potential outcome (FPO) framework, however, provides no mechanism for addressing this distinction. In the classical design-based framework of 17 and Rubin (1974) , potential outcomes are treated as fixed quantities, and randomness arises solely through the treatment assignment mechanism. Under this FPO formulation, the canonical estimand is \tau_{\mathrm{FPO}}=\frac{1}{n}\sum_{i=1}^{n}\bigl\{Y_{i}(1)-Y_{i}(0)\bigr\}, (1.1) where the realised potential outcome schedule is regarded as fixed throughout the experiment. The framework therefore conditions on a particular realised experimental world and does not distinguish between inference conditional on that realised world and recoverability of a mechanism-level quantity defined across stochastic environments. Design-based identification is achieved directly through the randomisation design without requiring a probabilistic model for the outcome process. This framework has been extended to settings involving interference, network dependence, and complex experimental designs ( Aronow and Samii, 2017 ; Sävje et al., 2021 ; Athey et al., 2021 ) . These developments preserve the FPO assumption and continue to define causal estimands conditionally on realised potential outcome schedules. Consequently, the object of inference remains tied to a realised experimental environment rather than to an underlying stochastic mechanism. In this paper, we consider a random potential outcome (RPO) formulation. The potential outcome for unit i is represented by \tilde{y}_{i}(z,\omega) , which depends jointly on the treatment assignment z and a latent stochastic environment \omega . A canonical expectation-based estimand under this formulation is \tau_{\mathrm{RPO}}=\mathbb{E}_{\omega}\left[\frac{1}{n}\sum_{i=1}^{n}\bigl\{\tilde{y}_{i}(1,\omega)-\tilde{y}_{i}(0,\omega)\bigr\}\right]. (1.2) More generally, the paper studies mechanism-level treatment-effect functionals defined through expectations over latent stochastic environments rather than conditionally on realised potential outcome schedules. This differs from model-assisted approaches ( Rubin, 1978 ; Rosenbaum and Rubin, 1983 ; Chernozhukov et al., 2018 ) , which also allow stochastic outcome structures but rely on modelling assumptions on the outcome or assignment mechanism rather than on the randomisation design itself. The classical FPO framework is recovered as the degenerate special case in which the latent environment reduces to a singleton. In this degenerate case, the recoverability problem disappears because no averaging over latent stochastic environments is required. It is useful to distinguish the present framework from sampling-based or superpopulation formulations ( Cassel et al., 1977 ; Little, 2004 ; Abadie et al., 2020 ) . Those approaches study population-level quantities under sampling uncertainty, where randomness arises through the sampling mechanism. By contrast, the estimands considered here are expectations over latent stochastic environments and therefore represent mechanism-level quantities rather than population-level quantities. Consequently, the inferential objective differs fundamentally from that of sampling-based formulations. In particular, the local-dependence and neighbourhood-growth conditions developed in Section 3 address recoverability of expectations over latent stochastic environments, a problem that is not characterised within the sampling-based framework. Once causal estimands are defined as expectations over latent stochastic environments, a new inferential problem emerges. Although the estimand is defined through averaging over \omega , only one realised experiment corresponding to one draw of \omega is observed. Identification alone therefore does not guarantee recoverability from a single realised experiment. The central inferential question becomes whether cross-sectional averaging across units can asymptotically recover expectations over the latent stochastic environment. Recoverability fundamentally depends on the dependence structure induced by the latent environment. If all units share a single common latent shock, increasing the sample size provides no additional information about expectations over the stochastic environment. Cross-sectional averaging cannot recover mechanism-level quantities because the experiment still contains only one realised draw of \omega . Additional probabilistic structure is therefore required to connect cross-sectional aggregation to averaging over latent stochastic environments. In the design-based experimental settings considered here, such structure often arises naturally from the experimental design itself through clustering, spatial organisation, controlled interaction patterns, or partially engineered dependence structures. We show that suitably sparse local dependence provides such a bridge. Under the conditions developed below, local dependence induces an ergodic-type property under which cross-sectional averaging asymptotically recovers expectations over the latent stochastic environment. A single realised experiment therefore becomes sufficient for asymptotically valid inference on mechanism-level causal estimands. The resulting asymptotic analysis is conducted on the joint probability space generated by treatment assignment and latent outcome variation. In canonical discrete-treatment settings, the resulting estimator coincides algebraically with familiar unbiased estimators from the FPO framework, including the Horvitz–Thompson estimator ( Horvitz and Thompson, 1952 ) . Under the stochastic framework developed here, however, the same statistic targets the mechanism-level estimand \tau_{\mathrm{RPO}} and satisfies asymptotic normality under local dependence. A further contribution of this paper concerns variance estimation. In the classical FPO framework, Neyman-type variance estimators are structurally limited to conservative upper bounds rather than consistent estimators of the variance itself. This limitation is not a technical artefact but a consequence of finite-population reasoning under FPO schedules. Within the proposed stochastic framework, this structural limitation can be overcome. By exploiting the averaging structure induced by local dependence, we develop feasible variance estimators that are consistent under stochastic outcome variation and account jointly for randomisation and latent outcome stochasticity. This permits asymptotically valid inference from a single realised experiment. The resulting improvement arises from shifting the inferential target from fixed realised schedules to stochastic outcome-generating mechanisms rather than from technical refinement within the finite-population framework itself. Treatment-effect functionals admitting continuous linear representations remain identifiable through the randomisation design under the proposed framework. Our representer-based construction draws on the use of representers for identification in design-based settings developed by Harshaw et al. (2022) . Their framework studies identification of treatment-effect functionals under FPO. By contrast, the present paper studies recoverability and asymptotic inference for expectation-based causal estimands under random potential outcomes. The central question is not whether a treatment-effect functional is identified, but whether a mechanism-level estimand defined through expectations over latent stochastic environments can be consistently recovered from a single realised experiment. This recoverability problem is specific to estimands defined through averaging over latent stochastic environments and therefore has no direct counterpart within the FPO setting, where inference is conditional on a realised potential-outcome schedule. In this sense, representer