Untangling Sample and Population Level Estimands in Bayesian Causal Inference

\name Arman Oganisian \email [email protected] · 2025

Read Untangling Sample and Population Level Estimands in Bayesian Causal Inference by \name Arman Oganisian \email [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15016.txt.

Introduction

Bayesian methods for cause and effect can target effects in the sample at hand or in the wider group. Oganisian warns that standard computing routines may silently aim at the wrong one. The paper untangles the two targets across four steps: stating the target, modeling, computing, and reading the results.

Sample-level targets often need cross-world modeling and direct simulation of what-if outcomes from their joint posterior. Group-level targets usually need only a posterior over the model inputs, sometimes with a Monte Carlo step afterward. Four examples, some with flexible models, show how near-identical steps can produce draws of quite different quantities and so support wrong conclusions. The paper closes with common mistakes and advice on choosing the right target first.

Catalog record

CREATOR\name Arman Oganisian \email [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-15016

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Untangling Sample and Population Level Estimands in Bayesian Causal Inference \name Arman Oganisian \email [email protected] Abstract Bayesian inference for causal estimands has been growing in popularity, however many misconceptions and implementation errors arise from conflating sample and population-level estimands and inference. We have anecdotally witnessed these at conference talks, in the course of peer review service, and even in published and arXiv-ed papers. Our goal here is to elucidate the crucial differences between sample and population-level quantities when it comes to identification, modeling, Bayesian computation, and interpretation. For example, common sample-level estimands require cross-world assumptions for identification, whereas common population-level estimands do not. Similarly, the former requires explicit imputation of counterfactuals from their joint posterior, whereas the latter typically only requires a posterior distribution over parameters. We start by defining some examples of both types of estimands, then discuss the full joint posterior over all unknowns (both missing counterfactuals and population distribution parameters). We continue to outline how inference for different estimands are derived from different marginals of this joint posterior. Because the differences are conceptually subtle but can be practically substantial, we provide an illustration of using synthetic data in Stan 1 1 1 https://github.com/stablemarkets/untangle . We also provide a detailed appendix with derivations and computational tips along with a discussion of common implementation errors. The overarching message here is to always engage in first-principles thinking about which marginal of the joint posterior is of interest in a particular application, then follow the strict logic of Bayes’ theorem and probability to avoid common implementation errors. Keywords: Bayesian inference, causal inference, missing data 1 Data Structure and notation Throughout, we assume familiarity with basic causal inference concepts in the potential outcomes framework and Bayesian probability calculations in the point-treatment setting. We consider a setting with an observed continuous, real-valued outcome, Y_{i} , for subject i=1,2,\dots,n , a continuous real-valued confounder L_{i} , and a binary treatment indicator A_{i}\in\{0,1\} . We let Y_{i}(a) denote the potential outcome for subject i under treatment a\in\{0,1\} . Throughout we use capital letters to denote random variables and lowercase to denote realized values. We assume the following: stable unit treatment value assumption (SUTVA), Y=(1-A)Y(0)+AY(1) ; joint conditional exchangeability, Y(1),Y(0)\perp A\mid L ; and positivity, P(A=1\mid L=l)>0 for all l for which f_{L}(l)>0 . Under SUTVA, each subject’s observed outcome is their factual potential outcome Y_{i}=Y_{i}(a_{i}) . Similarly, their counterfactual potential outcome is missing, which we denote as Y_{i}^{M}=Y_{i}(1-a_{i}) . We let \bm{Y}^{M}=(Y_{1}^{M},Y_{2}^{M},\dots,Y_{n}^{M}) denote the full set of counterfactuals. The observed data is denoted by D^{O}=\{y_{i}(a_{i}),a_{i},l_{i}\}_{i=1}^{n} , which contains the factual potential outcome, y_{i}(a_{i}) , under observed treatment assignment a_{i} . Equivalently, D^{O}=\{y_{i}(1),a_{i}=1,l_{i}\}_{i\mid a_{i}=1}\cup\{y_{i}(0),a_{i}=0,l_{i}\}_{i\mid a_{i}=0} . 2 Sample vs. Population-level Causal Estimands In the potential outcomes framework, causal effects are defined as comparisons of potential outcomes within the same population (e.g. perhaps the same unit, some defined sub-population, or an overall target population). We will consider the following estimands throughout which are quite common in point-treatment settings in the literature: 1. ITE: The individual treatment effect (ITE) for subject i\in\{1,2\dots,n\} , \theta_{i}(Y_{i}^{M})=Y_{i}(1)-Y_{i}(0) we write it as a function of Y_{i}^{M} to emphasize that uncertainty about the ITE is due to uncertainty in subject i ’s (unobserved) counterfactual - if we only knew the counterfactual, then we could compute the difference using the factual we have to get \theta_{i}(Y_{i}^{M}) . That is, uncertainty is due to lack of knowledge about the counterfactual. We will suppress this notational dependence and write \theta_{i} when this emphasis is unwarranted. 2. SATE: The finite-sample average treatment effect (SATE) is simply the sample average of the ITEs: \theta(\bm{Y}^{M})=\frac{1}{n}\sum_{i=1}^{n}Y_{i}(1)-Y_{i}(0) again, we write \theta(\bm{Y}^{M}) to emphasize that uncertainty in the SATE comes from the full set of missing counterfactuals, but will generally suppress this notation. 3. CATE: The conditional average treatment effect (CATE) \psi(l,\phi_{Y1},\phi_{Y0})=E[Y(1)\mid L=l;\phi_{Y1}]-E[Y(0)\mid L=l;\phi_{Y0}] This is the difference in potential outcomes averaged in the subpopulation of the target with L=l . It contrasts the average effect had everyone in this subpopulation got treatment 1 versus 0. Here \phi_{Ya} is the parameter vector governing he conditional outcome distribution under treatment a . We write this as a function of (\phi_{Y1},\phi_{Y0}) to emphasize that all uncertainty about the CATE comes from lack of knowledge about these parameters, but will generally suppress this and simply write \psi(l) . 4. PATE: The related population-level average treatment effect (PATE) \Psi=E[Y(1)]-E[Y(0)] - the difference in average outcome across the entire target population had everyone taken treatment A=1 versus A=0 . This is obtained by averaging the CATEs over the population distribution of the confounder: \Psi(\phi_{Y1},\phi_{Y0},\phi_{L})=\int\Big(E[Y(1)\mid L=l;\phi_{Y1}]-E[Y(0)\mid L=l;\phi_{Y0}]\Big)f_{L}(l;\phi_{L})dl (1) Here, \phi_{L} is a parameter vector governing the marginal covariate distribution. Again, the notation \Psi(\phi_{Y1},\phi_{Y0},\phi_{L}) emphasizes that uncertainty in this quantity stems from uncertainty about the three parameter vectors, but we will generally suppress and simply write \Psi . Note that under the causal identification assumptions, each of the two expectations is E[Y(a)\mid L=l;\phi_{Ya}]=E[Y\mid A=a,L=l;\phi_{Ya}] for a\in\{0,1\} . Substituting this equivalence into \Psi is known as the standardization formula. It is a special case of the g-formula ( Robins 1986 ) . The first two estimands are not functions of unknown population-level parameters, but rather functions of unknown counterfactuals for specific units in our sample. Hence, we call these finite-sample estimands. The latter two, on the other hand, are purely functions of the unknown parameters governing the population distribution of the data, hence we call these population-level estimands. 3 The Joint Posterior Distribution Making Bayesian inferences for any of these estimands follows from the same procedure we use to make Bayesian inferences for all other problems: we condition on quantities we know, and find a distribution over the quantities we want to know but do not (i.e. the posterior). That is, a Bayesian statistician is one who “analyzes data by calculating, via Bayes’ theorem, the conditional distribution of unknowns given knowns” ( Rubin 1978 ) . What we know is the observed data, D^{O} . What do we not know? First, we do not know each subject’s counterfactual. To borrow terminology from the missing data literature, the “complete data” we ideally would observe is the union of the missing data and the observed data: D^{C}=\bm{Y}^{M}\cup D^{O}=\{y_{i}(1),y_{i}(0),a_{i},l_{i}\}_{i=1}^{n} . Together, the complete data for all n row-exchangeable subjects arises from the joint distribution f(D^{C};\omega)=\prod_{i=1}^{n}f(y_{i}(1),y_{i}(0),a_{i},l_{i};\omega) . This is governed by some parameter vector \omega , which is the second quantity unknown to us. Bayesian inference for all causal estimands is done using the joint posterior distribution which, up to a proportionality constant, is given by Bayes’ rule as f(\bm{y}^{M},\omega\mid D^{O})\propto f(D^{C};\omega)f(\omega) (2) The first factor is the complete-data distribution while the second is the joint prior distribution. As is commonly done in Bayesian analysis, we assume \omega=(\phi_{A},\phi_{Y},\phi_{L}) is comprised of distinct component parameter blocks and factorize the joint distribution of the complete data as follows using the chain rule, \begin{split}f(D^{C};\omega)&=\prod_{i=1}^{n}f(A=a_{i}\mid y_{i}(1),y_{i}(0),l_{i};\phi_{A})\cdot f(y_{i}(1),y_{i}(0)\mid l_{i};\phi_{Y})\cdot f(l_{i};\phi_{L})\end{split} (3) Note the presence of the bivariate distribution of a given subject’s potential outcomes, which is not identifiable even under the usual causal assumptions. Suppose, generally, it is parameterized in terms of \phi_{Y}=(\phi_{Y1},\phi_{Y0},\rho) , where the first two (identifiable) components govern the induced marginal distribution of each potential outcome whereas the (non-identifiable) third component, \rho , governs the dependence

Read the complete text