Moving toward best practice when using propensity score weighting in survey observational studies

Yukang Zeng, Fan Li, Guangyu Tong · 2025

Read Moving toward best practice when using propensity score weighting in survey observational studies by Yukang Zeng, Fan Li, Guangyu Tong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16156.txt.

Introduction

Yukang Zeng, Fan Li, and Guangyu Tong sort out a statistics dispute in this 2025 paper. Researchers often blend survey data with propensity weighting to estimate treatment effects. Survey weights and propensity weights interact in tricky ways. Past papers disagreed on how to combine them, leaving users without clear rules.

The team gives one framework that handles both weights together. They derive formulas for several target populations, including treated and overlap groups. A matching variance formula comes with a proof that estimates settle into a normal shape. Long simulation runs show where their method beats rivals. Two case studies show the approach on real survey data. Practical guidelines close the paper.

Catalog record

CREATORYukang Zeng, Fan Li, Guangyu Tong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE113 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-16156

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Moving toward best practice when using propensity score weighting in survey observational studies Yukang Zeng, Fan Li, Guangyu Tong Abstract Propensity score weighting is a common method for estimating treatment effects with observational data, by addressing confounding due to measured baseline covariates. However, when the observational data sample is drawn based on a survey, the existing literature does not reach a consensus on the optimal use of survey weights for population-level causal inference. Under the balancing weights framework, we provide a unified solution for incorporating survey weights and derive a set of weighting and augmented weighting estimators for different target populations, including the combined, treated, controlled, and overlap populations. We also develop closed-form sandwich variance estimators for each estimator via the theory of M-estimators. Through an extensive series of simulation studies, we examined the performance of our estimators and compared the results to those of alternative methods. We carried out two case studies to illustrate the application of the different propensity score methods with complex survey data. We concluded with a discussion of our findings and provided some practical recommendations for propensity score weighting analysis of survey observational data. Keywords: causal inference, propensity score weighting, survey weights, overlap weights, covariate balance, augmented weighting estimators 1 Introduction The use of observational data to estimate the causal effects is prevalent in medical research, particularly when randomized trials are unavailable. In many cases, researchers resort to observational data garnered through surveys with complex sampling techniques. These techniques are designed to ensure data from a sample can accurately represent the external target population, and include features such as multistage sampling and deliberate oversampling of less representative subgroups. [ 1 ] The survey weights computed based on the design are integral to recovering the population features so that the differentiated selection probabilities and non-responses are appropriately accounted for. There is consensus on the importance of incorporating the survey design into the analytical process to derive causal effect estimates that are applicable to the target population. [ 2 , 3 , 4 ] Survey weights also play a crucial role in ensuring the accuracy and representativeness of the propensity score method, which is among the most commonly used tool for causal inference using observational data. [ 5 ] In the absence of randomization, systematic differences in baseline characteristics usually exist between exposed and unexposed individuals, leading to the issue of confounding. The propensity score methods act on the treatment assignment mechanism with measured baseline covariates, [ 6 ] and with estimated propensity scores, differences in the observed confounders across groups can be reduced so that the estimated treatment effect will no longer be attributable to pre-existing group differences. In practice, implementing propensity score methods involve a two-step procedure, including propensity score estimation and the subsequent treatment effect estimation. The latter step leverages the estimated propensity score through stratification, [ 7 ] matching, [ 8 ] or weighting. [ 9 ] The appropriate use of survey weights in stratification to obtain population causal effect estimates has been previously addressed, with a recommendation that survey weights should be used when estimating treatment effects within strata but not in the estimation of the propensity scores. [ 10 , 2 ] DuGoff et al. [ 3 ] later suggested that survey weights can carry unique design features that are not captured by observed covariates, and therefore should be included as an additional covariate in the propensity score model to enhance covariate balance and address potential bias due to unmeasured covariates. Similar recommendations have also been extended to propensity score matching and weighting, where the estimation of propensity score can incorporate survey weights as an additional covariate; but in those contexts, whether to perform survey-weighted analysis in the outcome model depends on the goal of the analysis (i.e., whether a sample or the underlying population defines the target of inference). [ 3 ] Austin et al. [ 11 ] illustrated through simulations that including survey weight as a covariate in the propensity score model showed no clear advantage over a survey-weighted propensity score model. Neither approach can consistently outperform the other regarding covariate balance and bias in the treatment effect estimation across various scenarios. Dong et al. [ 12 ] evaluated multiple strategies for incorporating survey weights into both propensity score and outcome models, recommending their use at both stages, although they observed that estimates from different strategies did not yield apparent differences in their data application. Lenis et al. [ 13 ] found through simulations that incorporating survey weights in the propensity score model does not necessarily improve the estimation of population-level treatment effect using matching, especially when sufficient population-level balance in confounders can already be achieved. They also showed through simulations that incorporating survey weights in propensity score estimation can reduce bias when the missingness in outcome is correlated with survey weights. [ 13 ] Regarding propensity score weighting specifically, Ridgeway et al. [ 4 ] provided a theoretical justification for the use of a survey-weighted propensity score model in obtaining a consistent estimator of the population-level average treatment effect. They proved that using survey weights in both stages leads to consistent and robust population average treatment effect estimators under various generating mechanisms. Salerno et al. [ 14 ] recently explored a complex survey mechanism in health disparities research, where survey weights depend on the group variable of interest under comparison, such as race. They further revealed that whether the propensity score should be survey-weighted depends on whether the available survey weights are tied to comparison variables such as treatment or exposure. Whereas Ridgeway et al. [ 4 ] derived the results and performed simulations for the continuous outcome scenario, Yang et al. [ 15 ] extended the results to a binary outcome and evaluated the performance of different ways to incorporate survey weights in the propensity score model and the outcome model. Both Ridgeway et al. [ 4 ] and Yang et al. [ 15 ] considered the potentially differentiated practice of incorporating survey weights when estimating the average treatment effect (ATE) and the average treatment effect of the treated (ATT). In general, the emerging message from the existing literature is that the use of survey weights at the propensity score model stage and the outcome model stage can potentially be beneficial in reducing the bias of population-level inference. However, a consensus on the best practice and the underlying justifications for using survey weights to target different causal estimands of interest have not been fully elucidated. This paper aims to contribute to the literature on propensity score weighting in several ways. First, the target estimands in the prior development on survey observational data methods were limited to ATE and ATT, and have not covered other weighted average treatment effect estimands. [ 9 ] We address this by supplying a unified framework that incorporates survey weights into the general framework of balancing weights. [ 9 ] Second, as a further extension, we detail the use of survey weights for augmented weighting estimators under the balancing weights framework, which can potentially improve efficiency over weighting alone by leveraging information from a posited outcome regression function. Third, we developed closed-form variance estimators that accommodate survey weights in each estimator we considered

Read the complete text