Causal Effect Estimation with TMLE: Handling Missing Data and Near-Violations of Positivity

Christoph Wiederkehr, Christian Heumann, Michael Schomaker · 2025

Read Causal Effect Estimation with TMLE: Handling Missing Data and Near-Violations of Positivity by Christoph Wiederkehr, Christian Heumann, Michael Schomaker (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22202.txt.

Introduction

Christoph Wiederkehr, Christian Heumann, and Michael Schomaker stress-test a respected causal method against two real-world messes. The first mess is missing data. The second is near-failure of the positivity assumption. That assumption demands that every kind of patient could receive either treatment. The method under test is targeted maximum likelihood estimation. It is prized for estimating treatment effects with machine learning inside.

Simulations walk through five missing-data diagrams common in epidemiology. Some cases lose values for reasons tied to the unobserved values themselves. A design-based simulation uses a real water-and-sanitation trial dataset from Bangladesh. Eight missing-data strategies are compared. Plain complete-case analysis paired with a model for outcome missingness showed lower bias. Fancier imputation options did worse. The authors report the result against the usual instinct to impute.

Catalog record

CREATORChristoph Wiederkehr, Christian Heumann, Michael Schomaker
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE92 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-22202

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Causal Effect Estimation with TMLE: Handling Missing Data and Near-Violations of Positivity Christoph Wiederkehr, Christian Heumann, Michael Schomaker Abstract We evaluate the performance of targeted maximum likelihood estimation (TMLE) for estimating the average treatment effect in missing data scenarios under varying levels of positivity violations. We employ model- and design-based simulations, with the latter using undersmoothed highly adaptive lasso on the ’WASH Benefits Bangladesh’ dataset to mimic real-world complexities. Five missingness-directed acyclic graphs are considered, capturing common missing data mechanisms in epidemiological research, particularly in one-point exposure studies. These mechanisms include also not-at-random missingness in the exposure, outcome, and confounders. We compare eight missing data methods in conjunction with TMLE as the analysis method, distinguishing between non-multiple imputation (non-MI) and multiple imputation (MI) approaches. The MI approaches use both parametric and machine-learning models. Results show that non-MI methods, particularly complete cases with TMLE incorporating an outcome-missingness model, exhibit lower bias compared to all other evaluated missing data methods and greater robustness against positivity violations across. In Comparison MI with classification and regression trees (CART) achieve lower root mean squared error, while often maintaining nominal coverage rates. Our findings highlight the trade-offs between bias and coverage, and we recommend using complete cases with TMLE incorporating an outcome-missingness model for bias reduction and MI CART when accurate confidence intervals are the priority. 1 Introduction Missing data on multiple variables pose a common challenge in observational studies investigating the effects of treatments on health outcomes. Traditional approaches often rely on the classifications of being missing completely at random (MCAR), at random (MAR) and not at random (MNAR) [ 1 , 2 ] . However, Mohan and Pearl [ 3 ] and Mohan et al. [ 4 ] introduced missingness-directed acyclic graphs (m-DAGs) as an alternative framework to describe missingness mechanisms. This framework enables a more nuanced assessment of whether unbiased and consistent causal effect estimation is possible [ 3 ] , depending on the conditions of recoverability with respect to the chosen missing data method [ 4 ] . Recoverability, in this context, refers to whether a target parameter, such as a causal effect, can be expressed solely in terms of the distribution of the observed, incomplete data—without requiring knowledge of the unobserved data distribution [ 5 ] . Since no general algorithm exists to determine recoverability in arbitrary settings, identification and estimation strategies must be developed on a case-by-case basis [ 6 ] . The procedure involves defining the target estimand without missing data, constructing an m-DAG to represent relationships between variables and missingness indicators, assessing recoverability under m-DAG assumptions, and selecting an appropriate missing data method [ 7 ] . Thus, m-DAGs provide a valuable tool for assessing missing data mechanisms in epidemiological research, particularly in point-exposure studies where missing data are common. [ 5 ] . In causal inference, the average treatment effect (ATE) is a key target parameter, representing the expected difference in outcomes under treatment versus no treatment [ 8 ] . Estimation of the ATE from observational data generally relies on the assumptions of consistency, conditional exchangeability, and positivity [ 8 , 9 ] . Targeted Maximum Likelihood Estimation (TMLE) is commonly used for estimating the ATE due to its double robustness properties [ 10 , 11 , 12 ] . Specifically, TMLE combines both outcome and treatment models, ensuring consistent estimates even if one of these models is misspecified [ 11 ] . Additionally, TMLE is a plug-in estimator, meaning it directly targets the parameter of interest, while allowing for the incorporation of data-adaptive methods, such as machine learning methods via super learning [ 13 ] . This flexibility makes TMLE suitable for a wide range of data-generating processes and less reliant on strict parametric assumptions, thus enhancing its applicability [ 12 ] . Previous research within the m-DAG framework has explored various methods for handling missing data in causal effect estimation, including TMLE with data-adaptive approaches [ 14 ] . However, the comparative performance of general missing data methods, such as complete case (CC) analysis and multiple imputation (MI), when used in combination with TMLE, remains uncertain across diverse data settings, particularly in the presence of positivity violations. Positivity, a crucial assumption in causal inference, requires that within each stratum of the confounders, every individual has a non-zero probability of receiving either exposure condition [ 8 ] . In observational studies, this assumption is often violated when certain subgroups almost never receive a particular treatment due to socioeconomic, demographic, or clinical factors [ 15 ] . Such violations can severely impact the identifiability of causal effects, leading to high variability in estimates [ 16 ] . Although TMLE can partially mitigate these issues by bounding propensity scores away from extremes, it may introduce bias when positivity violations are severe [ 17 , 16 ] . Moreover, MI methods, which lack such a protective procedure, may be inconsistent under severe positivity violations, raising concerns about their applicability even when the MAR assumption holds. The performance of MI depends on the assumptions made about the data-generating process (DGP) and the missingness mechanism (typically MAR). To ensure valid inference, the imputation model must be ’congenial’—that is, correctly specified and properly aligned with the analysis model [ 18 , 19 ] . It is essential to evaluate how MI, in conjunction with TMLE, performs in scenarios where data are MNAR or where positivity violations occur. In particular, comparing MI-based approaches to complete case (CC) analysis and the missing covariate missing indicator (MCMI) approach [ 20 ] is crucial for understanding their relative strengths and limitations in handling missing data. These methods represent a spectrum of strategies commonly used in applied research [ 14 , 21 ] . Building upon previous studies [ 5 , 14 , 22 ] , this paper provides recommendations on the suitability of different missing data methods in conjunction with TMLE in settings both with and without positivity violations. First, we extend existing simulation frameworks by incorporating sophisticated model-based DGPs that capture a broad range of variable types and dependencies, including structured near violations of positivity. To systematically evaluate their impact, we vary the degree of near positivity violations and assess how different levels influence estimation. Second, we bridge the gap between theoretical simulations and practical applications by incorporating a design-based DGP approach. While model-based simulations are widely used, their findings may not always generalize to empirical data, which often deviate from idealized distributions [ 23 ] . To address this limitation, we complement model-based simulations with a data-driven approach that employs undersmoothed highly adaptive lasso [ 24 ] on empirical data from the ’WASH Benefits Bangladesh’ study [ 25 ] . Lastly, we expand the comparative analysis of missing data methods by systematically evaluating multiple approaches, both MI and non-MI, when used in conjunction with TMLE. This comprehensive comparison offers new insights into the robustness and limitations of these methods in the presence of positivity violations and complex missing data mechanisms. 2 Framework In this work, we follow the approach proposed by Lee et al. [ 7 ] , using m-DAGs to depict missingness assumptions and assess their implications for the recoverability of the causal estimand. 2.1 Causal Estimand We are interested in estimating

Read the complete text