A Systematic Review of Conformal Inference Procedures for Treatment Effect Estimation: Methods and Challenges
Pascal Memmesheimer, Vincent Heuveline, Jürgen Hesser · 2025
Read A Systematic Review of Conformal Inference Procedures for Treatment Effect Estimation: Methods and Challenges by Pascal Memmesheimer, Vincent Heuveline, Jürgen Hesser (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21660.txt.
Introduction
Estimating treatment effects is not enough for high stakes decisions; doubt around the estimate matters too. Pascal Memmesheimer and 2 co-authors step by step review conformal methods for that task in a 2025 paper. Conformal output gives finite sample coverage guarantees under minimal givens. It stays cheap to compute and tolerates spread shifts. Any point predictor model can be wrapped by these procedures.
The review supplies the theoretical background for conformal treatment effect estimation. A systematic filtering process selects and analyses the eligible studies. Methods and challenges are organized for healthcare, economics, and public policy readers. Heterogeneous treatment effects from flexible machine learning models are a key use case. The authors assess where conformal inference stands for decision grade doubt. Good calls in care, trade, and law need not just a best guess but a range that can be trusted.
Catalog record
| CREATOR | Pascal Memmesheimer, Vincent Heuveline, Jürgen Hesser |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-21660
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Systematic Review of Conformal Inference Procedures for Treatment Effect Estimation: Methods and Challenges Pascal Memmesheimer, Vincent Heuveline, Jürgen Hesser Abstract . Treatment effect estimation is essential for informed decision-making in many fields such as healthcare, economics, and public policy. While flexible machine learning models have been widely applied for estimating heterogeneous treatment effects, quantifying the inherent uncertainty of their point predictions remains an issue. Recent advancements in conformal prediction address this limitation by allowing for inexpensive computation, as well as distribution shifts, while still providing frequentist, finite-sample coverage guarantees under minimal assumptions for any point-predictor model. This advancement holds significant potential for improving decision-making in especially high-stakes environments. In this work, we perform a systematic review regarding conformal prediction methods for treatment effect estimation and provide for both the necessary theoretical background. Through a systematic filtering process, we select and analyze eleven key papers, identifying and describing current state-of-the-art methods in this area. Based on our findings, we propose directions for future research. nameAlgorithm 1. Introduction Uncertainty quantification is crucial for high-stakes decision-making, especially in the medical domain. Here, it is essential to establish robustness and trustworthiness of models. In the past, there has been widespread adoption of black-box models like neural networks or other difficult-to-interpret models like gradient-boosted trees that often only provide point predictions. The reliance on such point predictions is insufficient for assessing risks for reliable, robust, and trustworthy decision-making. Conformal prediction promises remedy. This framework of methods conformalizes the outputs of any point predictor, s.t. marginally valid prediction regions can be computed. Recently, these methods have become popular which can be seen by the sheer number of papers published on this topic ( Wohlin et al. 2012 ) . The application of conformal prediction for causal inference is not trivial, however, since conformal prediction relies on the notion of exchangeability which is inherently violated in causal inference since the counterfactual is never observed, thus, the inference problem involves a potential covariate shift between the target distribution and the sampling distribution ( Lei and Candès 2021a ) . The recent introduction of weighted exchangeability has intensified interest in conformal prediction for treatment effect estimation, as it allows for handling distributional shifts ( Tibshirani et al. 2020 ) . 1.1. Our Contribution In this systematic review, we provide a comprehensive overview of conformal prediction techniques for treatment effect estimation. By organizing, filtering, and summarizing various approaches, we highlight their suitability for different types of problems. To the best of our knowledge, no other review paper exists for applying conformal prediction for treatment effect estimation. We perform a comprehensive, structured search on all research papers from 2005 until 2025 using multiple electronic databases. For our filtration process, we establish both inclusion criteria and quality standards to ensure a transparent and well-structured approach. Through this process, we identify eleven papers, which we discuss in detail and use to answer our research questions. This allows interested researchers to join this important research area. This paper is organized as follows. In Section 2 , we review the foundations of conformal prediction and discuss key concepts and definitions in causal inference, including popular methods for estimating treatment effects. Thereafter, we discuss our methodology for the search and filtration process in Section 3 . In Section 4 we summarize various methods of conformal inference for estimating treatment effects and analyze the meta-data of the papers published. We discuss our findings and possible future research directions in Section 5 . Conclusions are given in Section 6 . 2. Preliminaries In the following two sections, we introduce the theoretical foundations of the relevant conformal prediction methods for this work. Thereafter, we introduce important causal inference terms as well as popular methods for estimating treatment effects. 2.1. Conformal Prediction Conformal prediction, also known as conformal inference, is a method originally introduced by Vladimir Vovk et al. ( Vladimir Vovk 2005 ; Papadopoulos et al. 2002a ; Lei and Wasserman 2013 ) , and has since been refined over many years. It has garnered significant attention in both the statistics and machine learning communities for its ability to provide distribution- and model-free finite-sample uncertainty quantification in the form of statistically valid prediction regions . The method can be used for classification tasks for which it generates prediction sets (e.g., \{\text{cat},\text{dog},\text{crocodile}\} ). In regression tasks, prediction intervals are generated (e.g., [1000,5000] ). We will refer in both cases to the more general nomenclature prediction region and specify only in more detail when needed. There are many variants of conformal prediction that have been introduced and refined in the past two decades. For a more rigorous definition and introduction, we refer to Angelopoulos et al. 2025 , Fontana et al. 2023 ; Angelopoulos and Bates 2022 and especially the comprehensive book by Vladimir Vovk 2005 . We will focus on the predominant method of split conformal prediction since it decreases the heavy computational burden of the original method, full conformal prediction . Other variants have been introduced, notably variants that make use of cross-validation. For this, we refer to Barber et al. 2020a ; Vovk 2015 , as well as to Vovk et al. 2017a who also describe a procedure for calculating conformal predictive distributions. In split conformal prediction , the data is divided into three subsets: training, calibration, and test, as opposed to the usual train/test division. It then uses the aforementioned training data set to fit an arbitrary learner \hat{f}:\mathbb{R}^{d}\rightarrow\mathbb{R} , i.e., producing one-dimensional point estimates from d -dimensional inputs, in a first step. In a second step, the learners predictions are being conformalized (or calibrated), s.t. we can make statistically valid statements of our models predictions with only minimal guarantees which we will elaborate later on. The procedure guarantees marginal coverage regardless of the learner’s predictive accuracy, though the resulting regions may be wide. The goal is therefore to construct non-trivial prediction regions that both attain the desired marginal coverage and remain as tight as possible. Let (X_{1},Y_{1}),\dots,(X_{m},Y_{m}) be a sequence of random variables sampled i.i.d. from some distribution \mathscr{P}_{XY} on \mathbf{X}\times\mathbf{Y} , with marginals \mathscr{P}_{X} and \mathscr{P}_{Y} . Each pair is called an example , Z_{i}:=(X_{i},Y_{i})\in\mathbf{Z}:=\mathbf{X}\times\mathbf{Y} for i=1,\dots,m . Here, X_{i}\in\mathbf{X} represents the features and Y_{i}\in\mathbf{Y} the response. Throughout this work, we focus on the regression setting and assume without loss of generality that \mathbf{Y}=\mathbb{R} and \mathbf{X}=\mathbb{R}^{d} , where d denotes the dimensionality of the feature space. We denote by lowercase letters realizations of random variables. For both classification and regression, the data is split (see Figure 1 ) into three distinct multisets: (1) A training set Z_{\text{train}}:=Z^{0}_{1:n_{0}}=\{Z^{0}_{1},\dots,Z^{0}_{n_{0}}\} (2) A calibration set Z_{\text{calib}}:=Z_{1:n}=\{Z_{1},\dots,Z_{n}\} (3) A test set Z_{\text{test}}:=Z_{n+1:m}=\{Z_{n+1},\dots,Z_{m}\} A central notion of conformal prediction is the nonconformity measure . Informally, a nonconformity measure estimates how "unusual" a sample looks with respect to the population of data points. A nonconformity measure is a function A:\mathbf{Z^{(*)}}\times\mathbf{Z}\rightarrow\overline{\mathbb{R}} , where \mathbf{Z^{(*)}} is a finite multiset of elements in \mathbf{Z} , and \overline{\mathbb{R}} is the extended real numbers, i.e., \overline{\mathbb{R}}=\mathbb{R}\cup\{-\infty,\infty\} . A nonconformity measure induces a nonconformity score at each data point: R_{i}:=A((X_{i},Y_{i});Z_{\text{calib}}),\quad i=1,\dots,n. In regression problems, a common choice for A is the absolute residual defined as A((x,y);Z_{\text{calib}}):=|\hat{f}(x;Z_{\text{train}})-y| , where \hat{f}(x) is a point prediction made by a learner