Conformal Prediction under Lévy–Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations
Liviu Aolaritei, Zheyu Oliver Wang, Julie Zhu, Michael I. Jordan, Youssef Marzouk · 2025
Read Conformal Prediction under Lévy–Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations by Liviu Aolaritei, Zheyu Oliver Wang, Julie Zhu, Michael I. Jordan, Youssef Marzouk (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-14105.txt.
Introduction
Liviu Aolaritei, Zheyu Oliver Wang, Julie Zhu, Michael Jordan, and Youssef Marzouk studied prediction intervals that survive data shifts in a 2025 paper. Conformal prediction gives intervals with firm coverage promises, but those promises assume new data match the old. Real deployments rarely cooperate.
The authors model shifts with Lévy-Prokhorov ambiguity sets, which allow both small local wiggles and larger global moves. They show these sets push through a model's scoring rule into simple one-dimensional shifts. That makes worst-case coverage computable exactly. Their intervals stay valid under the stated shifts, with width tied openly to how much shift is allowed. Tests on real data sets back the theory.
Catalog record
| CREATOR | Liviu Aolaritei, Zheyu Oliver Wang, Julie Zhu, Michael I. Jordan, Youssef Marzouk |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-14105
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Conformal Prediction under Lévy–Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations Liviu Aolaritei, Zheyu Oliver Wang, Julie Zhu, Michael I. Jordan, Youssef Marzouk Abstract Conformal prediction provides a powerful framework for constructing prediction intervals with finite-sample guarantees, yet its robustness under distribution shifts remains a significant challenge. This paper addresses this limitation by modeling distribution shifts using Lévy–Prokhorov (LP) ambiguity sets, which capture both local and global perturbations. We provide a self-contained overview of LP ambiguity sets and their connections to popular metrics such as Wasserstein and Total Variation. We show that the link between conformal prediction and LP ambiguity sets is a natural one: by propagating the LP ambiguity set through the scoring function, we reduce complex high-dimensional distribution shifts to manageable one-dimensional distribution shifts, enabling exact quantification of worst-case quantiles and coverage. Building on this analysis, we construct robust conformal prediction intervals that remain valid under distribution shifts, explicitly linking LP parameters to interval width and confidence levels. Experimental results on real-world datasets demonstrate the effectiveness of the proposed approach. 1 Introduction 1 1 footnotetext: The first three authors contributed equally. Their order is alphabetical. Conformal prediction has emerged as a versatile framework for constructing prediction intervals with finite-sample coverage guarantees [ 32 , 40 , 2 ] . By leveraging the concept of nonconformity, it provides valid confidence sets for predictions, regardless of the underlying data distribution. This framework has gained significant traction in fields such as medicine [ 29 , 38 ] , bioinformatics [ 14 ] , finance [ 41 ] , and autonomous systems [ 27 , 28 ] , where decision-making under uncertainty is critical. However, the standard conformal prediction framework relies on the assumption of exchangeability between training and test data [ 4 ] . When this assumption is violated due to distribution shifts, the coverage guarantees of conformal prediction may break down, limiting its applicability in real-world scenarios [ 37 ] . Distribution shifts—systematic changes between the training and test distributions—are ubiquitous in practice. Examples include covariate shift in medical diagnostics, where the population characteristics evolve over time [ 36 ] , or adversarial perturbations in image classification, where small, targeted changes to inputs can drastically alter predictions [ 31 ] . Addressing such shifts is essential for ensuring the reliability of predictive models, particularly in high-stakes applications. Many extensions of conformal prediction to distribution shift rely on restrictive assumptions—such as covariate or label shift [ 37 , 34 ] , local \ell_{2} perturbations or global contamination [ 16 , 11 ] , or f -divergence-based models [ 9 ] —or require access to likelihood ratios between training and test distributions [ 37 , 34 ] . While effective in certain settings, these approaches can struggle with more complex shifts that involve both local perturbations (e.g., small, pixel-level changes in images) and global perturbations (e.g., population-wide shifts in feature distributions) [ 6 ] . To bridge this gap, we propose a novel framework based on Lévy–Prokhorov (LP) ambiguity sets, a class of optimal transport-based discrepancy measures that simultaneously capture local and global perturbations. LP ambiguity sets offer a flexible and interpretable way to model distributional uncertainty. Unlike f -divergences, which are limited to absolutely continuous shifts, LP metrics naturally handle broader scenarios, including discrete and transport-based perturbations [ 7 ] . For example, LP metrics can capture local shifts such as minor variations in image textures or sensor readings, as well as global shifts like changes in population demographics. This dual capability makes LP metrics particularly suited for robust prediction in dynamic and heterogeneous environments. In this paper, we leverage the LP ambiguity set to develop a distributionally robust extension of conformal prediction. By propagating LP ambiguity sets through the scoring function, we simplify high-dimensional shifts into one-dimensional shifts in the score space, enabling exact quantification of worst-case quantiles and coverage. This approach leads to interpretable and robust prediction intervals, with explicit control over how the local and global LP parameters influence interval width and confidence levels. Specifically, the main contributions of this paper can be summarized as follows. 1. Distribution shifts as LP ambiguity sets. We provide a brief, yet self-contained, overview of Lévy-Prokhorov (LP) ambiguity sets, highlighting their connection to well-known ambiguity sets derived from Wasserstein and Total Variation (TV) distances. 2. Propagation of LP ambiguity sets. We show that the propagation through the scoring function s of the LP ambiguity set around the training distribution \mathbb{P} is captured by another LP ambiguity set around the scoring distribution s_{\#}\mathbb{P} (the pushforward of \mathbb{P} via the map s ). This naturally translates the LP distribution shift in the input-label pair (X,Y) into a much simpler one-dimensional LP distribution shift in s(X,Y) . 3. Distributionally robust conformal prediction. The propagated LP ambiguity set around s_{\#}\mathbb{P} enables precise quantification of both the worst-case quantile and worst-case coverage . These quantities are essential for constructing valid conformal prediction intervals under LP distribution shifts. Our approach is interpretable, elucidating how the local and global parameters of the LP ambiguity set influence the width of the interval and the confidence level. Finally, we validate the proposed approach on three benchmark datasets: MNIST [ 25 ] , ImageNet [ 13 ] , and iWildCam [ 6 ] , the latter of which captures real-world distribution shifts, demonstrating its empirical coverage guarantees and efficiency in terms of prediction set size. 1.1 Related Work Under train-test distribution shifts that violate exchangeability, conformal prediction often fails to maintain valid coverage guarantees [ 37 ] . Extensions to conformal prediction under such shifts can be summarized into three main categories: sample reweighting, ambiguity sets, and sequential learning. Sample Reweighting. This approach assigns weights to calibration samples based on their relevance to the test data. For instance, [ 37 ] proposed weighted conformal prediction for covariate shift, where the marginal distribution \mathbb{P}_{X} changes while the conditional distribution \mathbb{P}_{Y|X} remains fixed. Likelihood ratios are used to adjust for compositional differences, enabling valid predictions. Subsequent extensions address label shift [ 34 ] , causal inference [ 26 ] , and survival analysis [ 8 , 20 ] . However, these methods rely on the accurate estimation of likelihood ratios, which may be challenging in practice. For spatial data, [ 30 ] proposed weighting samples based on proximity to test points. Still within the covariate shift setting, [ 35 ] and [ 46 ] leverage semiparametric theory to design more efficient conformal methods with asymptotic conditional coverage, bypassing the need for explicit sample reweighting. Compared to these approaches, our method handles distribution shifts in the joint distribution \mathbb{P} of (X,Y) , without requiring likelihood ratios, and remains effective under more complex local and global perturbations. Ambiguity Sets. Ambiguity sets provide a flexible framework for modeling uncertainty in the data distribution. For instance, [ 9 ] used an f -divergence ambiguity set around the training distribution to derive worst-case coverage guarantees and adjusted prediction sets. This work is most closely related to ours, and while their analysis inspired our approach, we rely on fundamentally different tools, particularly drawing on optimal transport techniques. A key limitation of f -divergences is that they are restricted to distribution shifts that are absolutely continuous with respect to the