Conformal Prediction for Generative Models via Adaptive Cluster-Based Density Estimation

Qidong Yang, Julie Zhu, Jonathan Giezendanner, Youssef Marzouk, Stephen Bates, Sherrie Wang · 2026

Read Conformal Prediction for Generative Models via Adaptive Cluster-Based Density Estimation by Qidong Yang, Julie Zhu, Jonathan Giezendanner, Youssef Marzouk, Stephen Bates, Sherrie Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22298.txt.

Introduction

Qidong Yang and five co-authors bring honest error bars to models that dream up new samples. Such models map inputs to rich, high-end outputs, from images to forecasts. Their 2026 preprint notes a gap: users get no firm sense of how much to trust one output.

The team adapt a method that groups model outputs into clusters. Density reads on those clusters then set the trust bounds. The bounds aim to hold at the stated rate, case by case. Trust marks like these matter most where stakes are high. They help users know when to check a dream twice.

Catalog record

CREATORQidong Yang, Julie Zhu, Jonathan Giezendanner, Youssef Marzouk, Stephen Bates, Sherrie Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22298

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Conformal Prediction for Generative Models via Adaptive Cluster-Based Density Estimation Qidong Yang, Julie Zhu, Jonathan Giezendanner, Youssef Marzouk, Stephen Bates, Sherrie Wang Abstract Conditional generative models map input variables to complex, high-dimensional distributions, enabling realistic sample generation in a diverse set of domains. A critical challenge with these models is the absence of calibrated uncertainty, which undermines trust in individual outputs for high-stakes applications. To address this issue, we propose a systematic conformal prediction approach tailored to conditional generative models, leveraging density estimation on model-generated samples. We introduce a novel method called CP4Gen, which utilizes cluster-based density estimation to construct prediction sets that are less sensitive to outliers, more interpretable, and of lower structural complexity than existing methods. Extensive experiments on synthetic datasets and real-world applications, including climate emulation tasks, demonstrate that CP4Gen consistently achieves superior performance in terms of prediction set volume and structural simplicity. Our approach offers practitioners a powerful tool for uncertainty estimation associated with conditional generative models, particularly in scenarios demanding rigorous and interpretable prediction sets. 1 Introduction Modern machine learning applications increasingly rely on generative models to produce complex, high-dimensional outputs such as images, text, and molecular structures ( Karras et al. 2019 ; Brown et al. 2020 ; Gómez-Bombarelli et al. 2018 ) . (a) Calibration of CP4Gen (b) Inference of CP4Gen Figure 1 : The two phases of CP4Gen: calibration and inference. (a) K -means is applied to prediction ensemble members. Each identified cluster is treated as one mode of a Gaussian Mixture Model, whose sample mean \boldsymbol{\mu}_{k} and covariance matrix \boldsymbol{\Sigma}_{k} are calculated. Nonconformity score s_{i} is defined as the minimal s_{i}^{k} between observation Y_{i} and each cluster. Nonconformity score quantile Q_{1-\alpha} is then computed on the calibration set. (b) Given a new set of covariates, ensemble members are sampled and K -means is applied on top. Prediction set is obtained by inverting nonconformity score function with Q_{1-\alpha} resulting as a union of K ellipsoids. These models, such as variational autoencoders ( Kingma & Welling 2022 ) , generative adversarial networks ( Goodfellow et al. 2014 ) , and diffusion models ( Song et al. 2021 ) , have shown impressive performance across domains ranging from natural language processing to scientific discovery. However, a critical limitation of these generative models is their lack of calibrated uncertainty estimates ( Gal & Ghahramani 2016 ) . Although they can generate realistic samples, it remains unclear how much we should trust a single output, particularly in high-stakes domains like medical diagnosis or autonomous systems. Conformal prediction (CP) provides a model-agnostic, post-hoc framework that yields prediction sets with finite-sample coverage guarantees while maintaining sharpness ( Vovk et al. 2005 ; Angelopoulos et al. 2025 ) . However, classical conformal prediction methods are primarily designed for regression and classification tasks. Given the need of uncertainty quantification for generative models, a conformal prediction extension is desired. There have recently been some initial attempts to extend conformal prediction to the generative setting. For example, Wang et al. 2022 proposed probabilistic conformal prediction (PCP), which constructs prediction sets as unions of balls centered at ensemble members from conditional generative models. While PCP is broadly applicable in the sample-only setting, its isotropic, structure-free construction tends to over-cover and produce disconnected components sensitive to outliers. This work proposes a new CP method, called CP4Gen, which successfully addresses these limitations. It introduces a lightweight structural assumption on ensemble members (i.e., Gaussian mixture) through clustering and local covariance estimation, enabling prediction sets that are sharper, more robust, and more interpretable. This approach adopts the split conformal prediction framework ( Lei et al. 2017 ) , which avoids the high computational cost of leave-one-out CP by splitting the dataset into training and calibration sets. In this paper, we also provide a density estimation view of CP for generative models. It unifies CP4Gen and PCP under the same framework and outlines design space for more generative CP methods. Our main contributions are summarized as follows: • We formulate an extension of the conformal prediction framework to the generative setting. This new approach works with any existing conditional generative models and only requires the capability to draw samples from the models. • We define a new evaluation metric for generative model prediction sets, called structural complexity. Structural complexity measures the prediction set’s geometric intricacy and reflects how easily the prediction set can be interpreted and used for decision-making. • We propose a new CP method, CP4Gen, following our approach. CP4Gen introduces a tunable hyper-parameter which controls resulting set’s structural complexity, rendering PCP as one special case. • Evaluated on a variety of datasets, we show that CP4Gen outperforms state-of-the-art PCP in terms of prediction set volume and structural complexity, and consistently demonstrates superior performance on responses of various dimensionality. 2 Related Works Conformal Prediction with Deterministic Predictions Conformal prediction was conventionally designed for deterministic models, which produce a single regression prediction for each input. Literature explicitly leveraging output geometry structure to construct prediction sets is extensive. For instance, Johnstone & Cox 2021 ; Messoudi et al. 2022 approximate output residual distribution with a Gaussian model, resulting in ellipsoidal prediction sets. Messoudi et al. 2021 uses copulas to capture non-linear dependencies across output dimensions, and Tumu et al. 2024 constructs prediction regions using a union of convex templates for multi-modal coverage. However, these methods produce global prediction sets that do not adapt to input variations. Another line of approaches leverages transport maps to construct prediction sets. Essentially, these methods transform a complex residual distribution into a simple reference distribution, so that well-defined quantile sets from the reference distribution can be pulled back to form conformal prediction sets in the original output space. Klein et al. 2025 ; Thurin et al. 2025 build such transport maps using optimal transport, and Feldman et al. 2023 ; Fang et al. 2024 learn them using generative models, such as variational autoencoders ( Kingma & Welling 2022 ) and normalizing flows ( Rezende & Mohamed 2015 ) . Conformal Prediction with Conditional Distribution Predictions In many settings, predictive models output a full conditional distribution given the input. A common approach is to design a family of nested sets on the output distributions and then select the smallest sets within the family that achieves the desired marginal coverage. In multiclass classification, where models output a probability mass function over a finite label set, CP methods return a prediction set consisting of all labels whose total probability exceeds a calibrated threshold ( Sadinle et al. 2019 ) . In the case of univariate continuous response, Chernozhukov et al. 2021 and Sesia & Romano 2021 use estimated conditional cumulative distribution functions to produce valid prediction intervals. In multivariate continuous settings, CD-split ( Izbicki et al. 2019 ) constructs prediction sets by thresholding an estimated conditional density, and Izbicki et al. 2021 refine CD-split to improve conditional coverage. Conformal Prediction with Generative Models Modern generative models fall into two classes: those with tractable conditional densities and those with intractable conditional densities. The tractable class, including normalizing flows ( Rezende & Mohamed 2015 ) and flow matching models ( Lipman et al. 2024 ) , permits point-wise evaluation of conditional density functions, and the conformal methods with distribution

Read the complete text