Conformalised imprecise inference for robust extrapolation under limited data
Yu Chen, Scott Ferson · 2026
Read Conformalised imprecise inference for robust extrapolation under limited data by Yu Chen, Scott Ferson (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25882.txt.
Introduction
Chen and Ferson offer a conformalised loose inference approach for robust extrapolation under limited data. Work stresses chance-based and knowledge doubt, yet methods lack promises beyond the domain. Their approach is model-neutral, adding imprecision and distance awareness. It yields probability boxes that stay valid under distribution shift, keeping cover while growing doubt when going beyond. Tests on synthetic and benchmark data show improved robustness and cover, above all with limited data.
Catalog record
| CREATOR | Yu Chen, Scott Ferson |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28228 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.25882
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Conformalised imprecise inference for robust extrapolation under limited data Yu Chen, Scott Ferson Abstract Recent advances in uncertainty quantification increasingly emphasise the distinction between aleatory and epistemic uncertainty in machine learning, motivating the need for more unified frameworks. However, despite much progress in producing reliable predictions, existing methods often lack rigorous guarantees when generalising beyond the training domain. We propose a conformalised imprecise inference framework for robust extrapolation, which is model-agnostic and augments predictive models with imprecision and distance awareness. The proposed approach yields imprecise predictions (probability boxes) that remain valid under distributional shift, maintaining coverage while adaptively expanding uncertainty in extrapolation regimes. Experiments on synthetic and benchmark datasets demonstrate improved robustness and reliable coverage compared to standard probabilistic approaches, particularly under limited data. K eywords Imprecise probability \cdot Probability box \cdot Conformal prediction \cdot Extrapolation \cdot Limited data 1 Introduction Generalising beyond the data seen from learning is an essential task of machine learning systems [ 15 ] . Much progress in uncertainty quantification (UQ) for artificial intelligence (AI) has focused on addressing the overconfidence of trained models in high-stakes settings [ 10 , 20 , 22 , 15 ] . Importantly, recent work has increasingly distinguished between aleatory and epistemic sources of predictive uncertainty, with particular emphasis on the latter, focusing on challenges in limited data, measurement noise, model assumptions, distributional shift, and extrapolation [ 16 , 8 , 24 ] . Modern theories of uncertainty, e.g. the imprecise probability framework such as credal sets, evidence theory (Dempster Shafer structures), hierarchical higher-order distribution, uncertain numbers, fuzzy sets etc [ 6 , 7 , 4 ] , can represent much weaker statements of knowledge and more diverse types of uncertainty than conventional probability theory to reason under polymorphic uncertainty. Following the rapid development of approximate Bayesian inference in deep learning, these modern theories are applied to comprehensively analyse the uncertainty landscape of the machine learning systems [ 3 , 14 , 1 , 25 ] . The pursuit of uncertainty research align with the goal of enhancing the robustness of the AI systems, wherein statistical guarantee for generalisation is desired, even with distribution shift. Epistemic learning theory [ 21 ] emphasizes that the system must be designed to be prepared for data it has not yet encountered. Despite much progress towards the guarantee on the generalisation performance, with notable examples including PAC-Bayesian theory [ 11 ] and classic conformal prediction [ 22 ] , it is challenging to produce rigorous bounds that are guaranteed to enclose the ground truth under distributional shift, particularly in extrapolation settings. In this work, we investigate epistemic uncertainty in extrapolation settings, while systematically analysing the impact of limited data through controlled experiments. We propose a conformalised imprecise inference (CII) framework that integrates distance awareness with imprecise probability representations. In particular, we leverage a scalar distance score to characterise deviation from the training data manifold and use it to drive a distance-dependent discrepancy model. This discrepancy is embedded within a novel ascloseas bounds construction, yielding conformalised p-box predictions that maintain coverage while adaptively expanding uncertainty under distributional shift. 2 Conformalised framework for robust extrapolation 2.1 Background on predictive capability with uncertainty modelling Consider a general learning problem in a regression setting. Given observation data \mathcal{D}=\{(\mathbf{x}_{i},y_{i})_{i=1}^{N}\}\in(\mathcal{X}\times{\mathcal{Y}}) which are assumed to independent and identically distributed (i.i.d) with a distribution \mathcal{P}_{D} , the expected loss of a hypothesis h\in\mathcal{H}:\mathcal{X}\rightarrow\mathcal{Y} is given as \mathcal{L}=\mathbb{E}_{(\mathbf{x},y)\sim\mathcal{P}_{D}}\big[\ell({\mathbf{x}},y)\big] . To formulate heteroscedastic aleatory uncertainty, neural networks can be viewed as probabilistic models with a data-dependent noise structure p(y_{i}\mid f^{\boldsymbol{\omega}}(\mathbf{x}_{i}))=\mathcal{N}\left(\mu(\mathbf{x}_{i}),\sigma^{2}(\mathbf{x}_{i})\right) . The empirical loss, given by the negative log-likelihood, is written as: \mathcal{L}^{AU}=\frac{1}{N}\sum_{i=1}^{N}\frac{1}{2\sigma(\mathbf{x}_{i})^{2}}(y-\mu(\mathbf{x}_{i}))^{2}+\frac{1}{2}\log\sigma(\mathbf{x}_{i})^{2} (1) To model the epistemic uncertainty on model parameters, Bayesian deep learning evaluates the posterior weights p(\boldsymbol{\omega}|\mathcal{D}) given the observation data, see xx for variants of approximate Bayesian inference to handle the otherwise intractable inference problem. Notably, SVI turns the inference problem into optimisation of a stochastic objective named ELBO by introducing variational distribution q_{\phi}(\boldsymbol{\omega}) parameterised by \phi : \mathcal{L}^{EU}=\text{KL}[q_{\phi}(\boldsymbol{\omega})\ \|\ p(\boldsymbol{\omega})]-\mathbb{E}_{q_{\phi}(\boldsymbol{\omega})}\log p(\mathcal{D}|\boldsymbol{\omega}) (2) In compounding these two sources, researches differ in a mixture route to get compound distribution [ 17 , 19 ] or a credal route to obtain imprecise structures [ 3 , 14 ] . In addition, conformal prediction (CP) [ 13 ] presents as a new paradigm for quantifying predictive uncertainty by enriching a prediction from a hypothesis h into prediction sets. 2.2 Conformalised inference with distance awareness and distributional imprecision Compared to the classic CP where conformity guarantee is constructed on the assumption of exchangeability which may be violated under distributional shift, we proposed an extrapolation-focused approach where in-distribution (ID) and out-of-distribution (OOD) data are explicitly differentiated, in principle allowing the robustness against extrapolation problems on the basis of distributional awareness. Meanwhile, the proposed conformalised framework is also agnostic to any pre-trained model which may possess some capacity of uncertainty quantification. With respect to the trans-probabilistic predictive distributions yielded by the base hyposis, in which the predictive uncertainty could be manifested in different uncertainty models: probability distributions, a set of distributions (e.g. p-boxes), or intervals; whereas the observation may be precise or subject to measurement uncertainty. We employ the stochastic area metric as the conformal score, d(x) , suggesting the discrepancy between two random data generating processes, the predictive model p(y|\mathbf{x}) and the real-world observation \mathcal{Y}_{obs} , under polymorphic uncertainty. That is, in recognising the measurement uncertainty, the observation data could be elicited as intervals (say plus-minus form) or distributions (say PERT distribution). Generally, these are effectively uncertain numbers [ 4 ] at varying level of imprecision, and the stochastic area metric [ 9 ] in a generalised formulation is given by: d(x)\coloneqq\mathbb{E}_{x}\Delta([\underline{F}(x),\overline{F}(x)],[\underline{S}(x),\overline{S}(x)]) (3) where [\underline{F}(x),\overline{F}(x)] represent the bounds of the probability box of the prediction, while [\underline{S}(x),\overline{S}(x)] represent the empirical probability box from the data. Distance awareness has been identified as a key heuristic to reflect distribution shift [ 20 ] . To characterise extrapolation in the covariate space, we project inputs onto a distance space \mathcal{R}\in\mathbb{R} embedding the deviation of a testing data point from the training data manifold. Specifically, We define the distance score as: r(\mathbf{x})=\sqrt{(\mathbf{x}-\boldsymbol{\mu})^{T}\Sigma^{-1}(\mathbf{x}-\boldsymbol{\mu})} , which corresponds to the Mahalanobis distance. When \Sigma=I , this reduces to the Euclidean distance. The score r(x) induces an ordering over inputs that reflect their degree to distributional shift related to the training data. To learn how predictive uncertainty evolves with distance, we construct a calibration subset \mathcal{D}_{c}\subseteq\mathcal{D} in a hybrid manner that balances coverage of the input space using distance with an emphasis on boundary regions using the scenario optimisation [ 5 ] . Conformal scores are computed in \mathcal{D}_{c} whereby an interval predictor model (IPM) \mathcal{M}:r\rightarrow\mathcal{I}_{d} , is constructed to rigorously characterise the range \mathcal{I}_{d}=[\underline{d},\overline{d}] for unseen inputs for generalisation. It benefits from the statistical guarantee of high probability of covering the unseen response [ 2 ] . \mathcal{I}_{d}(r,\mathcal{P})=[\underline{d},\overline{d}]=\{f_{l}(r)\leq d(r)\leq f_{u}(r)\} (4) where \mathcal{M}=\{d(r)=M(r,\boldsymbol{q}),\boldsymbol{q}\in\mathcal{P}\} encodes a set of models parameterised by a vector \boldsymbol{q} ranging in the set \mathcal{P} . Practically it is prescribed by its upper and lower boundaries f_{u}(r)=\sum_{j=0}^{n_{u}}u_{j}\psi_{j}(r) and f_{l}(r)=\sum_{j=0}^{n_{l}}l_{j}\psi_{j}(r) , polynomial functions in which \psi_{j}(r) are basis function and n_{u} is the number of coefficients, that fully enclose the response with the minimal spread. This can be solved by