Multi-Fidelity Quantile Regression

Yixiang Liu, Yao Zhang · 2026

Read Multi-Fidelity Quantile Regression by Yixiang Liu, Yao Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10406.txt.

Introduction

Yixiang Liu and Yao Zhang help estimate quantiles when the best data are scarce and costly. High-quality measures, such as wet-lab binding assays taking days or weeks, are costly, while abundant low-quality data are imperfect proxies. Their 2026 paper proposes a two-stage, general method for multi-fidelity quantile regression, estimating response quantiles given covariates.

The core idea is a local link: at each covariate value, the high-quality quantile is written as a low-quality quantile at a varying level. Estimating that level function can be easier than estimating the high-quality quantile directly when the two distributions have similar shapes. A correction step improves robustness where that edge weakens. Theory pins down when the estimator converges faster than direct high-quality-only regression and when correction helps further. Tests on synthetic and real data yielded more accurate quantile estimates and tighter conformal prediction intervals.

Catalog record

CREATORYixiang Liu, Yao Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.10406

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Fidelity Quantile Regression Yixiang Liu, Yao Zhang Abstract High-fidelity (HF) data are often expensive to collect and therefore scarce, making conditional quantiles difficult to estimate accurately. We propose a two-stage, model-agnostic method for multi-fidelity quantile regression. The central idea is a local quantile link: at each covariate value, the HF quantile is represented as a low-fidelity (LF) quantile evaluated at a covariate-dependent level. This reformulation reduces the problem to estimating the level function, which can be smoother than the HF quantile itself when the LF and HF conditional distributions have similar shapes. We also study the complementary regime in which this advantage weakens and introduce a correction step to improve robustness. Our theory characterizes when the proposed estimator converges faster than direct quantile regression using HF data alone and when the correction step provides further improvement. Experiments on synthetic and real data show that our method yields more accurate quantile estimates and tighter conformal prediction intervals. 1 Introduction In many scientific applications, the most reliable measurements are also the most expensive and time-consuming to obtain. In molecular design, for example, wet-lab binding assays provide accurate estimates of binding affinity, but each experiment, from assay setup to quality control, may take days or even weeks to complete ( 49 ; 5 ) . Constraints of this kind are common across experimental science and form a major obstacle in areas such as drug discovery and materials design, where faster progress could have substantial societal impact by helping address urgent challenges such as major diseases and climate change ( 41 ; 39 ; 2 ) . In recent years, researchers have increasingly turned to machine learning methods trained on multi-fidelity data to accelerate scientific discovery ( 18 ; 21 ; 6 ; 44 ) . However, achieving highly accurate point prediction remains difficult because abundant low-fidelity data are only imperfect proxies for the scarce high-fidelity measurements obtained from real experiments. In settings like these, quantifying uncertainty in model predictions becomes crucial for decision-making, for example, in guiding future data collection or deciding which candidates should be validated through expensive real-world experiments ( 45 ) . In quantitative fields, quantile regression (QR) is a standard tool for uncertainty quantification ( 19 ; 20 ) . It estimates conditional quantiles of the response given covariates and can be implemented using a range of machine learning models, such as random forests ( 24 ) and neural networks ( 40 ; 3 ) . QR is especially useful for modeling heteroscedastic, asymmetric, or heavy-tailed data, where lower and upper conditional quantiles can be combined to form prediction intervals for the response without specifying a parametric model for the full conditional distribution. More recently, 37 combined QR with conformal prediction to construct marginally valid prediction intervals. Under the standard i.i.d. assumption, these intervals are guaranteed to achieve the desired coverage on average over the population, regardless of the model used to estimate the quantiles. However, with limited HF data, estimating conditional quantiles can be substantially harder than estimating conditional means. At a fixed covariate value x , the variance of a conditional quantile estimator typically scales as \mathrm{Var}\!\left(\hat{Q}_{Y\mid X}(\tau\mid x)\right)\approx\frac{\tau(1-\tau)}{n\,f_{Y\mid X}\!\left(Q_{Y\mid X}(\tau\mid x)\mid x\right)^{2}}, (1) where n is the sample size, and f_{Y\mid X}(\cdot\mid x) is the conditional density evaluated at the target quantile ( 20 ) . When both the sample size and this density are small, the fitted quantiles can be highly variable. Even when conformal prediction guarantees marginal coverage, the resulting intervals may still be wide or unstable if the fitted quantiles are noisy. This makes it important to exploit auxiliary information whenever it is available. Although HF data are scarce, low-fidelity (LF) data can often be obtained at scale. In molecular design, for example, molecular docking or lower-level quantum calculations provide cheaper but less accurate approximations to binding affinity ( 28 ; 17 ) . Multi-fidelity learning seeks to leverage such abundant LF observations to compensate for the scarcity of HF data ( 31 ) . However, most existing multi-fidelity methods focus on conditional means or other average response functionals ( 18 ; 21 ; 32 ; 25 ; 7 ) , and extending these ideas to quantile regression is less straightforward. Unlike conditional means, conditional quantiles can differ across fidelity levels in nonlinear, level-dependent ways. For instance, even if molecular docking scores track average binding affinity well after a simple rescaling, their conditional distribution may be substantially noisier or more skewed than the wet-lab measurements. Consequently, it is not obvious which LF quantile level should correspond to a given HF quantile, or how to correct this mismatch without imposing strong parametric assumptions. 1.1 Overview Motivated by these challenges, we propose Multi-Fidelity Quantile Regression (MFQR), which rests on the following link between the two fidelity levels: Assumption 1 (Local Quantile Link) . For a fixed target level \tau and any covariate value x\in\mathcal{X} , the HF and LF quantile functions satisfy Q^{H}(\tau\mid x)=Q^{L}(u_{\tau}(x)\mid x), (2) for some covariate-dependent level function u_{\tau}(x)\in(0,1) . Assumption 1 means that the target HF quantile can be matched locally by an LF quantile at a different, covariate-dependent probability level u_{\tau}(x) . For instance, if the LF distribution is heavier-tailed than the HF distribution, then u_{\tau}(x) should lie closer to 0.5 than \tau , so that a more central LF quantile matches the more extreme HF quantile. Thus, the assumption is mild whenever the support of the LF conditional distribution contains the target HF quantile. In particular, it holds formally for common full-support location-scale models, such as Gaussian, Student- t , logistic, or Laplace conditional distributions. If the two conditional distributions are centered at different locations, we can first center the responses by subtracting conditional mean estimators and then apply the same quantile-link formulation to the resulting residuals. When the two fidelity levels have similar conditional shapes, the shifted probability level u_{\tau}(x) may vary more smoothly over the covariate space than the original HF quantile function. This suggests that reformulating quantile estimation on the probability scale can reduce the sample complexity of estimating the high-fidelity quantile. Motivated by this idea, we propose a model-agnostic wrapper that uses the low-fidelity distribution to learn the level function u_{\tau}(x) , together with a correction step based on the high-fidelity quantile equation to refine the quantile estimate. Our theory studies when the wrapper estimator converges faster than direct HF quantile regression and when the correction step can yield further improvements. Experiments on synthetic and real datasets show that the proposed method produces more accurate quantile estimates and narrower conformal intervals than baseline methods while preserving marginal coverage guarantees. The remainder of this paper is organized as follows. Section 2 reviews related work. Section 3 introduces the wrapper estimator and establishes its theoretical convergence properties. Section 4 develops a correction framework for refining the wrapper estimator. Finally, Section 5 demonstrates the empirical performance of MFQR on synthetic and scientific datasets. 2 Related Work This paper relates to several strands of literature: multi-fidelity modeling, transfer learning, quantile regression, and conformal prediction. A classical starting point for multi-fidelity modeling is the autoregressive framework of 18 , which models the high-fidelity (HF) response as a scaled version of the low-fidelity (LF) response plus a discrepancy term. This framework

Read the complete text