Localized Uncertainty Quantification in Random Forests via Proximities
1 st Jake S. Rhodes, 2 nd Scott D. Brown, 3 rd J. Riley Wilkinson · 2025
Read Localized Uncertainty Quantification in Random Forests via Proximities by 1 st Jake S. Rhodes, 2 nd Scott D. Brown, 3 rd J. Riley Wilkinson (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22928.txt.
Introduction
Jake S Rhodes, Scott D Brown, and J Riley Wilkinson add local uncertainty scores to random forests. Knowing how sure a model is matters most in high-stakes work. Common tools use quantile models or Monte Carlo runs. Their method instead uses the test points and closeness scores that a random forest already produces.
They build local error spreads from out-of-bag errors of nearby points. Those spreads give ranges for numeric forecasts and trust scores for class calls. Changing the count of nearby points tunes coverage to a target level. For classification, dropping points flagged as unclassifiable raised accuracy. Their accuracy-rejection scores beat rival methods in tests.
Catalog record
| CREATOR | 1 st Jake S. Rhodes, 2 nd Scott D. Brown, 3 rd J. Riley Wilkinson |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-22928
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Localized Uncertainty Quantification in Random Forests via Proximities 1 st Jake S. Rhodes, 2 nd Scott D. Brown, 3 rd J. Riley Wilkinson Abstract In machine learning, uncertainty quantification helps assess the reliability of model predictions, which is important in high-stakes scenarios. Traditional approaches often emphasize predictive accuracy, but there is a growing focus on incorporating uncertainty measures. This paper addresses localized uncertainty quantification in random forests. While current methods often rely on quantile regression or Monte Carlo techniques, we propose a new approach using naturally occurring test sets and similarity measures (proximities) typically viewed as byproducts of random forests. Specifically, we form localized distributions of OOB errors around nearby points, defined using the proximities, to create prediction intervals for regression and trust scores for classification. By varying the number of nearby points, our intervals can be adjusted to achieve the desired coverage while retaining the flexibility that reflects the certainty of individual predictions. For classification, excluding points identified as unclassifiable by our method generally enhances the accuracy of the model and provides higher accuracy-rejection AUC scores than competing methods. I Introduction Although traditional machine learning models usually provide point estimates, there is growing recognition of the need to incorporate uncertainty to support more informed decisions [ 1 ] . By quantifying uncertainty, users can assess the reliability of model outputs and better interpret results, especially for out-of-distribution samples through calibrated confidence estimates. Global measures, such as mean squared error (MSE), overall accuracy, precision, or recall, are usually used to quantify the quality of machine learning models. However, additional insights into model performance can be provided by assessing local or instance-based contributions [ 2 ] . This involves considering the likelihood of prediction accuracy based on similarities with other observations, thus providing a more nuanced understanding of model reliability. In doing so, users can identify regions of the data in which high or low levels of confidence in the predicted values should be placed. In this work, we focus on the uncertainty quantification of random forests. We use a naturally occurring test set that is often viewed as a byproduct of random forests to determine an error distribution that we can extend to new points. Localized error distributions are used to generate prediction intervals (for regression problems) or trust scores (for classification problems) that provide additional information on the decision-making process. I-A Random Forests Random forests were introduced in 2001 by Leo Breiman [ 3 ] and continue to serve as a reliable model in scientific fields. However, like many other machine learning models, random forests are still widely considered black-box models. That is, the path from the data to the prediction is not conveyed in a human-interpretable way. As a partial resolution, the random forest predictions can be described in terms of a weighted sum of responses, as first suggested in [ 4 ] , and later extended to classification problems in [ 5 ] . From this point of view, the random forest predictions can be broken down into instance-based contributions, where the weights describe how specific training instances influence the prediction. In [ 5 ] , the authors described these random forest weights as proximities or similarity measures. They demonstrated their superiority over other random forest proximities in several tasks, including data imputation, outlier detection, and visualization [ 6 ] . It was argued that because these proximities could serve as weights capable of reconstructing the random forest predictions, they preserved the data geometry learned by the random forest and are thus called random forest geometry and accuracy-preserving proximities, or RF-GAP. We use the RF-GAP proximities to provide instance-based (localized) measures of uncertainty. II Related Works Measures of uncertainty in random forests span several approaches, including variance- or density-based measures at the leaf or tree level, bootstrapping or Monte Carlo resampling, and methods leveraging OOB (OOB) errors or proximities. Most works focus on regression, though classification uncertainty has also been addressed. We summarize these works below. II-A Regression Methods Early approaches assessed uncertainty using structural properties of the trees. Dutschmann and Baumann [ 7 ] introduced high-variance leaf nodes, deeming predictions less certain when many such nodes were involved, though this rarely improved over standard deviations of the ensemble. Meinshausen [ 8 ] proposed quantile regression forests (QRFs), which extend random forests by estimating conditional distributions rather than conditional means. Instead of storing only node averages, QRFs retain all response values within each terminal node, from which a weighted empirical distribution is constructed. This enables direct estimation of quantiles, allowing for prediction intervals and outlier detection. While QRFs provide flexible, distributional predictions, they do not explicitly account for random forest error distributions. Bootstrap-based strategies estimate variance through resampling. Wager et al. [ 9 ] developed jackknife and infinitesimal jackknife variance estimates, attributing uncertainty to Monte Carlo and sampling noise, though requiring impractically large resamples. Coulston et al. [ 10 ] trained thousands of forests on bootstrap samples, producing well-calibrated intervals but at high computational cost. Hooker and Mentch [ 11 ] applied PAC theory and U-statistics for prediction intervals with guaranteed coverage, though exhaustive computation is infeasible, so incomplete subsampling is required. OOB error distributions provide a natural test set for interval construction. Zhang et al. [ 12 ] used OOB residuals with conformal analysis to build prediction intervals, which achieve valid asymptotic coverage and are narrower than QRFs, but remain globally uniform and thus less informative at the instance level. Recent work also highlights the role of proximities for local explanation and error attribution, showing that only a small fraction of nearest neighbors contribute meaningfully to predictions [ 2 ] . II-B Classification Methods For classification, uncertainty measures often rely on density or proximity information. Fornaser et al. [ 13 ] proposed the sigma-z random forest, estimating uncertainty from split-based probability densities. The method assigns lower confidence to misclassified points but is slow to train and lacks comprehensive evaluation. Proximity-based methods instead exploit neighborhood structure. Devetyarov and Nouretdinov [ 14 ] measured uncertainty by comparing average proximities of same-class versus other-class neighbors, producing effective conformity scores for OOB samples, though extension to unseen points remains unclear. More generally, proximity information can be leveraged to identify heterogeneous decision regions where predictions are less trustworthy. Random forests provide information about local relationships between points encoded as weights or proximities. Information from nearby points can be used to supplement the model’s prediction. In [ 2 ] , random forest weights are used to provide instance-level model explainability. They show that only a small fraction (15% in their case) of the nearest neighbors contributed to the actual prediction of their regression problem. Nearest-neighbor errors were attributed to errors in the nearby training points. Similarly to this approach, we use the distribution of out-of-bag (OOB) errors to approximate test error. Unlike global uncertainty estimates, however, our method builds localized error distributions by incorporating proximity between training and test points. This accounts for varying noise levels across the feature space: a test error estimate is weighted by the closeness of training points. Test points with predictions far from the truth thus receive wider intervals, reflecting nearby OOB errors. III Random Forest Proximities A random forest is a supervised learning model that predicts