Post Hoc Regression Refinement via Pairwise Rankings
Kevin Tirta Wijaya, Michael Sun, Minghao Guo, Hans-Peter Seidel, Wojciech Matusik, Vahid Babaei · 2025
Read Post Hoc Regression Refinement via Pairwise Rankings by Kevin Tirta Wijaya, Michael Sun, Minghao Guo, Hans-Peter Seidel, Wojciech Matusik, Vahid Babaei (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-16495.txt.
Introduction
Expert rankings can rescue a regression model starved of labels, Kevin Tirta Wijaya and co-authors show in a 2025 paper. Their add-on method takes any trained predictor and a small reference set with known values. Pairwise rankings of a new item against that set blend with the model's guess, weighted by trust.
No retraining is needed. On molecular property tasks, twenty comparisons from a general chat model cut error by up to a tenth. Human experts rank easily where exact labels are costly, so the trick travels across science and engineering wherever data are scarce.
Catalog record
| CREATOR | Kevin Tirta Wijaya, Michael Sun, Minghao Guo, Hans-Peter Seidel, Wojciech Matusik, Vahid Babaei |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-16495
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Post Hoc Regression Refinement via Pairwise Rankings Kevin Tirta Wijaya, Michael Sun, Minghao Guo, Hans-Peter Seidel, Wojciech Matusik, Vahid Babaei Abstract Accurate prediction of continuous properties is essential to many scientific and engineering tasks. Although deep-learning regressors excel with abundant labels, their accuracy deteriorates in data-scarce regimes. We introduce RankRefine , a model-agnostic, plug-and-play post hoc method that refines regression with expert knowledge coming from pairwise rankings. Given a query item and a small reference set with known properties, RankRefine combines the base regressor’s output with a rank-based estimate via inverse-variance weighting, requiring no retraining. In molecular property prediction task, RankRefine achieves up to 10% relative reduction in mean absolute error using only 20 pairwise comparisons obtained through a general-purpose large language model (LLM) with no finetuning. As rankings provided by human experts or general-purpose LLMs are sufficient for improving regression across diverse domains, RankRefine offers practicality and broad applicability, especially in low-data settings. 1 Introduction Accurate prediction of continuous properties is crucial across scientific and engineering disciplines. In molecular-property prediction (MPP), for instance, reliable estimates of physical or chemical attributes can accelerate drug discovery, materials design, and catalyst development. Recent advances in deep learning have enhanced regression models by leveraging large datasets to uncover complex, nonlinear relationships between structured inputs (e.g., molecular graphs, crystal structures) and their associated properties. Yet, unlike computer vision or natural-language processing, where large labeled corpora can be mined or crowd-sourced, many specialized fields face a fundamental bottleneck: acquiring ground-truth labels requires expert-led experiments that are both costly and slow. Consequently, real-world tasks often operate in data-scarce regimes where even 50 labeled samples may be considered plentiful [ Sun et al. 2024 ] . When large-scale labels are unavailable, expert knowledge becomes a valuable but under-utilized asset. One accessible form of such knowledge is relative comparison: pairwise judgments about which of two samples exhibits a higher (or lower) property value. Pairwise rankings convey substantial information, are often easier for human experts to provide, and can even be generated by general-purpose large language models (LLMs), which have shown surprising ranking skills in chemistry and related domains [ Sun et al. 2025 , Guo et al. 2023 ] . We introduce RankRefine , a post-hoc refinement framework that improves regression predictions by incorporating pairwise rankings. Given a query sample and a small set of labeled references, which can conveniently be drawn from training data, a ranker infers the relative ordering between the query and each reference; these comparisons define a likelihood whose minimization yields a rank-based property estimate. RankRefine then fuses this estimate with the base regressor’s prediction via inverse-variance weighting, all without retraining. Because RankRefine is largely model-agnostic and requires no architectural changes, it suits low-data, few-shot, and meta-learning scenarios. Leveraging ranking signals from publicly hosted LLMs, it provides substantial improvements in regression accuracy with minimal labeled data and negligible client-side computation. Experiments on synthetic and real-world benchmarks, including multiple MPP tasks, demonstrate that RankRefine consistently boosts predictive performance. Further analysis links its effectiveness to ranking quality, underscoring the promise of LLMs and other expert-informed rankers as practical complements to data-scarce regression models. The source code is provided in the supplementary materials. 2 Background 2.1 Combining Regression and Ranking Huang et al. 2024 reformulate regression as joint regression–pairwise-ranking objectives and show that the combination improves predictive accuracy. Tynes et al. 2021 , Fralish et al. 2023 instead train a regressor to predict pairwise differences in molecular properties; at inference, a query molecule is compared with reference molecules of known labels, and the mean of the predicted differences yields the final estimate. The post hoc method proposed by Gonçalves et al. 2023 refines regression outputs by computing a weighted average of the original prediction and those of its top-k nearest neighbors. A closely related approach is the consolidation method for document relevance proposed by Yan et al. 2024 . Given a pointwise rating and a set of pairwise comparisons, their post-processor makes minimal adjustments to preserve the ranker-derived inequalities. For a one-dimensional rating \hat{y} this projection simplifies to \hat{y}^{*}=\max\!\bigl(\min(\hat{y},U),L\bigr), where L and U are the lower and upper bounds of the feasible interval. Preference learning with Gaussian processes [ Chu and Ghahramani 2005 , Houlsby et al. 2012 , Chau et al. 2022 ] utilize pairwise data but differ fundamentally from our setting. Preference learning assumes a latent score f(x) whose absolute scale is irrelevant; the aim is to preserve ordering and learn-to-rank, not to predict calibrated values. In contrast, we target real-world quantities, e.g., solubility or toxicity, and treat pairwise comparisons as auxiliary signals that refine supervised regression into meaningful, calibrated estimates. Similarly, many learning-to-rank approaches [ Burges et al. 2005 , Yildiz et al. 2020 , Tom et al. 2024 ] may seem related to our work, but they actually address a different problem, i.e., ranking rather than regression. 2.2 LLM as a Pairwise Ranker Large language models (LLMs) have recently been explored in molecular-property prediction and generation [ Sun et al. 2025 ] . Several efforts fine-tune GPT-3- or LLaMA-based models with chemistry-specific prompts [ Jablonka et al. 2024 , Xie et al. 2024 , Jacobs et al. 2024 ] , while MolecularGPT [ Liu et al. 2024 ] is instruction-tuned on 1 000 property-prediction tasks and achieves few-shot performance competitive with graph neural networks. Fine-tuning, however, is resource-intensive and incurs substantial computational overhead. We therefore ask: can general-purpose, publicly hosted LLMs be leveraged directly, without domain-specific adaptation? We hypothesize that such models can supply useful signals when deployed as pairwise rankers. Human cognition offers an analogue. People often judge quantities either by estimating absolute magnitudes (regression) or by deciding which of two items is greater (ranking). A long-standing hypothesis holds that comparative judgments are easier and less biased [ Thurstone 1927 , Miller 1956 , Stewart et al. 2005 ] . Pairwise tasks impose lower cognitive load [ Routh et al. 2023 ] and avoid scale-interpretation bias [ Hoeijmakers et al. 2024 ] . Because LLMs are pretrained on corpora rich in relative statements and later aligned with human feedback [ Zhang et al. 2023 , Bai et al. 2022 , Rafailov et al. 2023 ] , they may inherit a similar advantage. As a real example, Guo et al. 2023 show that off-the-shelf LLMs rival domain-specific models on chemistry classification and ranking benchmarks, underscoring the promise of pairwise prompting in specialized tasks. 3 Method Figure 1 : Overview of the RankRefine framework. During training, a regressor is trained using labeled data. At inference, a query sample is paired with reference samples with known properties and compared by an external ranker to produce pairwise rankings. These rankings are used to estimate the query’s property value via a rank-based estimator. The final prediction is obtained by fusing the regressor’s output and the rank-based estimate using inverse variance weighting. Our goal is to reduce the test-time error of an existing regressor by fusing its prediction with pairwise-ranking information in a plug-and-play fashion. We present RankRefine , a framework that augments a base regressor with rankings from an external source (e.g., an LLM or a domain expert). RankRefine is largely model-agnostic and applies to any regressor that returns both a point estimate and an uncertainty measure,