Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation
Camilo Gomez, Pengyang Wang, Yanjie Fu* · 2026
Read Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation by Camilo Gomez, Pengyang Wang, Yanjie Fu* (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15101.txt.
Introduction
Gomez, Wang, and Fu design a ranking learner that is not tied to a single score. Current methods tune to one metric such as NDCG or MAP, which makes training rough and weakens transfer to other scores. Their listwise framework pairs a smooth ranking approximation with a per-query squared-error loss, trained by gradient boosting. Tests show it beats current leaders on retrieval measures at similar speed.
Catalog record
| CREATOR | Camilo Gomez, Pengyang Wang, Yanjie Fu* |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28441 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.15101
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation Camilo Gomez, Pengyang Wang, Yanjie Fu* Abstract Learning-to-Rank (LTR) is a supervised machine learning approach that constructs models specifically designed to order a set of items or documents based on their relevance or importance to a given query or context. Despite significant success in real-world information retrieval systems, current LTR methods rely on one prefix ranking metric (e.g., such as Normalized Discounted Cumulative Gain (NDCG) or Mean Average Precision (MAP)) for optimizing the ranking objective function. Such metric-dependent setting limits LTR methods from two perspectives: (1) non-differentiable problem: directly optimizing ranking functions over a given ranking metric is inherently non-smooth, making the training process unstable and inefficient; (2) limited ranking utility: optimizing over one single metric makes it difficult to generalize well to other ranking metrics of interest. To address the above issues, we propose a novel listwise LTR framework for efficient and generalizable ranking purpose. Specifically, we propose a new differentiable ranking loss that combines a smooth approximation to the ranking operator with the average mean square loss per query. Then, we adapt gradient-boosting machines to minimize our proposed loss with respect to each list, a novel contribution. Finally, extensive experimental results confirm that our method outperforms the current state-of-the-art in information retrieval measures with similar efficiency. I Introduction In the realm of machine learning and information retrieval, ranking is a pivotal process that organizes a collection of items or documents in an order that reflects their relevance or significance to a specific query or context. This process is integral to a wide array of machine learning applications, encompassing search engines, recommendation systems, and natural language processing [ 1 ] . A specialized branch of this discipline, known as Learning-to-Rank (LTR), employs a supervised machine learning approach to build models specifically designed to tackle ranking tasks. The learning-to-rank paradigm operates by training a model on a dataset composed of queries, each paired with a list of items. Each item within these lists is marked with a relevance grade, providing a measure of its pertinence to the associated query. This structured approach allows the model to learn the intricate relationships between queries and their relevant items, thereby enabling it to effectively rank new, unseen queries. Existing LTR literature can be categorized into three groups: (1) pointwise LTR, where each individual item in the training set is treated as an independent instance and is assigned a real-valued score or a class label indicating its degree of relevance to a query. The main drawback of pointwise LTR is ignoring the relative ordering or ranking of items within the same query, leading to suboptimal performance. (2) pairwise LTR, that considers pairs of items in the training set and aims to learn a model that correctly orders each pair based on their relative relevance to a query. The limitation of pairwise LTR is neglecting the overall ranking structure among all items associated with a query, resulting in inconsistencies in the final ranking output. (3) listwise LTR, which addresses the limitations of the above pointwise and pairwise LTR methods by treating the complete set of items linked to a query as a single entity during the training process, aiming to learn a model that optimizes the comprehensive ranking order of these items. Despite demonstrating notable performance, listwise learning-to-rank methods hinge on the optimization of specific ranking metrics, such as Normalized Discounted Cumulative Gain (NDCG) or Mean Average Precision (MAP). This reliance on metric-dependent learning criteria can give rise to two significant issues: (1) non-differentiable problem . To derive values for these ranking metrics, a ranking or sorting operation must be executed. However, these functions are inherently non-smooth, which poses a significant challenge when attempting to optimize them directly using gradient-based methods. To address this issue, the optimization of a continuous approximation of these metrics has been suggested. For example, SoftNDCG [ 2 ] provides a smooth, differentiable approximation to the NDCG, which can then be minimized using gradient descent. Nevertheless, these continuous approximations have been found to be computationally intensive, rendering them impractical for use in large-scale production systems. Specifically, SoftNDCG and ApproxAP [ 3 ] require O(n^{3}) and O(n^{2}) computations respectively to achieve these ranking approximations or “soft” ranks. Recently, an Optimal Transport (OT)-based soft ranking operator [ 4 ] has been introduced, which achieves soft ranking in O(Tnm) , where T is the number of Sinkhorn [ 5 ] iterations and m and n are hyperparameters. The first O(nlogn) soft ranking with O(n) differentiability has also been developed recently [ 6 ] . However, its performance in the context of Learning-to-Rank for Information Retrieval benchmark datasets (LETOR) remains to be evaluated, we do so in this paper. (2) Limited ranking utility . The information retrieval community has traditionally operated under the premise that optimizing evaluation metrics, such as NDCG or MAP, directly correlates with enhanced performance. However, recent studies have begun to challenge this assumption, suggesting that these metrics may intentionally serve as information bottlenecks and provide limited utility to user-centric applications [ 7 ] [ 8 ] . The need for evaluating ranking utility over a large spectrum of increasingly complex metrics has in turn increased the need for metric-agnostic algorithms [ 9 ] . This raises a pertinent question: How can we develop a versatile ranker that is not constrained by specific evaluation metrics, yet still delivers performance on par with the current state-of-the-art methods when evaluated using commonly used metrics? The pursuit of such a general-purpose ranker would significantly enhance the flexibility and applicability of ranking algorithms in various contexts. To address the above issues, in this paper, we propose a new performant metric-agnostic LTR framework for information retrieval systems. Our contributions and novelty can be summarized as follows: 1. We propose a novel differentiable loss function specifically designed for the ranking task. Our approach involves utilizing the average mean square loss between the true and predicted approximate ranks across all lists, thereby providing a genuine listwise measure. Specifically, we use “fast-soft-ranking” [ 6 ] as a building block for computing rank approximations. Our proposed measure is both continuous and differentiable, which are desirable properties for optimization. 2. To further enhance the learning process, we weigh these gradients by the derivatives of rank approximations. This strategy serves two key purposes. First, it eliminates the dependency on the evaluation metric, thereby increasing the flexibility of the ranking task. Second, it provides valuable ranking gradient information to the learners, which can guide the learning process more effectively. The ultimate goal of this approach is to minimize the loss function with respect to the learnable ranking function. 3. To validate the performance and efficiency of our proposed approach, we conducted a comprehensive series of experiments using real-world data sourced from major commercial search engines. The results demonstrate that our approach consistently outperforms the state-of-the-art LambdaMART [ 10 ] across a variety of information retrieval metrics. The remainder of the paper is organized as follows. We briefly introduce the information retrieval ranking problem formally and notation used throughout the paper in section 2. In section 3, we present our algorithm SoftRankGBM. We describe in detail the modifications to gradient boosting machines (GBM) and its integration with a