Learning to Rank with Variable Result Presentation Lengths Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Developmen
Norman Knyazev, Harrie Oosterhuis · 2025
Read Learning to Rank with Variable Result Presentation Lengths Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Developmen by Norman Knyazev, Harrie Oosterhuis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-23319.txt.
Introduction
Norman Knyazev and Harrie Oosterhuis study search rankings where results can take different amounts of space in a paper from 2025. Standard ranking methods assume each result is shown in the same format. Yet giving a result more room changes how users judge it and where their eyes go. The pair define a joint task: choose both the order of results and the space each one gets.
They show that a classic ranking rule breaks down in this setting. Ordering and sizing also cannot be solved apart. Their new method family, called VLPL, learns both choices together. Tests on semi-synthetic data favored the joint approach. Even simple space-aware methods beat fixed-format ones.
Catalog record
| CREATOR | Norman Knyazev, Harrie Oosterhuis |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-23319
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning to Rank with Variable Result Presentation Lengths Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’25), July 13–18, 2025, Padua, Italy DOI: 10.1145/3726302.3730020 ISBN: 979-8-4007-1592-1/2025/07 232 CCS: Information systems Learning to rank CCS: Information systems Presentation of retrieval results Norman Knyazev, Harrie Oosterhuis Abstract . LTR ( LTR ) methods generally assume that each document in a top- K K ranking is presented in an equal format. However, previous work has shown that users’ perceptions of relevance can be changed by varying presentations, i.e., allocating more vertical space to some documents to provide additional textual or image information. Furthermore, presentation length can also redirect attention, as users are more likely to notice longer presentations when scrolling through results. Deciding on the document presentation lengths in a fixed vertical space ranking is an important problem that has not been addressed by existing LTR methods. We address this gap by introducing the variable presentation length ranking task, where simultaneously the ordering of documents and their presentation length is decided. Despite being a generalization of standard ranking, we show that this setting brings significant new challenges: Firstly, the probability ranking principle no longer applies to this setting, and secondly, the problem cannot be divided into separate ordering and length selection tasks. We therefore propose VLPL – a new family of Plackett-Luce list-wise gradient estimation methods for the joint optimization of document ordering and lengths. Our semi-synthetic experiments show that VLPL can effectively balance the expected exposure and attractiveness of all documents, achieving the best performance across different ranking settings. Furthermore, we observe that even simple length-aware methods can achieve significant performance improvements over fixed-length models. Altogether, our theoretical and empirical results highlight the importance and difficulties of combining document presentation with LTR . † † cc-license: by 1. Introduction Ranking models generally aim to identify the most relevant documents from a set of candidates and present them to the user in the order of decreasing relevance on a SERP ( SERP ) ( Liu 2009 ) . However, the presentation of individual documents within a SERP is rarely considered when choosing the document ordering, with the documents usually simply modeled as fixed units of information or relevance. Nevertheless, in many cases the platforms can choose how each document is presented to the user, which in turn affects how the user perceives and interacts with those documents. We call this setting where both the order of the documents as well as their presentation lengths are chosen as the variable document presentation length setting – see Figure 1 for an example. This setting is characterized by the trade-off between presenting more information for displayed documents – potentially making them more attractive – or displaying more documents on the SERP – potentially including more relevant results. Figure 1 . Rankings with variable presentation length. Compared to SERP 1, SERP 2 presents the second item with more vertical space (length), allowing more information to be displayed. In SERP 3, the second and third item are both presented with less vertical space, resulting in less information for each, but allowing an additional item to be displayed. Earlier research has shown that increasing the size of individual SERP results to include additional textual information may make the user more likely to click through to the full document ( Maxwell et al. 2017 ) , whilst shorter results may discourage the users from clicking ( Kohavi 2015 ) . Similarly, in e-commerce, adding images and product details has also been found to increase the engagement with those items ( Bland et al. 2005 ; You et al. 2023 ) . Making a search result larger may reduce the probability that a user will miss it when browsing ( Cutrell and Guan 2007 ) , and thereby, alleviate some position bias on that result ( Marcos et al. 2015 ) . However, as noted by Pathak et al. 2024a ( Pathak et al. 2024b ; Pathak et al. 2024a ) , increasing the vertical size of some results generally means that fewer results fit on the SERP ( Kohavi 2015 ) . In general, increasing the presentation size of an individual retrieved document also means that the documents below it are pushed down by the same amount. This, in turn, may make them less likely to be observed by the user and lead to an increase in the effects of position bias ( Kelly and Azzopardi 2015 ; Kim et al. 2016 ; Craswell et al. 2008 ) . Hence this setting requires a balancing of document ordering and their presentation lengths. To the best of our knowledge, no existing method can simultaneously optimize a ranking and variable presentation lengths. In this work, we formally introduce and study the variable document presentation length setting. Our analysis reveals challenges that are not present in standard ranking: most importantly, the PRP ( PRP ) does not apply, and as a result, the optimal placement strategy cannot be separated into independent ranking and length-selection strategies. In order to tackle this problem, we extend the PL ( PL ) ranking approach to optimize both the order and presentation lengths of documents. Our novel model, the VLPL ( VLPL ) distribution \pi_{\text{VLPL}} , represents each choice as a document placement paired with a specific length. We build on the PL-Rank family of gradient estimators ( Oosterhuis 2021 ; Oosterhuis 2022 ) to introduce the novel VLPL -1 and VLPL -2 estimators. These efficiently estimate the gradient of \pi_{\text{VLPL}} , with VLPL -2 having a higher sample-efficiency. Moreover, as we show that the gradient estimation can be leveraged during model training as either in-processing or post-processing ( Caton and Haas 2024 ) , in total we propose four distinct methods for jointly optimizing ordering and length. Our semi-synthetic experiments demonstrate that our approaches achieve a large and consistent improvement over baseline methods. Overall, our findings underscore the need for ranking optimization methods with a more holistic view of result presentation that goes beyond the “ ten blue links ", as some links may look very different from others, and perhaps there should not be ten. 2. Related Work LTR models aim to identify the most relevant documents and present them to the user, most commonly in the form of a ranked list. This is usually done by scoring each document and then sorting the documents based on their scores. According to the PRP ( Robertson 1977 ) , a scoring function should reflect the probability of each document being relevant to the user to produce optimal rankings. In recent years, the Plackett-Luce family of list-wise methods ( Plackett 1975 ) have gained popularity, owing to their simplicity, robustness and interpretability ( Oosterhuis 2021 ; Oosterhuis 2022 ; Buchholz et al. 2022 ) . Importantly, however, these and other widely used ranking models ( Fuhr 1989 ; Joachims 2002 ; Burges 2010 ; Wang et al. 2018 ; Cao et al. 2007 ; Xia et al. 2008 ; Oosterhuis 2021 ; Oosterhuis 2022 ; Buchholz et al. 2022