RAQG-QPP: Query Performance Prediction with Retrieved Query Variants and Retrieval Augmented Query Generation
Fangzheng Tian, Debasis Ganguly, Craig Macdonald · 2026
Read RAQG-QPP: Query Performance Prediction with Retrieved Query Variants and Retrieval Augmented Query Generation by Fangzheng Tian, Debasis Ganguly, Craig Macdonald (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27244.txt.
Introduction
Tian, Ganguly, and Macdonald propose RAQG-QPP, query quality guessing using retrieved query variants and query generation from retrieved queries. Guessing retrieval quality without human ratings helps query-specific choices, but label-free methods lag for neural rankers. Query variants, queries with similar information needs, can boost accuracy. Yet term-added variants often turn incoherent, made-up, or off-topic. Retrieved variants and generation from retrieved queries aim to supply variants that stay faithful to the original need.
Catalog record
| CREATOR | Fangzheng Tian, Debasis Ganguly, Craig Macdonald |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 86487 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.27244
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RAQG-QPP: Query Performance Prediction with Retrieved Query Variants and Retrieval Augmented Query Generation DOI: XXXXXXX.XXXXXXX Journal: TOIS Volume: Special Issue on Query Performance Prediction Towards Novel Information Retrieval Paradigms x xxx 8 Thanks: This is a preprint version of an accepted manuscript. The final version will appear in ACM Transactions on Information Systems . CCS: Information systems Information Retrieval Fangzheng Tian, Debasis Ganguly, Craig Macdonald Abstract . Query Performance Prediction (QPP) estimates the retrieval quality of ranking models without the use of any human-assessed relevance judgements, and finds applications in query-specific selective decision making to improve overall retrieval effectiveness. Although unsupervised QPP approaches are effective for lexical retrieval models, they usually perform weaker for neural rankers. Recent work shows that leveraging query variants (QVs), i.e., queries with potentially similar information needs to a given query, can enhance unsupervised QPP accuracy. However, existing QV-based prediction methods rely on query variants generated by term expansion of the input query, which is likely to yield incoherent, hallucinatory and off-topic QVs. In this paper, we propose to make use of queries retrieved from a log of past queries as QVs to be subsequently used for QPP. In addition to directly applying retrieved QVs in QPP, we further propose to leverage large language models (LLMs) to generate QVs conditioned on the retrieved QVs, thus mitigating the limitation of relying only on existing queries in a log. Experiments on TREC DL’19 and DL’20 show that QPP enhanced with RAQG outperform the best-performing existing QV-based prediction approach by as much as 30% on neural ranking models such as MonoT5. \acmArticleType Review \acmCodeLink https://github.com/borisveytsman/acmart \acmDataLink https://zenodo.org/link \acmContributions FT and DG designed the study; FT conducted the experiments, FT analysed the results, and all authors participated in writing the manuscript. 1. Introduction Query performance prediction (QPP) is the task of estimating the quality of retrieval results for a given query (referred to as the target query ) without relevance judgements. QPP plays an important role in that it can indicate how difficult a query is for a specific retrieval model ( He and Ounis 2006 ) , and whether retrieval pipeline optimisation is needed ( Datta et al. 2024 ) . As neural ranking models are increasingly applied in Information Retrieval (IR), developing effective QPP methods for those neural models has become an important task. However, unsupervised post-retrieval statistical QPP methods, such as NQC ( Shtok et al. 2012 ) and RSD ( Roitman et al. 2017 ) , have been reported to not perform satisfactorily well for neural rankers ( Datta et al. 2022 ) . The ineffectiveness of unsupervised QPP approaches for neural rankers primarily stems from their reliance on only the target query. As queries are usually short expressions of information needs comprised of a small number of terms, relying on only the input query may not adequately represent the underlying information need ( Zendel et al. 2019 ) . To alleviate this problem, incorporating retrieval results from queries related to the target query — commonly referred to as query variants ( QV s) — can provide additional useful signals to enhance QPP accuracy ( Shtok et al. 2016 ) . The effectiveness of leveraging QVs in QPP has been demonstrated by the frameworks proposed by Zendel et al. 2019 and Datta et al. 2022 . Datta et al. 2022 used a relatively simple generative approach based on query expansion (specifically, by means of pseudo-relevance feedback ( Jaleel et al. 2004 ) , and skip-gram word vectors ( Mikolov et al. 2013 ) ) to generate QVs in their study. However, the generated QVs often suffer from issues like hallucinations ( Gospodinov et al. 2023 ) and topical drift ( Alaofi et al. 2023 ) . These term-based approaches for generating query variants may be limited by restricted vocabularies and may not effectively capture the target query’s underlying information need. We hypothesise that the advent of large language models (LLMs) can be a promising approach for QV generation ( Brown et al. 2020 ) in the sense that LLMs can potentially generate QVs that better encapsulate the target query’s information need ( Ran et al. 2025 ) and reformulate it in different variations. However, zero-shot outputs from LLMs in QV generation task can be non-realistic and hallucinatory ( Farquhar et al. 2024 ) , in the sense that they may differ from the style of queries that human users can issue to a search engine. To illustrate with an example, when asked to reformulate the query ‘ lps laws definition ’, one of the zero-shot outputs is ‘ Limited Power of Sale laws: a detailed explanation ’, which resembles a declarative title more than an information-seeking query. We argue that this problem can be solved by appending real-life queries as context to the QV generation process, leading to k -shot contextual QV generation. In our proposed method, the queries acting as a context for k -shot QV generation are retrieved from a query log containing a large repository of queries submitted by users to a search system. Specifically, for our experiments, we use the MS MARCO training query set, which comprises queries collected from the Bing query log ( Bajaj et al. 2016 ) with a small number (close to 1 on average) of relevant documents for each query. We posit that these retrieved candidate QVs from query logs or IR training sets can potentially be useful in grounding an LLM’s QV generation process. Since these query variants potentially contain clues about how information needs evolve within search sessions of different users, they are likely to be more useful for QPP in comparison to the query log agnostic methods, e.g., LLM 0-shot or term-based generation approaches ( Datta et al. 2022 ) . Figure 1 . A visualisation of the idea behind our proposed QV retrieval method. For a target query (the first line of texts and the blue point in the graph), the first step is to retrieve a list of 1-hop QVs from an index constructed from the training query set. 1-hop QVs are represented by the green points. In the next step, the relevant document associated with each 1-hop candidate (shown as the italicised text) is used as a query to further retrieve a list of 2-hop QVs from the query index. These 2-hop QVs are shown in amber. All the 1-hop and 2-hop QVs comprise the candidate QV set, which are subsequently ranked into a list (shown as the ordered list on the left) by their topical similarity to the target query (illustrated by their distance to the blue point) to prepare for being used in QPP. Since our proposed QPP method relies on a set of top-retrieved QVs from a query log, an effective QV retrieval method is likely to improve downstream QPP performance. We hypothesise that using only the input query for retrieving similar queries from a collection of queries is limiting in the sense that these input queries are short expressions of information needs. To enrich the target query’s representation, we can leverage its associated documents ( Frayling et al. 2024 ) . Therefore, we propose to formulate the relevant documents of the top-retrieved QVs ( 1-hop QVs )