How You Ask Matters! Adaptive RAG Robustness to Query Variations

Yunah Jang, Megha Sundriyal, Kyomin Jung, Meeyoung Cha · 2026

Read How You Ask Matters! Adaptive RAG Robustness to Query Variations by Yunah Jang, Megha Sundriyal, Kyomin Jung, Meeyoung Cha (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10745.txt.

Introduction

Jang and three colleagues build the first large benchmark of diverse queries that mean the same thing, for adaptive retrieval-augmented generation, combining human-written and model-made rewrites. Adaptive RAG fetches text only when needed, but real queries vary in surface form with the same intent, and that impact was little studied. Judging answer quality, compute cost, and fetch choices, they find a critical weak spot: small surface changes sharply alter fetch behavior and accuracy. Larger models perform better, yet toughness does not improve accordingly.

Catalog record

CREATORYunah Jang, Megha Sundriyal, Kyomin Jung, Meeyoung Cha
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74383
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10745

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

How You Ask Matters! Adaptive RAG Robustness to Query Variations Yunah Jang, Megha Sundriyal, Kyomin Jung, Meeyoung Cha Abstract Adaptive Retrieval-Augmented Generation (RAG) promises accuracy and efficiency by dynamically triggering retrieval only when needed and is widely used in practice. However, real-world queries vary in surface form even with the same intent, and their impact on Adaptive RAG remains under-explored. We introduce the first large-scale benchmark of diverse yet semantically identical query variations, combining human-written and model-generated rewrites. Our benchmark facilitates a systematic evaluation of Adaptive RAG robustness by examining its key components across three dimensions: answer quality, computational cost, and retrieval decisions. We discover a critical robustness gap, where small surface-level changes in queries dramatically alter retrieval behavior and accuracy. Although larger models show better performance, robustness does not improve accordingly. These findings reveal that Adaptive RAG methods are highly vulnerable to query variations that preserve identical semantics, exposing a critical robustness challenge. 1 1 footnotetext: Work done during an internship at Max Planck Institute for Security and Privacy. 2 2 footnotetext: Co-corresponding authors. 1 Introduction Retrieval-Augmented Generation (RAG) reduces hallucinations in large language models (LLMs) by grounding outputs in retrieved evidence, thereby improving factual accuracy in downstream tasks 22 ; 25 ; 2 . While effective in reducing factual errors, conventional RAG often retrieves irrelevant or noisy documents, which can mislead reasoning and degrade answer quality 26 ; 12 ; 11 . Figure 1: Retrieval-decision flip rates of the Qwen-32B model, measuring how often the model changes its retrieval judgment under meaning-preserving human query rewrites. Higher rates signal instability. Flips are categorized as One-flip (only one rewrite flips) and Both-flip (both rewrites flip). To address these limitations, Adaptive RAG dynamically decides when and what to retrieve based on the query and evolving context 19 ; 47 . If the model can answer confidently from parametric knowledge, it may skip retrieval and avoid interference from noisy documents. While simple factoid questions may need only a few retrievals, multi-hop questions often benefit from iterative subqueries that accumulate evidence across hops 17 ; 45 ; 46 . Prior adaptive methods have therefore improved both accuracy and efficiency by allocating retrieval and LLM calls according to query difficulty and intermediate states 43 ; 20 ; 12 . These gains have driven growing adoption of Adaptive RAG in real-world applications 42 . Figure 2: Example of Adaptive RAG responses under human, original, and LLM-generated query rewrites, highlighting differences in answer correctness, computation overhead, and retrieval score. In real-world settings, users express the same intent in diverse ways, including paraphrases, stylistic variations, and typos or other noise 13 ; 50 ; 10 . To assess decision stability, we measure how often the model’s retrieval decision (retrieve or not) flips between the original query and two human rewrites. As shown in Figure 1 , human rewrites cause decision flips in up to 55% of cases, revealing severe instability in retrieval decisions. Despite this, robustness to real-world query variation remains underexplored, which is especially concerning in high-stakes settings where factual correctness and user trust are critical 40 ; 32 . This paper presents the first empirical study of Adaptive RAG robustness under realistic query variations. To facilitate this, we introduce a benchmark spanning controlled linguistic paraphrases and naturally written human queries. We evaluate on six widely used question-answering (QA) datasets covering both single-hop and multi-hop questions and analyze how surface-level differences affect both intermediate retrieval trajectories and final answers (Figure 2 ). For each question, we include six model-generated variants and two human rewrites, yielding 27K QA pairs spanning diverse levels of complexity. Across these query variations, we comprehensively assess model behavior along three dimensions: answer quality, computational cost, and retrieval decisions. This work presents the following key findings: 2 2 2 The sample dataset and code are provided as supplementary materials. We will release them publicly upon acceptance. • Improved performance does not imply greater robustness; even strong baselines show substantially greater inconsistency in answer semantics and computational cost. • Comprehensive experiments reveal that all evaluated methods and models are consistently vulnerable to both human rewrites and spelling errors. • Semantically identical queries with different surface forms can trigger different intermediate trajectories, subquery generation, retrieval, and the final answer. 2 Related Works Adaptive RAG. Adaptive RAG implements dynamic, model-driven decisions about when to retrieve and what to retrieve 12 . We broadly categorize prior methods into two main streams: logit-based and generation-based methods. Logit-based methods monitor model confidence and trigger retrieval when the model appears uncertain, for example, using token-level probabilities as in 20 or uncertainty estimates based on token probabilities and attention weights as in 43 . Generation-based methods, on the other hand, let the model decide on the fly when it needs retrieval by generating search queries during the generation process 27 . Our work focuses on approaches that use a single model with a retriever and do not rely on external classifiers, such as 19 ; 47 . Original Query: Who is the child of the director of film Mukhyamantri (1996 Film)? Category Variation Sample Rewrites Style Formality (Form.) \downarrow Yo, quick one — who’s the kid of the director behind the 1996 flick Mukhyamantri? Readability (Read.) \downarrow By whom is the progeny of the director of Mukhyamantri (1996 Film) constituted? Type Declarative (Decl.) I want to know who is the child of the director of the film Mukhyamantri (1996 Film). Imperative (Impr.) Name the child of the director of film Mukhyamantri (1996 Film). Error Spelling (Spell.) \downarrow Who is ght childer ar the directer of filme Mukhyamantri (1996 Film )? Grammar (Gram.) \downarrow Who are the childs of the director from film Mukhyamantri 1996? Table 1: Examples of query rewrites grouped by category, illustrating stylistic, structural, and error-based variations. \downarrow denotes reduction in corresponding writing variation. Query Robustness. Prior work studied the robustness of RAG systems. More broadly, traditional IR research has shown that query variations can affect retrieval rankings 6 ; 9 ; 29 , compromise evaluation reliability 52 , and reveal substantial human query variability 5 ; 1 . In the context of QA and RAG, prior studies have demonstrated that linguistic perturbations, including query-entry errors, grammatical errors, typos, and stylistic variations, can noticeably degrade system performance 10 ; 35 ; 49 . Collectively, these results indicate that even small modifications to a query may lead to substantial changes in retrieved evidence and downstream answers 50 ; 13 ; 4 . Despite these advances, existing work has two important limitations: (1) it primarily considers conventional RAG, without addressing the added complexity of adaptive RAG; and (2) it does not investigate the impact of real human query reformulations across end-to-end RAG frameworks. 3 Benchmark Construction 3.1 Dataset We evaluate on six QA datasets: Natural Questions (NQ) 23 , TriviaQA 21 , and SQuAD v1.1 37 for single-hop QA, and 2WikiMultiHopQA (2WIKI) 17 , MuSiQue 45 , and HotpotQA 48 for multi-hop QA. 3.2 Query Variation Generation We design a taxonomy of machine-generated queries that captures realistic query variation while remaining interpretable and controllable for diagnostic analysis. It comprises three categories: sentence style, sentence type, and errors, as shown in Table 1 . While prior work has mainly studied style and

Read the complete text