RealRoute : Dynamic Query Routing System via Retrieve-then-Verify Paradigm

Jiahe Liu, Qinkai Yu, Jingcheng Niu, Xi Zhu, Zirui He, Zhen Xiang, Fan Yang, Jinman Zhao · 2026

Read RealRoute : Dynamic Query Routing System via Retrieve-then-Verify Paradigm by Jiahe Liu, Qinkai Yu, Jingcheng Niu, Xi Zhu, Zirui He, Zhen Xiang, Fan Yang, Jinman Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20860.txt.

Introduction

Liu, Yu, Niu, Zhu, He, Xiang, Yang, and Zhao introduce RealRoute, a framework that shifts from predictive routing to a Retrieve-then-Verify design. Existing routers use an LLM to send split sub-queries to specific sources in advance, which fails when source boundaries are unclear. RealRoute first pulls evidence in parallel from all sources, without regard to source type, then a dynamic verifier cross-checks the results and builds a factually grounded answer. Tests show RealRoute beats predictive baselines by a wide margin in multi-hop RAG reasoning.

Catalog record

CREATORJiahe Liu, Qinkai Yu, Jingcheng Niu, Xi Zhu, Zirui He, Zhen Xiang, Fan Yang, Jinman Zhao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27671
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.20860

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RealRoute : Dynamic Query Routing System via Retrieve-then-Verify Paradigm Jiahe Liu, Qinkai Yu, Jingcheng Niu, Xi Zhu, Zirui He, Zhen Xiang, Fan Yang, Jinman Zhao Abstract Despite the success of Retrieval-Augmented Generation (RAG) in grounding LLMs with external knowledge, its application over heterogeneous sources (e.g., private databases, global corpora, and APIs) remains a significant challenge. Existing approaches typically employ an LLM-as-a-Router to dispatch decomposed sub-queries to specific sources in a predictive manner. However, this "LLM-as-a-Router" strategy relies heavily on the semantic meaning of different data sources, often leading to routing errors when source boundaries are ambiguous. In this work, we introduce RealRoute System , a framework that shifts the paradigm from predictive routing to a robust Retrieve-then-Verify mechanism. RealRoute ensures evidence completeness through parallel, source-agnostic retrieval, followed by a dynamic verifier that cross-checks the results and synthesizes a factually grounded answer . Our demonstration allows users to visualize the real-time "re-routing" process and inspect the verification chain across multiple knowledge silos. Experiments show that RealRoute significantly outperforms predictive baselines in the multi-hop Rag reasoning task. The RealRoute system is released as an open-source toolkit with a user-friendly web interface. The code is available at the URL: https://github.com/Joseph1951210/RealRoute . System Multi-Hop Multi-Source Retrieval Cross-Domain Knowledge Check Naive RAG 24 ✗ ✗ ✗ ✗ HippoRAG 18 ✓ ✗ ✗ ✗ GraphRag system 6 ✗ ✓ ✗ ✗ Deepsieve system 11 ✓ ✓ ✗ ✗ Real Route ✓ ✓ ✓ ✓ Table 1: Comparison of Real Route with representative RAG systems across key system capabilities, including multi-hop reasoning, multi-source retrieval, cross-domain generalization, and built-in knowledge verification. 1 Introduction Retrieval-Augmented Generation (RAG) has become a practical paradigm for question answering and reasoning in settings 10 ; 1 ; 3 where the required information is distributed across multiple knowledge sources, such as general-purpose corpora, domain repositories, and organization-internal collections. In multi-hop question answering, a complex query is commonly decomposed into a sequence of dependent sub-questions, where each step requires retrieving evidence that is both relevant to the current sub-goal and compatible with downstream variable substitution and answer fusion 9 . A central systems question, therefore, concerns retrieval control: how to allocate a limited evidence budget across sources for each decomposed sub-query? A widely adopted design follows an LLM-as-a-Router strategy 36 ; 17 ; 25 ; 11 ; 34 . The LLM predicts (before retrieval) the most appropriate source for the current sub-query before retrieval, and then retrieves exclusively from that source. Representative hard-routing systems ( 11 ) are efficient when source boundaries are clearly separable. However, in realistic deployments, sources often exhibit topical overlap or are artificially partitioned (e.g., English Wikipedia shards). In such cases, an incorrect pre-retrieval routing decision prevents critical evidence from entering the context window, becoming a dominant failure mode 32 . Although reflection-based retry can partially mitigate this, it introduces additional routing rounds and remains constrained by single-source commitment. Core Insight: Real Route eliminates routing hallucinations by shifting from error-prone predictive selection to a robust parallel retrieval and factual verification workflow. To address the limitations of existing LLM-as-a-Router strategies in RAG system, we present Real Route , a lightweight multi-source RAG system that rethinks routing as an evidence-level verification process rather than a pre-retrieval source commitment. Instead of selecting a single source for each sub-query, Real Route performs parallel retrieval across all candidate sources and aggregates top-ranked results into a unified evidence pool. A dynamic selection module then filters this pool under a fixed context budget using retrieval scores, Reciprocal Rank Fusion (RRF), or an LLM-based relevance judge to construct the final context for answer generation. By decoupling evidence discovery from source prediction, RealRoute mitigates catastrophic failures caused by ambiguous or overlapping source boundaries, and maintains bounded context size and computational cost. Table 1 compares Real Route with representative RAG systems along four practical capabilities: multi-hop reasoning, multi-source retrieval, cross-domain generalization, and built-in knowledge verification. Existing approaches typically optimize for only a subset of these requirements. Naive RAG pipelines lack explicit mechanisms for multi-hop decomposition or source coordination, while hard-routing frameworks such as HippoRAG and DeepSieve enable multi-hop reasoning but still rely on single-source commitment, limiting robustness when evidence is distributed across overlapping repositories. Graph-based retrieval systems improve multi-source aggregation but do not explicitly address cross-domain routing ambiguity or factual consistency checks. To best of our knowledge, no prior system simultaneously satisfies all four criteria. Real Route is designed to fill this gap by jointly supporting parallel multi-source retrieval, cross-domain evidence aggregation, and post-retrieval verification under a unified budgeted framework, enabling reliable performance in realistic deployments where knowledge boundaries are uncertain and heterogeneous. We further demonstrate RealRoute on multi-hop and biomedical QA benchmarks, and release an open-source implementation with an interactive interface that visualizes how evidence from multiple sources is aggregated into a unified pool and verified before answer. Figure 1: Overall framework of the multi-source retrieval and evidence check pipeline. Phase I performs context-aware parallel retrieval by decomposing the input query into dependency-aware subqueries, predicting preferred sources, and retrieving top-k candidates from heterogeneous knowledge bases (e.g., local corpus, global corpus, BioASQ, SciQ..) to form a unified candidate pool. Phase II applies dynamic evidence selection via an adaptive token budget (cap) allocated per source, followed by score-based filtering to retain the most informative evidence. Phase III fuses the selected subquery answers within a structured prompt to produce the final output. 2 Related Work Agentic RAG Synergized RAG tightly interleaves retrieval and reasoning in an iterative loop, where intermediate hypotheses steer subsequent retrieval and new evidence updates the reasoning trajectory ( 43 ; 40 ; 57 ; 13 ) . This coupling is often implemented via explicit workflows that structure the search space, including chain-based routines ( 43 ; 40 ; 57 ; 13 ; 21 ) , tree-based variants ( 56 ; 23 ; 14 ) , and graph-based methods for multi-hop aggregation or exploration control ( 12 ; 38 ; 19 ) . Agent orchestration further scales this loop from a single decision-maker to coordinated teams: single-agent systems enable tool-augmented search via prompting patterns ( 52 ; 26 ) , supervised tool-use ( 33 ) , or reinforcement learning 20 ; multi-agent designs distribute roles through decentralized collaboration ( 45 ) , agent-based reasoning ( 47 ) , or centralized controllers that plan and route sub-tasks ( 27 ) . Reasoning-enhanced Retrieval Reasoning-enhanced RAG injects multi-step reasoning structure into retrieval ( 50 ; 16 ) . Collab-RAG emphasizes collaborative multi-perspective rewriting ( 50 ) , while DeepRetrieval uses explicit decomposition signals to target intermediate premises ( 16 ) . Beyond rewriting, retrieval is increasingly planned and structured via multi-step planning ( 44 ) , evidence fusion ( 5 ; 4 ) , relevance filtering ( 58 ; 53 ) , and structural inductive biases for coherent multi-hop retrieval ( 28 ) . Since generation can still drift under noisy contexts, recent work studies context-aware utilization ( 15 ) , evidence selection/reweighting ( 39 ) , and self-guided evidence-consistent reasoning ( 48 ) , alongside traceability via post-hoc revision ( 8 ) and explicit citation/verification ( 7 ) . LLM-as-a-Router Multi-source RAG is increasingly cast as a routing problem. An LLM predicts which source/tool to invoke for each (sub-) query before

Read the complete text