Multi-Step Semantic Reasoning in Generative Retrieval
Steven Dong, Yubao Tang, Maarten de Rijke · 2026
Read Multi-Step Semantic Reasoning in Generative Retrieval by Steven Dong, Yubao Tang, Maarten de Rijke (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-12368.txt.
Introduction
Dong, Tang, and de Rijke add multi-step reasoning to generative retrieval for numeric documents. Such models store a corpus in their parameters and emit document IDs for a query. Complex financial queries expose weak reasoning in these models today. ReasonGR pairs task instructions with stepwise reasoning guidance in its prompts. A focused adaptation module sharpens the parameters tied to that skill. On the FinQA dataset of finance queries, accuracy and consistency both improve in their tests.
Catalog record
| CREATOR | Steven Dong, Yubao Tang, Maarten de Rijke |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16186 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.12368
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Multi-Step Semantic Reasoning in Generative Retrieval Steven Dong, Yubao Tang, Maarten de Rijke Abstract Generative retrieval (GR) models encode a corpus within model parameters and generate relevant document identifiers directly for a given query. While this paradigm shows promise in retrieval tasks, existing GR models struggle with complex queries in numerical contexts, such as those involving semantic reasoning over financial reports, due to limited reasoning capabilities. This limitation leads to suboptimal retrieval accuracy and hinders practical applicability. We propose ReasonGR, a framework designed to enhance multi-step semantic reasoning in numerical contexts within GR. ReasonGR employs a structured prompting strategy combining task-specific instructions with stepwise reasoning guidance to better address complex retrieval queries. Additionally, it integrates a reasoning-focused adaptation module to improve the learning of reasoning-related parameters. Experiments on the FinQA dataset, which contains financial queries over complex documents, demonstrate that ReasonGR improves retrieval accuracy and consistency, indicating its potential for advancing GR models in reasoning-intensive retrieval scenarios. 1 Introduction Generative retrieval (GR) has emerged as a promising paradigm in information retrieval. Unlike traditional dense retrieval methods that rely on explicit index-retrieve pipelines [ 17 , 25 , 10 ] , GR models learn to encode the entire corpus within their parameters and directly generate relevant document identifiers (docids) in response to queries [ 27 ] . This approach reduces memory storage requirements and enables more integrated, end-to-end retrieval. While GR has demonstrated notable results in tasks such as question answering and fact verification [ 3 , 5 , 4 , 18 ] , it still faces challenges in retrieval tasks that require complex inference or aggregation of dispersed evidence, exemplified by financial queries over intricate reports in the FinQA dataset [ 6 ] . We refer to this challenge as multi-step semantic reasoning in numerical contexts, which focuses on interpreting relevant numerical information in complex documents [ 11 ] . Current GR methods struggle to effectively address these tasks, mainly due to two factors: (i) GR models often treat retrieval as a black-box operation, encoding it implicitly in a single forward pass without explicit, interpretable reasoning steps [ 26 ] . (ii) Most models are trained with maximum likelihood estimation (MLE) [ 13 ] on surface-level query-document pairs, which does not explicitly foster multi-step or logical reasoning capabilities [ 19 , 1 ] . To address these challenges, we propose ReasonGR, a framework designed to enhance multi-step semantic reasoning capabilities of GR models in numerical contexts. ReasonGR employs structured prompting that combines task-specific instructions with chain-structured reasoning prompts to guide step-by-step semantic inference. Moreover, we introduce reasoning adapter modules that enable the model to better capture reasoning-related parameters tailored to the task. We evaluate ReasonGR on the FinQA dataset [ 6 ] , which contains financial queries over complex documents. Our experiments show that ReasonGR improves retrieval accuracy over baseline methods. We also report that the introduced adaptation modules yield efficiency benefits in training. 2 Related Work Traditional information retrieval (IR). Traditional IR systems typically follow a “index-retrieve” pipeline [ 21 ] . Early approaches such as BM25 [ 24 ] rely on exact term matching, while dense retrieval methods [ 17 ] use neural networks to capture semantic similarity by encoding queries and documents into dense vectors. Generative retrieval GR learns to index the corpus and then directly generate docids given a query [ 27 , 29 ] . This paradigm internalizes corpus information in model parameters, enabling end-to-end retrieval without an external index. A number of related studies have advanced this line of work [ 23 , 29 ] . While these methods achieve promising retrieval performance, their ability to perform complex reasoning, particularly multi-step semantic reasoning required by tasks such as financial document retrieval, remains limited [ 16 ] . Addressing this reasoning gap is essential for improving the applicability of GR in such domains. 3 Methodology Figure 1: ReasonGR performing multi-step semantic reasoning on a FinQA query. The model extracts key info and locates relevant report sections to generate the docid, formed by the company name and report year. 3.1 Task definition Given a corpus \mathcal{D}=\{d_{1},\dots,d_{N}\} with size N , and a set of queries \mathcal{Q}=\{q_{1},\dots,q_{M}\} with size M , each document d_{i} is associated with a unique docid j_{i} , forming the set \mathcal{J} . The task is to generate the relevant docid j_{i} for a given query q_{i} using a generative model P_{\theta} , which models the conditional probability , P_{\theta}(j_{i}|q_{i})=\prod_{p=0}^{|j_{i}|}P_{\theta}(j_{i}^{p}|q_{i},j_{i}^{<p}), where j_{i}^{p} is the p -th token of the docid sequence j_{i} , and j_{i}^{<p} denotes all previously generated tokens. 3.2 Model architecture Our model adopts a transformer-based encoder-decoder architecture, following the GR paradigm [ 27 ] . The backbone encodes the input query and autoregressively decodes the corresponding docid. To learn reasoning-specific capabilities, we introduce Low-Rank Adaptation (LoRA) modules [ 15 ] into the backbone. LoRA freezes the original model weights \theta and injects trainable low-rank matrices \Delta\theta to adapt the model to the reasoning-intensive retrieval task: \theta^{\prime}=\theta+\Delta\theta,\quad\text{where}\quad\Delta\theta=AB^{\top},\quad A\in\mathbb{R}^{d\times r},B\in\mathbb{R}^{k\times r}, with rank r\ll\min(d,k) . This decomposition allows efficient learning of new parameters dedicated to reasoning, while keeping the number of trainable parameters small. Furthermore, we apply QLoRA [ 9 ] to quantize the frozen backbone weights to 4-bit precision, reducing memory and computational costs with minimal performance loss. LoRA and QLoRA are used jointly in our framework. 3.3 Reasoning-guided training framework Beyond naive optimization with MLE [ 27 ] , our framework improves the model’s reasoning ability through both input construction and training objectives. Reasoning-enhanced prompting To help the model to structure its generation in a way that encourages multi-step logical inference, we design our prompts by combining two sets of templates: task-specific instructions and Chain-of-Thought (CoT) additions, as shown in Figure 1 . During training, one of five task templates is randomly selected and combined with one of five CoT instructions to form diverse and informative prompts. The five task templates are: Answer the query with a document ID. Generate the document ID that answers the question. Based on the question, predict the document ID. Retrieve a document ID that fits the query. Using the question, find the document ID. The CoT additions encourage stepwise reasoning, examples of which include: Use step-by-step reasoning. You need to explain your answer. Think this through carefully. Let’s think step-by-step. Explain your reasoning before answering. These instructions are appended to the task templates to form full prompts. For CoT prompts, the expected output includes intermediate reasoning steps before the final answer. Training tasks To train the retrieval model, we have two tasks: (i) To memorize the corpus, we learn the mapping from documents to docids using the MLE loss [ 27 , 3 ] . Since these documents often contain tables with numerical data, which are crucial for retrieval generation, we also incorporate learning from the tables by splitting them and attaching each entry with its corresponding column and row headers. (ii) To learn relevance, the model takes queries as input and generates correct multi-step reasoning traces along with the corresponding docids as output. To better approximate the distribution of real-world queries, training data is augmented with multiple pseudo-queries generated per document using a query generation model [ 29 ] . Adaptive penalty scaling