RaDeR: Reasoning-aware Dense Retrieval Models

Debrup Das, Sam O’Nuallain, Razieh Rahimi · 2025

Read RaDeR: Reasoning-aware Dense Retrieval Models by Debrup Das, Sam O’Nuallain, Razieh Rahimi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18405.txt.

Introduction

Debrup Das, Sam ONuallain, and Razieh Rahimi train search models that understand reasoning steps, not keywords. Language models solve hard problems step by step and often need to fetch supporting material mid-solution, yet standard retrievers are built for plain questions. Writing in twenty twenty-five, the team creates training data from the solution traces of a language model working on math problems. The model own reflections judge which passages truly help, producing varied examples plus hard near-miss negatives for contrast.

Retrievers trained this way, though taught on math, transferred well to broader reasoning benchmarks covering science and coding. They beat strong baselines overall and by wide margins on math and code tasks. The team reports the first dense retriever to beat a classic keyword method when the query itself is a chain of thought. It uses only a small slice of the training data a rival effort needed.

Catalog record

CREATORDebrup Das, Sam O’Nuallain, Razieh Rahimi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18405

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RaDeR: Reasoning-aware Dense Retrieval Models Debrup Das, Sam O’Nuallain, Razieh Rahimi Abstract We propose RaDeR, a set of reasoning-based dense retrieval models trained with data derived from mathematical problem solving using large language models (LLMs). Our method leverages retrieval-augmented reasoning trajectories of an LLM and self-reflective relevance evaluation, enabling the creation of both diverse and hard-negative samples for reasoning-intensive relevance. RaDeR retrievers, trained for mathematical reasoning, effectively generalize to diverse reasoning tasks in the Bright and RAR-b benchmarks, consistently outperforming strong baselines in overall performance. Notably, RaDeR achieves significantly higher performance than baselines on the Math and Coding splits. In addition, RaDeR presents the first dense retriever that outperforms BM25 when queries are Chain-of-Thought reasoning steps, underscoring the critical role of reasoning-based retrieval to augment reasoning language models. Furthermore, RaDeR achieves comparable or superior performance while using only 2.5% of the training data used by the concurrent work ReasonIR , highlighting the quality of our synthesized training data. Our code, data, and retrieval models are publicly available. 1 1 1 https://anonymous.4open.science/r/project-D27D/ 1 Introduction Large language models (LLMs) have demonstrated impressive reasoning capabilities on a wide range of tasks. Yet, they often benefit from retrieval augmentation to enhance accuracy, attributability Asai et al. (2024) , and the interpretability Nakano et al. (2022) of their outputs. Retrieval models for LLM augmentation generally perform reasonably well at lexical and semantic term matching, however they face challenges when reasoning is needed for relevance prediction Su et al. (2024) . Figure 1: An example based on sample ‘TheoremQA_jianyuxu/pigeonhole3’ of Bright , where term matching retrievers face challenges in retrieving the relevant theorem w.r.t. both questions and CoT reasoning. Recent works have tried to address the reasoning limitation of existing models for relevance prediction. Two main approaches have emerged: (1) interleaved reasoning and retrieval Hu et al. (2025) ; Jin et al. (2025) ; Song et al. (2025) , and (2) reasoning-based re-ranking models Weller et al. (2025) ; Samarinas and Zamani (2025) . While the first group is more effective than in-context retrieval augmentation, they are limited to the reasoning steps of LLMs for retrieval. As the example in Figure 1 shows, the reasoning steps of LLMs may not align with those needed for retrieval. Solving the question in this example requires retrieving the pigeonhole principle , where there are no matching terms between the question and the principle. The reasoning steps by GPT-4 also do not simplify the retrieval of the pigeonhole principle, since they outline the steps of applying the pigeonhole principle to solve the question. On the other hand, reranking models are inherently limited by the candidate set produced by the first-stage retriever. To the best of our knowledge, there are no first-stage reasoning-based retrieval models. Developing reasoning-based retrievers poses multiple challenges . The primary challenge is automatic generation of diverse and high-quality training data. Specifically, training data should include queries of diverse formats and lengths as well as samples with varying degrees of reasoning complexity . Beyond this, training retrieval models using representation learning Dai et al. (2023) presents an additional challenge due to the need for generating hard-negative reasoning samples. We propose RaDeR, a set of first-stage reasoning-based retrieval models trained with synthesized data from mathematical reasoning. Specifically, we use an LLM for mathematical problem solving with a retrieval-augmented search-based reasoning approach, where the LLM can retrieve and apply theorems needed for solving intermediate subproblems. To generate training data, we then sample reasoning trajectories with retrieval nodes based on the assumption that information retrieved during intermediate steps of the LLM’s search process is likely to be relevant to the original question. This approach, illustrated in Figure 2 , naturally yields a diverse set of queries, varying in length and complexity. In addition, verifying LLM-generated answers to mathematical questions provides a proxy for evaluating the relevance of retrieved information. To further ensure the quality of generated training data, the relevance evaluation is enhanced by self-reflection . These evaluations help with generation of high-quality data. Additionally, any retrieved theorem evaluated as non-relevant is considered as a hard-negative reasoning sample. Figure 2: An overview of the RaDeR data generation pipeline. The OST action stands for one step thought generation, and CRS stands for complete remaining solution steps action. . We perform a comprehensive evaluation of RaDeR models, including evaluation of retrieval performance on reasoning-intensive benchmarks, traditional benchmarks mainly requiring term matching, as well as evaluation of target QA performance using retrieval augmentation. Experimental results on Bright Su et al. (2024) show that RaDeR outperforms strong baselines by at least 2 points in different settings. These findings demonstrate that training retrieval models for mathematical reasoning effectively generalizes to other types of reasoning for information retrieval. In addition to improvements in overall performance, RaDeR demonstrates particularly strong performance in Math and Coding splits of Bright . We observe nDCG@10 relative improvements of 37-40% in the theorem-Q split, and 8-26% over the Leet coding split. RaDeR presents the first dense retriever that outperforms BM25 in the zero-shot setting of using reasoning steps as retrieval queries. This achievement provides strong evidence for the necessity of reasoning-based retrievers even when retrievers augment reasoning language models. On the MMTEB Enevoldsen et al. (2025) reasoning subset, RAR-b Xiao et al. (2024) , RaDeR significantly outperforms all sparse and open-source models, performing on par with large proprietary models such as OpenAI-3-large. In a concurrent work, Shao et al. (2025) also train reasoning-based first-stage retrieval models. RaDeR achieves a 1.1 point increase in nDCG@10 performance, corresponding to a 4.5 % relative gain. Performance of RaDeR is particularly significant given that it is trained with 43,120 samples, about 2.5 % of samples used for ReasonIR (1,729,368), demonstrating the effectiveness of our synthesized data . 2 Related Works Interleaving reasoning with retrieval. Previous works have explored interleaving chain-of-thought (CoT) reasoning with retrieval using off-the-shelf retrievers Trivedi et al. (2023) ; Shao et al. (2023) ; Yao et al. (2023b) ; Schick et al. (2023) (see Appendix A.2 for details). In this paradigm, a retrieval action at a given step leverages a subset of the previously generated CoT reasoning steps as the query for retrieval. However, these methods are limited by their reliance on off-the-shelf retrievers. Recent works like Search-R1 Jin et al. (2025) and R1-Searcher Song et al. (2025) use reinforcement learning to optimize reasoning-based query generation while keeping the retrieval model fixed. In complex retrieval tasks, direct lexical or semantic overlap between the initial question and the relevant document is often limited. By incorporating intermediate CoT reasoning during retrieval, the system can better bridge this gap. Recent research explores search-based methods for exploring the space of CoT reasoning paths, such as random sampling Wang et al. (2023b) and Monte Carlo Tree Search (MCTS) Qi et al. (2025) ; Yao et al. (2023a) ; Guan et al. (2025) ; Hao et al. (2023) . Retrieval-augmented generation (RAG) has also been integrated into MCTS-based frameworks by introducing retrieval-related actions such as query generation and document retrieval Tran et al. (2024) ; Hu et al. (2025) . Reasoning-based re-rankers. Recent methods such as Rank-1 Weller et al. (2025) and InteRank Samarinas and Zamani (2025) train re-ranking models using knowledge distillation from reasoning LLMs and reinforcement learning

Read the complete text