FREESON : Retriever-Free Retrieval-Augmented Reasoning via Corpus-Traversing MCTS

Chaeeun Kim, Seungone Kim · 2025

Read FREESON : Retriever-Free Retrieval-Augmented Reasoning via Corpus-Traversing MCTS by Chaeeun Kim, Seungone Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16409.txt.

Introduction

Chaeeun Kim and Seungone Kim proposed FREESON in two thousand twenty-five, a way for reasoning models to fetch facts without a separate retriever. Standard setups bolt a search model onto the reasoner, which adds cost and can lose key facts in translation between the two. Their idea treats search as walking paths through the text corpus toward the answer.

A tree search variant guides the model through the corpus step by step. On five open question sets, FREESON beat four rival multi-step systems by a solid average margin, and topped the strongest rival on two sets. The model acts as both writer and retriever in one.

Catalog record

CREATORChaeeun Kim, Seungone Kim
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-16409

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FREESON : Retriever-Free Retrieval-Augmented Reasoning via Corpus-Traversing MCTS Chaeeun Kim, Seungone Kim Abstract Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in multi-step reasoning and calling search engines at appropriate steps. However, existing retrieval-augmented reasoning approaches rely on separate retrieval models, limiting the LRM’s role in retrieval to deciding when to retrieve and how to query. This separation not only increases hardware and operational costs but also leads to errors in the retrieval process due to the representation bottleneck, a phenomenon where the retriever’s embedding space is not expressive enough to meet the generator’s requirements. To address this, we shift our perspective from sequence-to-sequence matching to locating the answer-containing paths within the corpus, and propose a novel framework called FREESON (Retriever- FREE Retrieval-Augmented Rea SON ing). This framework enables LRMs to retrieve relevant knowledge on their own by acting as both a generator and retriever. To achieve this, we introduce a variant of the MCTS algorithm specialized for the retrieval task, which we call CT-MCTS ( C orpus- T raversing M onte C arlo T ree S earch). In this algorithm, LRMs traverse through the corpus toward answer-containing regions. Our results on five open-domain QA benchmarks, including single-hop and multi-hop questions, show that FREESON achieves an average improvement of 14.4% in EM and F1 over four multi-step reasoning models with a separate retriever, and it also performs comparably to the strongest baseline, surpassing it by 3% on PopQA and 2WikiMultihopQA. 1 Introduction Retrieval-Augmented Reasoning (RAR) is a widely used framework to reduce hallucinations and generate more factual responses by injecting external knowledge into the reasoning chain ( jiang2023activeretrievalaugmentedgeneration ; press2023measuringnarrowingcompositionalitygap ; asai2023selfraglearningretrievegenerate ; li2025searcho1agenticsearchenhancedlarge ; jin2025searchr1trainingllmsreason ; song2025r1searcherincentivizingsearchcapability ; yao2023reactsynergizingreasoningacting ; wang2025rareretrievalaugmentedreasoningmodeling ; schick2023toolformerlanguagemodelsteach ) . In such pipelines, external knowledge is crucial for guiding subsequent reasoning steps. However, conventional search engines—typically based on dual-encoder architectures—often suffer from inherent limitations, failing to retrieve appropriate documents due to an representation bottleneck, where embedding vectors cannot sufficiently represent subtle distinctions between documents or their relevance to the question ( wang2023neuralcorpusindexerdocument ; kim2024exploringpracticalitygenerativeretrieval ; magesh2024hallucinationfreeassessingreliabilityleading ) . For example, given the query “Where was the place of burial of John Tuchet, 6th Baron Audley’s father?”, E5 {}_{\text{base}} wang2024textembeddingsweaklysupervisedcontrastive (a state-of-the-art retriever) assigns higher similarity scores to “John Tuchet, 8th Baron Audley” and “George Tuchet, 9th Baron Audley” (both incorrect) than to “John Tuchet, 6th Baron Audley” (correct), retrieving an incorrect document in the first hop due to a single-character difference , and consequently failing to reach the final answer document, “James Tuchet, 5th Baron Audley”. To address this issue, prior works have proposed better representation learning methods, or scaling up either the model size or the amount of training data to train retrieval models ( izacard2022unsuperviseddenseinformationretrieval ; ram2022learningretrievepassagessupervision ; wang2024textembeddingsweaklysupervisedcontrastive ; wang2024improvingtextembeddingslarge ; lee2024geckoversatiletextembeddings ; shao2025reasonirtrainingretrieversreasoning ) . However, fundamentally resolving the representation bottleneck remains challenging due to the nature of the architecture. In addition, maintaining two separate models introduces additional hardware overhead and operational costs ( zhang2024onegenefficientonepassunified ; reichman-heck-2024-dense ) . In this paper, we revisit the conventional retrieval-augmented paradigm and pose a question: Can a single LRM autonomously identify the knowledge it needs from a corpus while performing reasoning, without relying on a separate retriever? To address this question, we shift our focus from sequence-to-sequence matching to locating the answer-containing paths within the corpus for retrieval tasks and propose a novel framework called FREESON (Retriever- FREE Retrieval-Augmented Rea SON ing). In this framework, a single LRM functions as both generator and retriever, traversing the corpus to obtain information it needs. To implement this, we first introduce CT-MCTS ( C orpus- T raversing Monte Carlo Tree Search), a retrieval-specialized MCTS ( 10.1007/11871842_29 ; Silver2016MasteringTG ; chen2024alphamathzeroprocesssupervision ; liu2024dontthrowawayvalue ; zhang2024restmctsllmselftrainingprocess ) , which defines its search nodes at the token level, allowing each node to represent a prefix of one or more tokens, where the prefix is constrained by a predefined prefix-based index to ensure that the search follows only sequences that exist in the corpus (§ 2.2 ). Implementing such retrieval-oriented MCTS introduces two key challenges: (1) the search operates at extremely fine granularity, with token-level nodes, making it difficult to capture meaningful semantics due to step-wise constraints; and (2) the model must obtain appropriate node value estimates to guide the search toward the desired location in the corpus that contains the answer. To address the first challenge, we increase node granularity while preserving single-token-level constraints, and incorporate stochastic beam search into the expansion process to better leverage the LRM’s capability during path selection. For the second challenge, we train an on-policy value network to estimate answer-containment, using CT-MCTS rollouts in environments aligned with actual inference-time scenarios (§ 4.4 ). We evaluate FREESON on five open-domain QA benchmarks comprising of single-hop and multi-hop questions. On average, FREESON achieves 14.4% improvement in EM and F1 compared to four reasoning models using a separate retriever in their reasoning pipeline. It also performs on par with the strongest baseline, Search-R1, surpassing it by average 3% on PopQA and 2WikiMultihopQA (§ 3.3 ). Our retrieval-specialized CT-MCTS plays a key role in this performance. Flexible node granularity yieds a 27.5% gain over single-token nodes (§ 4.2 ), and multi-node expansion guided by the LRM improves performance by 13% over single-node expansion (§ 4.3 ). FREESON does not require any training and is applicable to an arbitrary LMs when we could access the output logit values. Particularly, it is well suited for domain-specific applications that have unlabeled corpus, as it directly explores and reasons over the content without using any external search engine. Figure 1: Overview of retrieval-augmented reasoning process. Left: FREESON performs both reasoning and retrieval using a single generator model via token-level CT-MCTS . Right: conventional RAR methods compute similarity scores between query and document embeddings using a separate retrieval model. FREESON fully leverages the LRM’s reasoning over document structure, enabling precise access to relevant information from the corpus at inference time without relying on memorization. 2 Methodology FREESON is a model capable of functioning as both a generator and a retriever, built upon Search-R1 jin2025searchr1trainingllmsreason , a retrieval-augmented reasoning model trained with PPO schulman2017proximalpolicyoptimizationalgorithms and GRPO shao2024deepseekmathpushinglimitsmathematical . In this section, we detail how FREESON autonomously identifies and retrieves the knowledge it needs. See Figure 1 for an illustration of this process. 2.1 Dynamically Adapted Search Space When search is required, FREESON first generates a query in the form of (subject: [subject], question: [question]) , where [subject] is the entity or proper noun the question is mainly about, and [question] is the corresponding question. Then, based on the subject information, we adaptively construct a prefix-based index we call CorpusTree that narrows the search space by limiting the range of possible next valid tokens. We then input the question using a few-shot prompt that guides reasoning over document structure. The detailed prompt description is provided in Appendix D.2 . 2.2 CT-MCTS for Autonomous Retrieval Given a question and a subject-specific CorpusTree, FREESON locates the appropriate information using CT-MCTS designed to reinforce paths that are likely to contain the correct answer. Basically, CT-MCTS operates in a token-level search space where the LLM’s probability distribution is dynamically masked by the CorpusTree, allowing only valid sequences found in

Read the complete text