Can we Retrieve Everything All at Once? ARM: An A lignment-Oriented LLM-based R etrieval M ethod

Peter Baile Chen, Yi Zhang, Michael Cafarella, Dan Roth · 2025

Read Can we Retrieve Everything All at Once? ARM: An A lignment-Oriented LLM-based R etrieval M ethod by Peter Baile Chen, Yi Zhang, Michael Cafarella, Dan Roth (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-18539.txt.

Introduction

Peter Baile Chen and co-authors present a retrieval method that aims to fetch all the data a hard question needs at once. Their paper from 2025 is called ARM. Real questions can span many sources, and language models that split them into steps do not know how the data is stored or organized.

That blind split leads to weak retrieval, the authors argue. Step by step agent methods fare no better, since each new query rests on the last haul rather than on the data layout. ARM instead aligns the question with how the data collection is organized, probing links between data objects past plain wording match. On two data sets it beat standard and agent retrieval by clear margins.

Catalog record

CREATORPeter Baile Chen, Yi Zhang, Michael Cafarella, Dan Roth
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-18539

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Can we Retrieve Everything All at Once? ARM: An A lignment-Oriented LLM-based R etrieval M ethod Peter Baile Chen, Yi Zhang, Michael Cafarella, Dan Roth Abstract Real-world open-domain questions can be complicated, particularly when answering them involves information from multiple information sources. LLMs have demonstrated impressive performance in decomposing complex tasks into simpler steps, and previous work has used it for better retrieval in support of complex questions. However, LLM’s decomposition of questions is unaware of what data is available and how data is organized, often leading to a sub-optimal retrieval performance. Recent effort in agentic RAG proposes to perform retrieval in an iterative fashion, where a followup query is derived as an action based on previous rounds of retrieval. While this provides one way of interacting with the data collection, agentic RAG’s exploration of data is inefficient because successive queries depend on previous results rather than being guided by the organization of available data in the collection. To address this problem, we propose an LLM-based retrieval method — ARM, that aims to better align the question with the organization of the data collection by exploring relationships among data objects beyond matching the utterance of the query, thus leading to a retrieve-all-at-once solution for complex queries. We evaluated ARM on two datasets, Bird and OTT-QA. On Bird, it outperforms standard RAG with query decomposition by up to 5.2 pt in execution accuracy and agentic RAG (ReAct) by up to 15.9 pt. On OTT-QA, it achieves up to 5.5 pt and 19.3 pt higher F1 match scores compared to these approaches. 1 Introduction Answering real-world questions can be complicated, especially when required information is distributed across heterogeneous information sources, such as text corpus, databases, and even image collections. Consider an example question "What is the highest eligible free rate for K-12 students in the schools in the most populous county in California?" and the data collection shown in Figure 1 . Answering this question requires one passage (A) from the text corpus and three joinable tables (B, C, D) from the database. To find all of these passages and tables, it requires exploring the available text and tables in the data collection, reasoning about their relationships (e.g., joinable tables, entity connection), and determining the best organization of these objects that can fully answer the question. Figure 1: A summary of our approach ARM, and a comparison with retrieval in standard RAG, which leverages LLMs for query decomposition, and agentic RAG, which employs LLM-based agents to iteratively generate queries. With recent advances in LLMs, they have been used to decompose complex questions to boost the performance of RAG systems Khot et al. (2022) ; Zhou et al. (2022) ; Jhamtani et al. (2023) . However, all these decomposition methods are unaware of what data is available, and how they are organized. Therefore, these approaches are highly likely to overlook important data objects relevant to the question, especially when their information is not explicitly mentioned. As depicted in Figure 1 , this method can miss all bridging tables. A possible way to improve LLM’s decomposition is to make it interact with the data collection, which has been explored in the context of agentic RAG Yao et al. (2022) ; Trivedi et al. (2022) ; Press et al. (2022) ; Asai et al. (2023) ; Zhang et al. (2024) . Typically, an LLM-based agent, which has demonstrated impressive performance in reasoning and acting Kojima et al. (2022) ; Wei et al. (2022) ; Shinn et al. (2024) ; Yao et al. (2024) , iteratively reasons about actions (i.e., search queries). Search queries are then issued, and the retrieved passages will be fed back to the agent to reason about the next action (search query) until it determines that the question has been fully answered. Although we have seen promising results when adapting such agent-based iterative solutions, they have shortcomings as follows: First of all, although the iterative approach enables interaction with the data collection, each action is guided by the agent’s decision about what is still missing to fully answer the question based on the information retrieved in previous rounds, rather than what data objects are available to answer the question. Therefore, it can lead to an inefficient exploration with unnecessary LLM calls, increasing inference time and cost. Secondly, an iterative approach based on past experience can not enable a joint optimization , where earlier and later steps are planned together. As shown in Yao et al. (2022) , an agentic solution faces a critical issue known as reasoning derailment , a phenomenon in which agents struggle to recover from an initial erroneous action, resulting in a further round of incorrect reasoning. In this work, we argue that enabling an efficient and comprehensive retrieval for complex questions requires aligning the question and its decomposition to existing data objects and their organization. This means that during the retrieval process, in addition to performing semantic matching between the question and data objects, we need to simultaneously reason about how candidate data objects can be connected and whether they involve bridging entities or tables. Therefore, we propose an LLM-based a lignment-oriented r etrieval m ethod, ARM, for complex open-domain questions. Through reasoning about a better alignment to the data collection and its organization, we want to have a more effective and efficient retrieval solution, where we can achieve retrieve-everything-all-at-once . Inspired by Chen et al. (2024c) , which can be considered as a retrieve-all-tables-at-once solution that leverages a solver to reason about table relationships, we integrate its reasoning module as a way to search for potential alignment to the data collection for an LLM. Intuitively, we instruct an LLM to reason about required data objects, augment it with a search process powered by a reasoning solver to figure out potential alignments to the data collection, and verify them through self-evaluation. In the rest of the paper, we provide an overview of our methodology in Section 2 , and elaborate each component of our solution in Section 3 . We evaluated our solution on two datasets, i.e., Bird and OTT-QA in Section 4 , and conclude in Section 5 . 2 Overview This section provides an overview of our alignment-oriented LLM-based retrieval solution. Each component will then be elaborated in Section 3 . Intuitively, our solution adopts the idea of retrieving while generating Jain et al. (2024) , so that we can take the most advantage of LLM’s reasoning capability, and the decoding process enables an optimization of each component jointly through beam search. Rather than simply interleaving “retrieval” by constrained decoding based on evidence from the text corpus, and “thoughts” by unconstrained decoding Jain et al. (2024) , we propose to retrieve everything jointly through one decoding process guided by information from data objects available in the data collection, a solver’s reasoning, and LLM’s self-verification. Specifically, we consider the retrieval problem as a generative problem that an LLM needs to output a “reasoning process” to find all necessary data objects from the data collection to answer the given question. There are two main challenges for an off-the-shelf LLM to output such a reasoning process that can align with the data organization from the collection. First, while an

Read the complete text