Tree of Agents: Improving Long-Context Capabilities of Large Language Models through Multi-Perspective Reasoning

Song Yu, Xiaofei Xu, Ke Deng, Li Li, Lin Tian · 2025

Read Tree of Agents: Improving Long-Context Capabilities of Large Language Models through Multi-Perspective Reasoning by Song Yu, Xiaofei Xu, Ke Deng, Li Li, Lin Tian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06436.txt.

Introduction

Long documents defeat language models in the middle, and Song Yu and four co-authors answer with a tree of cooperating agents in a 2025 paper. Information buried mid-context gets underused, trimming risks losing key content, and longer windows disperse attention. Tree of Agents instead splits the input into chunks that independent agents read separately.

Each agent forms a local view, then agents exchange information along tree-structured paths and try different reasoning orders, which cuts position bias and hallucination. Prefix-hash caching and adaptive pruning hold the cost near single-model levels. Powered by a compact eight-billion-parameter model, the system beats multiple baselines and rivals much larger commercial models on long-context tasks, by the authors' experiments.

Catalog record

CREATORSong Yu, Xiaofei Xu, Ke Deng, Li Li, Lin Tian
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-06436

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tree of Agents: Improving Long-Context Capabilities of Large Language Models through Multi-Perspective Reasoning Song Yu, Xiaofei Xu, Ke Deng, Li Li, Lin Tian Abstract Large language models (LLMs) face persistent challenges when handling long-context tasks, most notably the “lost in the middle” issue, where information located in the middle of a long input tends to be underutilized. Some existing methods that reduce input have the risk of discarding key information, while others that extend context windows often lead to attention dispersion. To address these limitations, we propose Tree of Agents (TOA) , a multi-agent reasoning framework that segments the input into chunks processed by independent agents. Each agent generates its local cognition, then agents dynamically exchange information for collaborative reasoning along tree-structured paths. TOA enables agents to probe different reasoning orders for multi-perspective understanding, effectively mitigating position bias and reducing hallucinations. To improve processing efficiency, we incorporate prefix-hash caching and adaptive pruning strategies, achieving significant performance improvements with comparable API overhead. Experiments show that TOA, powered by compact LLaMA3.1-8B, significantly outperforms multiple baselines and demonstrates comparable performance to the latest and much larger commercial models, such as Gemini1.5-pro, on various long-context tasks. Code is available at https://github.com/Aireduce952/Tree-of-Agents . † † footnotetext: † Corresponding authors. Figure 1: The core difference between TOA and other multi-agent reasoning methods. (a) COA processes chunks sequentially, with a final manager agent making the decision. (b) LONGAGENT uses a leader agent to coordinate multi-turn discussions with others. (c) TOA probes multi-paths in a tree structure to prompt multi-perspective reasoning. Category Example Work PnP No Train Focus Cost Inter. Model Modification Position Encoding Zhu et al. 2024 ✘ ✘ ✘ High Low Attention Mechanism Chen et al. 2024b ✘ ✘ ✘ High Low Training Strategies Zhang et al. 2024a ✘ ✘ ✘ High Low Input Reduction RAG Zhao et al. 2024b ✘ ✘ ✔ Medium Medium Prompt Compression Jiang et al. 2024 ✘ ✔ ✔ Medium Low Multi-Agent Reasoning LONGAGENT Xiao et al. 2024 ✘ ✘ ✔ High High COA Zhang et al. 2024b ✔ ✔ ✔ High High Tree of Agents (Ours) ✔ ✔ ✔ High High Table 1: Comparison of various existing methods in long context tasks. Plug-and-play(PnP): whether the method can be applied directly without additional adaptation. Focus: ability to identify and prioritize critical information in long contexts. Inter.: interpretability of each method. 1 Introduction The capabilities of Large Language Models (LLMs) have seen rapid improvements in recent years. The advent of models such as OpenAI-o3 El-Kishky et al. 2025 and DeepSeek-R1 Guo et al. 2025 marks further progression in model inference capacity. Although LLMs perform well in many scenarios, they still fall short in tasks involving long contexts, such as financial report comprehension Reddy et al. 2024 , novel quizzes Wang et al. 2025 ; Xu et al. 2024b and legal contract analysis Shen et al. 2022 ; Shu et al. 2024 . There are three primary challenges in long-context tasks: (1) Position biases processing further impair performance, often referred to as the "lost in the middle" issue Liu et al. 2024b . (2) As input length increases, the presence of extraneous or redundant information grows, potentially diluting the model’s focus and degrading output quality Shi et al. 2023 . (3) The computational demands for both training and inference escalate dramatically with longer sequences Gao et al. 2024 . To address these issues, existing approaches can be categorized into three main directions, as shown in Table 1 . The first direction focuses on optimizing the model itself, either by enhancing long dependency capture through improvements in attention mechanisms Chen et al. 2024b ; Yen et al. 2024 , training strategies Zhang et al. 2024a ; Chen et al. 2024a , or position encoding Zhu et al. 2024 ; Jin et al. 2024 . The second direction aids the model in solving long contexts and reducing memory usage by enhancing key information density or reducing input length through retrieval augmented generation Zhao et al. 2024b ; Xu et al. 2024a or prompt compression Jiang et al. 2024 . The third direction addresses long text processing by employing multi-agent reasoning Zhang et al. 2024b ; Zhao et al. 2024a to break down the input into manageable chunks, which are subsequently integrated to arrive at the final result. Nonetheless, processing long contexts efficiently remains a challenging problem Hsieh et al. 2024 . Extending the context window for a model may hurt its performance on short texts Ding et al. 2024 . Retrieval-based method relies on the quality of external retrieval tools and may introduce noise Li et al. 2024 . Prompt compression may lose key information, especially when information is unevenly distributed in long texts. Such methods are more suitable for QA or information extraction tasks and have limited performance for tasks that require global understanding Jiang et al. 2023 . Multi-agent reasoning requires a well-designed agent communication mechanism to get the right results. As illustrated in Figure 1 , COA lets agents process text chunks sequentially and use a manager to integrate the results Zhang et al. 2024b , and LONGAGENT uses a leader agent to coordinate multi-turn discussions with others Xiao et al. 2024 . However, these approaches generally do not support multi-perspective understanding of the document. This study proposes a novel multi-perspective reasoning framework — Tree of Agents (TOA), inspired by cognitive science theories and empirical findings on problem-solving. As shown in Figure 1 , TOA splits a long text into chunks and employs multiple agents to explore different orders of chunks along multi-paths of a tree structure. This design draws from Kahneman’s dual-process theory Kahneman 2011 , which advocates for a multi-perspective approach, and Newell and Simon’s theory of problem-solving as an iterative exploration process NEWELL 1959 . Unlike sequential processing, which often struggles in tasks requiring strategic exploration or early decisions, our method disrupts the natural reading order to enable agents to re-assess and refine their understanding through cross-validation. Agents in TOA collaborate by exchanging local cognition and applying the mechanism for consensus formation. The multi-perspective reasoning significantly improves performance on various tasks but increases computational cost. To mitigate this issue, we introduce two efficiency-enhancing strategies: 1) A prefix-hash-based cache mechanism that reduces redundant cognition generations; 2) An adaptive pruning strategy that terminates useless reasoning paths early. We conduct extensive experiments with eight baselines on two long-context reasoning datasets and one benchmark, demonstrating the effectiveness of our approach. The main contributions are: 1) We propose TOA, the first tree-structured multi-agent reasoning framework that addresses long-context modeling via multi-perspective reasoning. 2) We introduce two optimization strategies that reduce the computational cost while maintaining the advantage of the proposed multi-perspective reasoning. 3) This study empirically validates that TOA consistently improves long-context reasoning for various tasks. More interestingly, our experiments show that the different reading order of a long text can lead to different perceptions, and thus different answers to queries. 2 Related Work This section summarizes key directions in long-context modeling for LLMs. 2.1 Specialized Long-Context Models Training specialized models capable of handling long-contexts is a straightforward approach to improving capabilities. Notable examples include Claude3 Anthropic 2024 and Gemini-1.5pro Team et al. 2024 , which support 200K and 2M context windows, respectively. While

Read the complete text