Analyzing Information Sharing and Coordination in Multi-Agent Planning

Tianyue Ou, Saujas Vaduguru, Daniel Fried · 2025

Read Analyzing Information Sharing and Coordination in Multi-Agent Planning by Tianyue Ou, Saujas Vaduguru, Daniel Fried (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-12981.txt.

Introduction

Tianyue Ou and co-authors study how teams of language model agents share facts and coordinate on long planning tasks. Their test bed is travel planning, a task with many linked constraints. Two aids are weighed: a shared notebook, and an orchestrator agent that steers the talk.

In their 2025 tests, the notebook cut errors from hallucinated details by 18 percent. The orchestrator pushed the team to focus and cut errors by up to 13.5 percent more inside those focus zones. Together, the aids reached a 25 percent pass rate on the TravelPlanner benchmark. That is a 17.5 point gain over a single agent at 7.5 percent. Structure, they argue, is the lever.

Catalog record

CREATORTianyue Ou, Saujas Vaduguru, Daniel Fried
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE24 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-12981

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Analyzing Information Sharing and Coordination in Multi-Agent Planning Tianyue Ou, Saujas Vaduguru, Daniel Fried Abstract Multi-agent systems (MASs) have pushed the boundaries of large language model (LLM) agents in domains such as web research and software engineering. However, long-horizon, multi-constraint planning tasks involve conditioning on detailed information and satisfying complex interdependent constraints, which can pose a challenge for these systems. In this study, we construct an LLM-based MAS for a travel planning task which is representative of these challenges. We evaluate the impact of a notebook to facilitate information sharing, and evaluate an orchestrator agent to improve coordination in free form conversation between agents. We find that the notebook reduces errors due to hallucinated details by 18%, while an orchestrator directs the MAS to focus on and further reduce errors by up to 13.5% within focused sub-areas. Combining both mechanisms achieves a 25% final pass rate on the TravelPlanner benchmark, a 17.5% absolute improvement over the single-agent baseline’s 7.5% pass rate. These results highlight the potential of structured information sharing and reflective orchestration as key components in MASs for long horizon planning with LLMs. 1 Introduction Large language models have enabled the design of systems that undertake complex tasks in rich, realistic environments like web browsing ( Zhou et al. 2023 ) , software engineering ( Yang et al. 2024 ) , flight booking ( Yao et al. 2024 ) , and general office computer tasks ( Xu et al. 2024 ) . These systems model a single agent reasoning and taking actions in an environment using tools to solve tasks specified in natural language. While these advances have expanded the abilities of AI systems greatly, solving long-horizon planning tasks, with many interdependent actions, requires reasoning about a number of interdependent constraints, drawing on information from heterogeneous sources that are accessed with different tools, and synthesizing information with accuracy and consistency. Since each action adds information to the agent’s context window, and each source of information requires different tools to access, a single agent solving this task needs to reason over increasingly long contexts and choose correctly from a large number of tools. This risks errors and hallucinations, which cascade in long-horizon tasks to result in a failure to fully solve the task. Multi-agent systems (MAS) offer an avenue to tackle these issues. When different agents work together, but each focuses on a part of the task, each agent can reason about less information ( Hong et al. 2024 ; Dong et al. 2024 ; Xia et al. 2024 ) , critique more effectively ( Du et al. 2023 ; Tang et al. 2024 ) , explore more solutions ( Pan et al. 2025 ) , and use tools more proficiently. Multi-agent systems have been applied to complex tasks such as web research ( Anthropic 2025 ) , stock trading ( Xiao et al. 2025 ) , software engineering ( Qian et al. 2024 ; Li et al. 2023 ; Xia et al. 2024 ) , and medical decision support ( Ke et al. 2024 ; Tang et al. 2024 ; Kim et al. 2024 ) to overcome the limitations of single agent systems. Multiple agents working together require a way to share information between agents, and a way to coordinate how agents act to navigate complex dependencies between different parts of the task. In this paper, we propose approaches to these problems, and systematically evaluate the impact of our solutions in the TravelPlanner benchmark ( Xie et al. 2024 ) . We use a notebook which multiple agents write to and read from to serve as an information sharing mechanism. We also study the benefits of an orchestrator agent that reasons about the current state of task completion and chooses how to sequence actions by different agents. We show that this dynamic form of coordination is more effective than a fixed workflow ( Xiao et al. 2025 ) . We find that adding a notebook to share information improves performance by 3.75% in final pass rate over using the dialog alone, with GPT-4o. In a system equipped with a notebook, we find that an orchestrator that reasons about the choice of the next agent improves performance by 1.25% over a fixed workflow to sequence agents. Overall, these proposed approaches combine to improve performance of a GPT-4o-based system by 5% over the single-agent baseline, and the same improvement over a multi-agent baseline that does not incorporate these approaches. The final system (that includes both the notebook and an orchestrator agent) based on Claude 4 Sonnet improves by 17.5% over single-agent baseline, further boosting performance. 2 Proposed Multi-Agent System We propose a multi-agent, LLM-based system for long-horizon planning in tasks that require resolving constraints between specialized sources of information (e.g., travel planning requires finding available hotels, coordinating transportation between them, and choosing restaurants that meet a user’s preferences). An overview of our system is shown in Figure 1 . The task is given by a natural language goal G . The system uses a set of LLM-based agents, \mathcal{A}=\{D,R,P\}\cup\{E_{1},\ldots,E_{K}\} where: • Experts E_{k} : Specialist agents which use tools to retrieve information, such as transportation experts and restaurant experts. • Orchestrator D : Decides which agent acts next. • Plan summarizer P_{S} : Surfaces the notebook and task query for plan compilation (not backed by an LLM). • Plan compiler P_{C} : Synthesizes a plan based on the task query and the facts from the notebook. • Plan critic P_{R} : Iteratively refines the plan in conjunction with P_{C} . We define the public conversational history to be the sequence of \ell messages produced by the agents, c=[m_{1},\ldots,m_{\ell}] . We additionally maintain a Notebook N for structured evidence (see Section 4 ), so that the full world state is w=(c,N,G) . Each agent A_{i} has an action space Y_{i} , a policy \pi_{i}(y\mid o_{i}(w)) that uses an observation function o_{i} to select parts of the state visible to each agent. The notebook is only visible to a subset of agents. In our implementation, we use the same underlying LLM (e.g. GPT-4o) for all expert agents, but each agent has a unique prompt for its role. The action spaces of each agent are: • The orchestrator D selects the next agent A_{j} using only the goal G and the public conversation c . • The experts E_{k} call tools and then write the results to the notebook N . The experts also add to the conversation c . Tool results are visible only to E_{k} ; structured facts must be recorded in N . • The plan summarizer P_{S} prepares a planning brief from G and N and then prompts the plan compiler P_{C} . • The plan compiler P_{C} produces the final plan as a public message in c , which is iteratively refined by continuing the conversation c with the plan critic P_{R} . Figure 1: Schematic diagram of our multi-agent system. We implement our multi-agent system using the AutoGen framework ( Wu et al. 2023 ) . The visibility rules are: 1. All agents observe G and the public conversation c . 2. Experts E_{k} see the returns of their own tool calls. These returns are not appended to

Read the complete text