LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems
RM Aratchige, Dr. WMKS Ilmini · 2025
Read LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems by RM Aratchige, Dr. WMKS Ilmini (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01963.txt.
Introduction
Aratchige and Ilmini survey the technology that lets teams of language-model agents work as one system. Single agents falter on jobs needing varied skills and quick coordination. The review sorts the field by four pillars: how agents are structured, how they remember, how they plan, and which frameworks builders actually use.
Designs like Mixture of Agents assign roles across many models, while the ReAct pattern interleaves reasoning with action. The authors catalog what works and what strains: systems slow down in real time, coordination frays as teams grow, and memory stays shallow. They close with practical advice for scaling these crews without losing coherence, aiming the roadmap at researchers and engineers alike.
Catalog record
| CREATOR | RM Aratchige, Dr. WMKS Ilmini |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-01963
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems RM Aratchige, Dr. WMKS Ilmini Abstract This survey investigates foundational technologies essential for developing effective Large Language Model (LLM)-based multi-agent systems. Aiming to answer how best to optimize these systems for collaborative, dynamic environments, we focus on four critical areas: Architecture, Memory, Planning, and Technologies/Frameworks. By analyzing recent advancements and their limitations—such as scalability, real-time response challenges, and agent coordination constraints—we provide a detailed view of the technological landscape. Frameworks like the Mixture of Agents architecture and the ReAct planning model exemplify current innovations, showcasing improvements in role assignment and decision-making. This review synthesizes key strengths and persistent challenges, offering practical recommendations to enhance system scalability, agent collaboration, and adaptability. Our findings provide a roadmap for future research, supporting the creation of robust, efficient multi-agent systems that advance both individual agent performance and collective system resilience. I Introduction “Individually, we are one drop. Together, we are an ocean.” - Ryunosuke Satoro The advent of Large Language Models (LLMs) has transformed artificial intelligence, with the introduction of the Transformer architecture in the landmark paper “Attention is All You Need” [1] marking a key turning point. The Transformer replaced traditional sequence models like recurrent neural networks with an attention-based mechanism, boosting machine translation performance and reducing training time. Since then, LLMs have evolved further, particularly with models like GPT, which demonstrate unprecedented performance in natural language processing tasks [2]. LLMs now handle tasks ranging from text generation to summarization, enabling software that can understand and reason with natural language. This versatility drives the current research: exploring how LLMs can build complex multi-agent systems that collaborate and specialize in tasks, especially in environments demanding more than a single model’s capabilities. Despite their successes, LLM-based applications have limitations. Hallucination remains a major issue, as models can produce inaccurate information without external validation, as noted by Adewumi et al. [3]. This limits reliability where precision is essential. LLMs also struggle with complex or abstract concepts, a challenge discussed by Cherkassky et al. [4], who showed that even advanced models like GPT-4 often fall short in imitating human reasoning. Multi-agent systems can mitigate these issues, allowing distinct agents to collaborate on complex tasks, as explored by Han et al. [5]. Such collaboration improves decision-making, especially for tasks requiring deeper reasoning or specialized skills. Research on LLM-based multi-agent systems exists [6], but there remains a gap in identifying the best technological approaches for building these systems. This survey aims to address this gap by answering two research questions: What state-of-the-art technologies and approaches are available for LLM multi-agent systems? And, which of these are most effective in practice? This survey seeks to identify optimal technologies and methodologies, helping researchers and practitioners navigate the evolving landscape of tools for building advanced multi-agent systems. II Literature Review The literature on large language models (LLMs) within multi-agent systems is still emerging, with significant research gaps, particularly in understanding and improving multi-agent paradigms. This review critically examines the current research in multi-agent LLM systems, focusing on architecture, planning, memory, and frameworks. These technological aspects shape the potential and limitations of LLM multi-agent systems, and further investigation can expand their applications and efficiency. This literature review is structured as given below: • Architecture : Exploring various frameworks like Conquer-and-Merge Discussion (CMD), Chain-of-Agents (CoA), Agent Forest, and Mixture-of-Agents (MoA). • Planning : Discussing frameworks such as AdaPlanner, ChatCoT, KnowAgent, RAP, Tree of Thoughts (ToT), and ReAct. • Memory : Covering the role of memory in LLM systems, with insights on Vector Databases, Retrieval Augmented Generation, ChatDB, MemoryBank, RET-LLM, and Self-Controlled Memory. • Technologies / Frameworks : Examining the technologies and frameworks that facilitate collaboration and task execution, including AutoGen, CAMEL, CrewAI, MetaGPT, and LangGraph. A summary of the findings of each section is provided in Tables I-IV. II-A Architecture Research on multi-agent architectures in the LLM space is relatively limited. A significant gap exists in developing frameworks that effectively orchestrate multiple agents to collaborate and solve complex reasoning tasks. While some research has started addressing these challenges, there remains a need for more comprehensive solutions. Below, we analyze some of the key papers exploring architectural designs in multi-agent LLM systems. II-A 1 Conquer-and-Merge Discussion (CMD) In their paper, Wang et al. [7] present the Conquer-and-Merge Discussion (CMD) framework. This architecture leverages multiple LLM-powered agents that engage in open discussions to address reasoning tasks. Inspired by Minsky’s Society of Mind (1988), the CMD framework simulates human-like debates, where each agent contributes different perspectives to improve overall reasoning capabilities. The CMD framework is structured to allow a group of agents to discuss a question, with each agent generating a viewpoint and explanation in several rounds of interaction. The discussion is guided by a shared history of responses, with agents building on each other’s inputs. This design outperforms single-agent methods like the Chain-of-Thought (CoT) approach, as demonstrated in their experiments. However, several limitations persist: the framework simplifies LLM sessions as agents, missing the opportunity to integrate more sophisticated reasoning techniques such as the Tree-of-Thought method or external knowledge bases; CMD has only been tested on reasoning tasks, leaving its applicability to broader domains such as strategic planning or real-time decision-making unexplored; and their experiments were limited to a few LLMs (Bard, Gemini Pro, and ChatGPT-3.5), so further analysis using other models is needed to assess generalizability. II-A 2 Chain-of-Agents (CoA) Zhang et al. [8], in their paper ”Chain of Agents: Large Language Models Collaborating on Long-Context Tasks,” propose the Chain-of-Agents (CoA) framework. This architecture is designed for handling long-context tasks that surpass the token limits of individual LLMs. It consists of worker agents that sequentially process portions of the input and pass their results to a manager agent, which aggregates the final output. The CoA architecture’s key innovation is the interleaved read-process method, allowing agents to process chunks of input before receiving the full context. This approach reduces the complexity of handling long-context tasks and enhances interpretability by splitting the work across multiple agents. However, CoA has some limitations: the communication between agents could be improved by leveraging in-context learning or fine-tuning LLMs to optimize their interaction; and the architecture needs further refinement to reduce computational costs and latency, particularly in tasks requiring multiple rounds of communication between agents. II-A 3 Agent Forest The Agent Forest method, introduced by Li et al. [9] in their paper ”More Agents is All You Need,” focuses on scaling LLM performance by simply increasing the number of agents. The method employs a sampling-and-voting approach: multiple agents generate responses, and the final answer is determined through majority voting. This technique demonstrates that increasing the number of agents improves performance, particularly on complex tasks. However, it also reveals limitations: the performance gains from Agent Forest depend on the inherent difficulty of the task, with diminishing returns for overly complex or overly simple tasks; and the approach increases computational costs due to the need for multiple LLM queries, requiring optimization of the sampling phase to enhance cost efficiency. II-A 4 Mixture-of-Agents The Mixture-of-Agents (MoA) architecture proposed by Wang et al. [10] offers a layered design where agents collaborate in both proposer and aggregator roles. Proposers generate