Language Model Teams as Distributed Systems

Elizabeth Mieczkowski, Katherine M. Collins, Ilia Sucholutsky, Natalia Vélez, Thomas L. Griffiths · 2026

Read Language Model Teams as Distributed Systems by Elizabeth Mieczkowski, Katherine M. Collins, Ilia Sucholutsky, Natalia Vélez, Thomas L. Griffiths (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-12229.txt.

Introduction

Mieczkowski, Collins, Sucholutsky, Vélez, and Griffiths frame teams of language models as distributed systems. LLM teams now deploy at scale, but basic design questions lack a firm base. When does a team help, how many agents should it hold, and when does one agent win? Distributed computing faced these same trade-offs for decades of study. Many core gains and pains from that field show up again in LLM teams. The authors mine that cross-talk for practical rules on team design and testing.

Catalog record

CREATORElizabeth Mieczkowski, Katherine M. Collins, Ilia Sucholutsky, Natalia Vélez, Thomas L. Griffiths
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51439
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.12229

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Language Model Teams as Distributed Systems Elizabeth Mieczkowski, Katherine M. Collins, Ilia Sucholutsky, Natalia Vélez, Thomas L. Griffiths Abstract Large language models (LLMs) are growing increasingly capable, prompting recent interest in LLM teams . Yet, despite increased deployment of LLM teams at scale, we lack a principled framework for addressing key questions such as when a team is helpful, how many agents to use, how structure impacts performance — and whether a team is better than a single agent. Rather than designing and testing these possibilities through trial-and-error, we propose using distributed systems as a principled foundation for creating and evaluating LLM teams. We find that many of the fundamental advantages and challenges studied in distributed computing also arise in LLM teams, highlighting the rich practical insights that can come from the cross-talk of these two fields of study. K eywords Large language models \cdot collaboration \cdot LLM teams \cdot distributed computing 1 Introduction “Adding manpower to a late software project, makes it later.” – Frederick P. Brooks, The Mythical Man Month [ 9 ] The quest for artificial intelligence (AI) has long captivated human minds [ 42 ] , and the remarkable capabilities of large language models (LLMs) have sparked widespread excitement that we are approaching this goal. Yet, single models remain fundamentally limited in memory, context, and ability. A natural response is to compose them into teams, or collections of interacting agents that can divide work, communicate, and pool their individual resources. The promise is compelling: just as people achieve far more together than alone [ 19 , 41 ] , teams of LLM agents could extend capabilities beyond those of any single monolithic model. However, achieving this promise requires addressing several questions: when do teams help, when do they hurt, and how should they be designed and deployed to maintain performance and efficiency? Failing to appropriately address these questions can have substantial implications. Individual LLM calls are already resource-intensive, demanding substantial compute and energy to query, and racking up hefty monetary costs to users. These resource demands grow rapidly as agents exchange messages and iterate on shared work. Without careful orchestration to address coordination challenges raised by teams [ 5 ] , agents may overwrite one another, produce redundant outputs, conflict in decisions, propagate errors through chains of reasoning, and reinforce incorrect conclusions through mutually sycophantic exchanges. The journey from single- to multi-agent LLM systems mirrors the historical evolution of computing itself [ 36 , 43 ] . Early computing systems relied on single processors, but as demands for scale grew, engineers turned to distributed architectures that combined many machines to achieve greater capacity and robustness (Figure 1 A). However, this also brought about new challenges that have defined core research topics in distributed computing for decades, such as issues of coordination, consistency, and task assignment (Figure 1 B) . LLM teams appear to be following a similar path; as single models approach limits in context and reliability, there is increasing interest in assembling teams of interacting agents to overcome their individual constraints. However, the design and deployment of these teams have revealed many additional complexities, such as limited scaling depending on the task [ 49 , 24 ] , reduced collective problem-solving abilities [ 31 ] , and amplifications of errors, false confidence, and confusion [ 38 ] . Viewing LLM teams through the lens of distributed systems (Figure 1 C) both explains observed inefficiencies and generates actionable solutions, providing principled design guidelines, testable hypotheses, and normative benchmarks for evaluating team behavior across varying tasks and conditions. Rather than optimizing LLM teams through trial and error, this framework offers a rigorous foundation for understanding where coordination breaks down and how to improve it. In this paper, we outline concrete guidelines from distributed computing for designing and evaluating efficient LLM teams, backed by initial empirical demonstrations. We begin by reviewing existing approaches to LLM team design and their limitations (Section 2), then establish the formal correspondence between LLM teams and distributed systems (Section 3). Section 4 provides an empirical test of this analogy. We first show that LLM teams collaborating in simplified settings with pre-assigned tasks exhibit efficiency gains that generally mirror classic scaling laws in distributed computing. We then extend this evaluation to self-coordinating decentralized teams, finding that LLM teams exhibit the same challenges, such as consistency conflicts, architectural tradeoffs, communication overhead, and stragglers, that distributed systems theory anticipates. Finally, Section 5 outlines future directions and limitations of this framework. These connections position distributed computing as a principled foundation for designing and evaluating LLM teams. As these teams are deployed, failure to address these scalability and coordination concerns risks more than inefficient computation. Poorly designed systems may waste enormous computational resources while producing unreliable outputs. Because these failures emerge from interactions among agents, they can be difficult to diagnose or correct without principled design frameworks. A formal foundation for LLM team architectures – and deciding whether to implement a team in the first place – therefore offers a path toward deploying these systems efficiently, robustly, and responsibly at scale. 2 Background 2.1 The rise of LLM teams LLM teams, or multi-agent systems in which multiple language model agents coordinate through task decomposition, role specialization, and inter-agent communication, are increasingly being deployed in both research and production. Experimental studies show that sufficiently capable LLM agents can successfully cooperate in simple economic settings like common-pool resource problems [ 32 ] . Researchers have argued that such teams may be particularly valuable for scientific discovery, where distributing subtasks across agents can increase effective capacity, enable parallel exploration, and reduce individual errors through cross-checking [ 52 ] . Several frontier labs are deploying agentic coding teams that allow users to coordinate multiple LLM instances working together on shared coding tasks [ 33 , 3 ] . Despite this enthusiasm, there is a growing need to evaluate these systems rigorously, both in terms of individual models [ 35 ] and their emergent collective behaviors [ 45 ] . 2.2 Benefits, limitations, and risks of LLM teams There is growing empirical evidence that the performance of multi-agent LLM teams relative to single agents is mixed and highly task-dependent. On one hand, teams can outperform individual agents by aggregating complementary reasoning paths, enabling cross-checking, and distributing long-horizon context across multiple agents. LLM teams have demonstrated impressive scientific discovery capabilities, including the design of novel nanobodies [ 40 ] and autonomous hypothesis-generation pipelines for chemistry [ 7 ] . Controlled studies further show that ensembles of interacting LLM agents can improve accuracy by enabling diversity and debate [ 6 , 14 ] , and by decomposing contexts that would exceed a single agent’s limits across multiple agents [ 53 ] . Early benchmarks reinforced this optimistic view: increasing the number of agents answering a question and asking them to vote upon the correct output often increased accuracy [ 28 ] . However, subsequent work has identified important limitations and failure modes. Performance gains from adding agents are neither linear nor guaranteed; instead, they depend strongly on factors such as agent heterogeneity and coordination structure [ 31 , 49 ] . Multi-agent systems tend to benefit on parallelizable or decomposable tasks but can degrade on

Read the complete text