Large Language Models over Networks: Collaborative Intelligence under Resource Constraints

Liangqi Yuan, Wenzhi Fang, Shiqiang Wang, H. Vincent Poor, Christopher G. Brinton · 2026

Read Large Language Models over Networks: Collaborative Intelligence under Resource Constraints by Liangqi Yuan, Wenzhi Fang, Shiqiang Wang, H. Vincent Poor, Christopher G. Brinton (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08626.txt.

Introduction

Liangqi Yuan and five coauthors survey how language models spread across phones, edge boxes, and cloud servers can work together. Cloud only service fails where connections drop, answers must be instant, or data must stay local, while single devices lack memory and power. Their collaborative intelligence view joins models at the task level through plain or structured messages.

Collaboration runs two ways: vertical splits between device and cloud, and horizontal teamwork among peer agents, often mixed in practice. The survey also covers learning how to route tasks and cooperate, plus open problems in scaling and trust. Networked AI research gains a map for sharing work under tight, uneven resources.

Catalog record

CREATORLiangqi Yuan, Wenzhi Fang, Shiqiang Wang, H. Vincent Poor, Christopher G. Brinton
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.08626

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Large Language Models over Networks: Collaborative Intelligence under Resource Constraints Liangqi Yuan, Wenzhi Fang, Shiqiang Wang, H. Vincent Poor, Christopher G. Brinton Abstract Large language models (LLMs) are transforming society, powering applications from smartphone assistants to autonomous driving. Yet cloud-based LLM services alone cannot serve a growing class of applications, including those operating under intermittent connectivity, sub-second latency budgets, data-residency constraints, or sustained high-volume inference. On-device deployment is in turn constrained by limited computation and memory. No single endpoint can deliver high-quality service across this spectrum. This article focuses on collaborative intelligence, a paradigm in which multiple independent LLMs distributed across device and cloud endpoints collaborate at the task level through natural language or structured messages. Such collaboration strives for superior response quality under heterogeneous resource constraints spanning computation, memory, communication, and cost across network tiers. We present collaborative inference along two complementary and composable dimensions: vertical device-cloud collaboration and horizontal multi-agent collaboration, which can be combined into hybrid topologies in practice. We then examine learning to collaborate, addressing the training of routing policies and the development of cooperative capabilities among LLMs. Finally, we identify open research challenges including scaling under resource heterogeneity and trustworthy collaborative intelligence. I Introduction Large language models (LLMs) have demonstrated unprecedented capabilities in question answering, code generation, and multi-modal reasoning, with applications spanning smartphone assistants, autonomous vehicles, and robotic systems. However, these capabilities rest on substantial computational and memory foundations. Frontier LLMs contain hundreds of billions to trillions of parameters, and both training and inference depend on resource-rich cloud data centers. At the same time, a growing number of applications cannot rely on cloud APIs alone. UAVs and field robots may enter connectivity-denied environments, closed-loop control and real-time agents cannot tolerate cloud round-trip latency, regulated domains such as healthcare and finance prohibit sensitive data from leaving the device, and sustained agentic workloads are bounded by per-token pricing and provider rate limits [ 15 ] . Lightweight LLMs with hundreds of millions to several billions of parameters make local execution on smartphones feasible, yet these compact LLMs still exhibit a significant capability gap relative to cloud frontier LLMs on complex tasks. Fig. 1: Collaborative intelligence of LLMs over networks. Prior work on deploying LLMs across network tiers has largely operated at the model level. On the training side, federated learning enables collaborative fine-tuning through gradient or adapter exchange [ 2 ] . On the inference side, techniques such as model partitioning, speculative decoding, and context compression reduce the latency or communication cost of serving a single LLM across devices and servers [ 8 ] . However, these model-level approaches share common limitations, as they require white-box access to model internals and assume tightly coupled execution across endpoints. More fundamentally, they accelerate a single model rather than orchestrate cooperation among heterogeneous endpoints, including proprietary cloud APIs accessible only through text interfaces, that must jointly handle tasks no single endpoint can complete alone. This article focuses on a distinct and increasingly important paradigm that we term collaborative intelligence , wherein multiple independent LLMs distributed across device and cloud endpoints collaborate at the task level through semantic exchanges of natural language or structured messages , rather than model parameters or intermediate tensors. Unlike model-level distribution, each node runs a complete, self-contained LLM instance and exchanges semantic-level information (queries, responses, task descriptions, and observation records) crosses the network. This design is inherently compatible with black-box API invocation, supports heterogeneous LLM architectures without requiring matched layer configurations, and enables flexible composition of specialized capabilities across nodes. As illustrated in Fig. 1 , such collaboration arises naturally in emerging applications, where smartphones offload complex queries to cloud LLMs via natural language, vehicles coordinate with home devices through structured commands, and cloud LLMs orchestrate UAV inspections by exchanging task descriptions with on-device LLMs — all without sharing LLM weights or internal states. Effective collaboration must happen automatically and in real time, jointly navigate quality, latency, communication, and cost under fluctuating conditions, and coordinate context and decisions across endpoints in ways that ad hoc workflows cannot support. The remainder of this article is organized around five contributions. We introduce collaborative intelligence as a task-level paradigm for networked LLMs, motivated by a resource landscape spanning computation, memory, communication, and cost that makes such collaboration both necessary and difficult (Sec. II ). Building on this, we present the system design of collaborative inference along two complementary and composable dimensions, vertical device-cloud and horizontal multi-agent (Sec. III ). We then review the learning techniques that equip LLMs to route requests and cooperate effectively (Sec. IV ). We further present a case study on device-cloud routing in multi-modal conversations, showing that a learned routing policy with stateful budget tracking breaks the quality-latency-cost tradeoff curve of prompt-based approaches (Sec. V ). Finally, we identify open challenges in scaling under heterogeneity and in building trustworthy collaborative systems, and outline directions for future research (Sec. VI ). TABLE I: Resource Constraints in Networked LLM Environments Resource On-Device Cloud Impact Computation FLOPs, GPU Availability • Compute resources shared with other tasks • High per-token latency GPU Clusters, Rate Limits • Abundant but shared; subject to queuing • Rate limits cap throughput • Inability to serve time-sensitive applications • Limits concurrent inference requests Memory RAM, Storage, KV-Cache • Constrained LLM sizes • Restricted KV-cache limits context length VRAM, Context Window • Memory allocation not user-configurable • Context window capped by provider • Unable to deploy more capable LLMs • Aggressive quantization degrades output quality Communication Bandwidth, Connectivity • High round-trip latency • Intermittent and unstable wireless links Throughput, Latency • Throttled responses under rate limits • Streaming delivery depends on link quality • Unreliable offloading and orchestration • Restricted real-time inference scenarios Cost Energy, Hardware • Finite battery limits sustained inference • Capital expense for device compute hardware Infrastructure, API Fees • Infrastructure fees for cloud compute • Per-token pricing bounds total queries • Bounded total inference under budget • Sustained high-volume inference economically infeasible II The Landscape of Networked LLMs: Challenges and Opportunities Today’s LLM ecosystem spans a wide resource spectrum. At one end, lightweight LLMs with a few billion parameters run locally on smartphones and edge devices, offering low latency and no per-query fees, though at the expense of device energy and hardware amortization. At the other end, cloud-based frontier LLMs with hundreds of billions of parameters deliver state-of-the-art quality but require network connectivity and incur per-token or subscription charges from the service provider [ 11 ] . Table I organizes the constraints that govern deployments between these extremes into four categories: computation, memory, communication, and cost. On the device side, limited compute and memory cap model size and context length, while energy and connectivity bound sustained or offloaded inference. On the cloud side, connectivity requirements, round-trip latency floors, provider rate limits, and data-residency rules render cloud-only inference infeasible for an important class of applications, independent of budget. The four categories are also tightly coupled, as quantizing an LLM to fit in device memory sacrifices output quality, while offloading to the cloud trades computation savings for latency and monetary cost. Fig. 2 makes the resulting tradeoff landscape concrete, showing that no single LLM dominates across all dimensions and that substantial heterogeneity

Read the complete text