When Does Hierarchy Help? Benchmarking Agent Coordination in Event-Driven Industrial Scheduling
Ziqi Wang, Yuhao Yang, Zhiwei Ling, Wenzhuo Qian, Hailiang Zhao · 2026
Read When Does Hierarchy Help? Benchmarking Agent Coordination in Event-Driven Industrial Scheduling by Ziqi Wang, Yuhao Yang, Zhiwei Ling, Wenzhuo Qian, Hailiang Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13172.txt.
Introduction
Wang and colleagues offer DESBench, a test suite for agent teamwork in factory planning driven by events. Current test suites cover weakly linked tasks and say little about ranks, shared limits and shifting states. DESBench scores success, limit fit, teamwork speed and toughness across central, ranked, heterarchical and holonic styles. Central control proves tough but scales poorly, ranked control gains speed yet slips between levels, and each style trades gains for costs.
Catalog record
| CREATOR | Ziqi Wang, Yuhao Yang, Zhiwei Ling, Wenzhuo Qian, Hailiang Zhao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 153733 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13172
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
When Does Hierarchy Help? Benchmarking Agent Coordination in Event-Driven Industrial Scheduling Ziqi Wang, Yuhao Yang, Zhiwei Ling, Wenzhuo Qian, Hailiang Zhao Abstract Recent advances in agent and multi-agent systems have shown strong performance on tool use, reasoning, and collaborative tasks. However, existing benchmarks mostly evaluate task completion in weakly coupled environments, and provide limited support for studying coordination in shared, dynamically evolving systems with hierarchy and coupled constraints. This leaves an important question underexplored: when do different coordination paradigms succeed or fail? We introduce Distributed Event-driven Scheduling Benchmark (DESBench), a benchmark for evaluating agent coordination in hierarchical event-driven scheduling. Built on a shared discrete-event driven environment in industrial scheduling, our benchmark captures multi-timescale decision making, partial observability, and dynamically coupled constraints. We define tasks and metrics that evaluate effectiveness, constraint alignment, coordination efficiency, and robustness, and focus on four representative coordination paradigms: centralized, hierarchical, heterarchical, and holonic. These paradigms correspond to distinct mechanisms of information flow, decision authority, and conflict resolution. Our controlled evaluations reveal clear coordination trade-offs: centralized coordination is robust and communication-efficient but scales poorly with difficulty; hierarchical coordination improves efficiency through decomposition but suffers from cross-level misalignment; heterarchical coordination is flexible but communication-heavy; and holonic coordination satisfies constraints well but loses global robustness. These findings demonstrate that coordination design fundamentally shapes agent system behavior in complex environments, revealing structural trade - offs that cannot be captured by outcome metrics alone and underscoring the imperative for more adaptive, principled, and dynamic coordination mechanisms in future MAS research. DESBench is available at: https://github.com/Mr-lander/DESBenchmark. 1 Introduction The rapid progress of agent and multi-agent systems (MAS) has expanded their use beyond single-step reasoning and tool invocation toward more complex forms of coordination and collaborative problem solving [ 1 ] . A growing body of work explores how multiple agents can coordinate to solve complex problems, often leveraging large language models (LLM) or learned policies [ 2 ] . However, despite these advances, current evaluation benchmarks remain largely focused on task completion in weakly coupled environments, where interactions are short-horizon, state transitions are loosely coupled, and coordination plays a limited role in shaping long-term system behavior. In contrast, real - world decision - making often takes place in shared, dynamically changing environments where interaction through a common system state yields delayed and widespread effects [ 3 , 4 ] . Industrial scheduling problems provide an excellent testing ground for these scenarios, where coordination is not merely a mechanism for dividing work, but a central factor that determines system performance, constraint satisfaction, and robustness. Existing benchmarks are insufficient for studying coordination for three key reasons: • Traditional scheduling and optimization benchmarks primarily evaluate solution quality under implicit centralized or fixed control assumptions, providing limited insight into how alternative coordination structures influence system behavior. • Many existing agent benchmarks lack a fully shared and dynamically evolving environment, and therefore only partially capture coordination settings where actions influence future dynamics through resource coupling and delayed feedback. • Current environments rarely integrate hierarchical organization with dynamically coupled constraints in a unified setting, making it difficult to systematically analyze how different coordination paradigms operate under multi-level decision processes. As a result, a fundamental question remains underexplored: When do different coordination paradigms succeed or fail in complex and dynamic environments? In the spirit of Aristotle’s insight that the whole is greater than the sum of its parts , coordination mechanisms can give rise to emergent collective behavior that cannot be inferred from isolated component performance alone. To address this gap, we introduce Distributed Event-driven Scheduling Benchmark (DESBench), a benchmark for evaluating agent coordination in hierarchical, event-driven industrial scheduling, using it as a structured testbed for studying coordination in dynamic MAS. Our benchmark is built on a shared discrete-event simulation (DES) environment, where agents operate under partial observability, multi-timescale decision processes, and dynamically coupled constraints such as resource budgets and system capacity. Unlike prior environments [ 5 , 6 ] , agent decisions directly influence the evolution of the underlying system, affecting future availability, resource usage, and constraint pressure. We focus on four representative coordination paradigms: centralized, hierarchical, heterarchical, and holonic, each corresponding to distinct mechanisms of information flow, decision authority, and conflict resolution (Figure 1 ). These paradigms are instantiated as executable agent systems using orchestration frameworks, enabling systematic comparison within a unified environment. We design evaluation indices that measure multiple dimensions of agent performance, including effectiveness, constraint alignment, coordination efficiency, and robustness under disruptions. Figure 1: Four typical coordination paradigms: centralized, hierarchical, heterarchical, and holonic. Each represents a distinct mechanism of information flow, decision authority, and conflict resolution. Our empirical results show that each paradigm exposes a distinct bottleneck: centralized coordination is robust but difficulty-sensitive, hierarchical coordination improves scheduling efficiency but risks level misalignment, heterarchical coordination gains adaptability at the price of heavy interaction, and holonic coordination preserves constraints but weakens global progress. These observations underscore that coordination design fundamentally governs system behavior in complex environments and point toward the need for adaptive and context - aware coordination mechanisms in future research. Our work presents three key innovations: 1. We introduce DESBench for evaluating agent coordination in event-driven industrial scheduling, capturing multi-timescale decision making and dynamically coupled constraints in shared and evolving environments. 2. We implement four representative coordination paradigms, including centralized, hierarchical, heterarchical, and holonic, each characterized by distinct mechanisms of information flow, decision authority, and conflict resolution. We also provide a scaffold that enables the design and evaluation of new coordination strategies within our benchmark setting. 3. We establish a unified evaluation framework with four metric dimensions, including effectiveness, constraint alignment, coordination efficiency, and robustness, and use it to reveal key trade-offs across coordination paradigms, providing insights for designing more adaptive and scalable coordination mechanisms in future MAS. 2 Related Work Agent Coordination Mechanisms. Agent coordination is fundamental in MAS, enabling distributed decision makers to share information and resolve conflicts toward collective goals [ 7 ] . Centralized coordination assigns a global controller to aggregate system-wide information and compute joint decisions. For example, EvoAgent [ 8 ] employs a centralized puppeteer to dynamically schedule and reprioritize agent actions during collaboration. While such designs can optimize global objectives under full observability, they scale poorly under uncertainty and partial observability [ 9 , 10 ] . To address these limitations, recent work explores structured and decentralized coordination paradigms that balance global coherence with local autonomy [ 11 ] . Hierarchical coordination decomposes tasks into layered subproblems, enabling modular decision making and controlled cross-level interaction [ 12 , 13 , 14 ] . For instance, SeqComm [ 11 ] combines inter-agent negotiation with top-down execution ordering across levels. Recent decentralized algorithms advance adaptive coordination through evolving interaction topologies and interaction-based reward design, as exemplified by AgentNet and CoMAS [ 15 , 16 ] . Extensions incorporate adaptive clustering [ 17 ] and privacy-aware knowledge sharing [ 18 ] to improve scalability while maintaining global consistency. Heterarchical coordination relies on peer-to-peer interaction and decentralized negotiation to reach consensus under local information constraints [ 19 ] , enhancing robustness in large-scale systems. Hybrid holonic coordination [ 20 ] integrates hierarchical structure with decentralized autonomy, treating agents as self-contained units within cooperative collectives to enable dynamic reconfiguration and multi-level interaction. Benchmarking