Agentic Lybic: Multi-Agent Execution System with Tiered Reasoning and Orchestration Conference: CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Creant el demà junts, Barcelona CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Creant el demà j
Liangxuan Guo, Bin Zhu, Qingqian Tao, Kangning Liu, Xun Zhao, Xianzhe Qin, Jin Gao, Guangfu Hao · 2025
Read Agentic Lybic: Multi-Agent Execution System with Tiered Reasoning and Orchestration Conference: CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Creant el demà junts, Barcelona CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Creant el demà j by Liangxuan Guo, Bin Zhu, Qingqian Tao, Kangning Liu, Xun Zhao, Xianzhe Qin, Jin Gao, Guangfu Hao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-11067.txt.
Introduction
Liangxuan Guo and 7 co-authors present Agentic Lybic, a team of AI agents that operates a desktop computer, in this 2025 paper. Single agents fumble long, multi-step tasks through poor coordination and weak checking. This system's whole structure runs as a finite-state machine, which routes work between parts as conditions change.
A controller and a manager direct three workers: one writes code, one clicks through graphical interfaces, and one supports decisions. An evaluator gates quality, and the state machine picks the route for each subtask, allowing replans after errors. On the OSWorld benchmark the system reached a success rate above half of tasks within fifty steps, which the authors report as the best result at the time.
Catalog record
| CREATOR | Liangxuan Guo, Bin Zhu, Qingqian Tao, Kangning Liu, Xun Zhao, Xianzhe Qin, Jin Gao, Guangfu Hao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-11067
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Agentic Lybic: Multi-Agent Execution System with Tiered Reasoning and Orchestration Conference: CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Creant el demà junts, Barcelona CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Creant el demà junts, Barcelona ISBN: 978-1-4503-XXXX-X/2026/04 CCS: Computing methodologies Multi-agent systems CCS: Computing methodologies Multi-agent planning CCS: Software and its engineering Layered systems CCS: Human-centered computing Graphical user interfaces Liangxuan Guo, Bin Zhu, Qingqian Tao, Kangning Liu, Xun Zhao, Xianzhe Qin, Jin Gao, Guangfu Hao Abstract . Autonomous agents for desktop automation struggle with complex multi-step tasks due to poor coordination and inadequate quality control. We introduce Agentic Lybic , a novel multi-agent system where the entire architecture operates as a finite-state machine (FSM). This core innovation enables dynamic orchestration. Our system comprises four components: a Controller, a Manager, three Workers (Technician for code-based operations, Operator for GUI interactions, and Analyst for decision support), and an Evaluator. The critical mechanism is the FSM-based routing between these components, which provides flexibility and generalization by dynamically selecting the optimal execution strategy for each subtask. This principled orchestration, combined with robust quality gating, enables adaptive replanning and error recovery. Evaluated officially on the OSWorld benchmark, Agentic Lybic achieves a state-of-the-art 57.07% success rate in 50 steps, substantially outperforming existing methods. Results demonstrate that principled multi-agent orchestration with continuous quality control provides superior reliability for generalized desktop automation in complex computing environments. Figure 1. Overview of the Agentic Lybic multi-agent system architecture. The Manager decomposes user queries into subtasks using DAG planning. Specialized Workers (Operator, Technician, Analyst) execute actions through the Executor, while the Evaluator provides continuous quality monitoring. The Global State maintains system information and enables dynamic coordination between components. † † cc-license: by 1. Introduction The automation of complex desktop tasks through autonomous agents represents one of the most challenging frontiers in artificial intelligence, requiring systems that can navigate intricate multi-step workflows while maintaining reliability and efficiency ( Zhang et al. 2025a ; Nguyen et al. 2025 ) . Recent advances in computer-using agents have demonstrated promising capabilities in executing tasks through Graphical User Interfaces (GUIs), with vision-language models enabling increasingly sophisticated interactions with visual elements ( Deepmind 2025 ; OpenAI 2025b ; OpenAI 2024 ; OpenAI 2025a ; Qin et al. 2025 ) . Current approaches to desktop automation typically fall into two categories: GUI-centric agents that rely exclusively on visual interaction ( Agashe et al. 2025a ; Yang et al. 2025 ) , and hybrid systems that combine GUI manipulation with programmatic execution. While GUI-only agents offer intuitive human-like interaction patterns, they suffer from brittleness in complex scenarios due to visual grounding ambiguity and accumulated error propagation over long sequences. Recent hybrid approaches, such as CoAct-1 ( Song et al. 2025 ) , have addressed some of these limitations by introducing specialized programming agents alongside GUI operators, achieving notable improvements in both efficiency and success rates. However, these systems’ core limitation lies in their handling of long-horizon tasks. They often employ a simplistic "delegate-and-forget" approach, which lacks the continuous oversight and adaptive re-planning needed for robust error recovery. Consequently, they fail to implement the sophisticated routing mechanisms and comprehensive quality control essential for coordinating multiple functional modules over extended workflows. The core challenge lies not merely in expanding the action space of computer-using agents, but in orchestrating multiple specialized components through principled coordination mechanisms that can dynamically adapt to changing task requirements and handle execution failures gracefully ( Tallam 2025 ) . Existing systems often treat task decomposition and execution as separate phases, lacking the continuous feedback loops necessary for robust long-horizon performance ( Hu et al. 2024 ) . Furthermore, current quality assessment approaches are typically binary and reactive, missing opportunities for proactive intervention ( Sun et al. 2025 ) and incremental course correction. To address these issues above, we draw inspiration from finite-state machines (FSMs), which provide a structured and predictable framework for managing complex, state-dependent workflows. In this work, we introduce Agentic Lybic , a novel FSM-based multi-agent execution system, which could addresse these fundamental limitations through a tiered reasoning architecture and sophisticated orchestration framework. Our system advances beyond simple hybrid execution by implementing a comprehensive four-tier architecture: a Controller that manages global state and decision triggers, a Manager for intelligent task decomposition and adaptive re-planning, a Worker subsystem with three specialized roles (Technician for system operations, Operator for GUI interactions, and Analyst for decision support), and an Evaluator that provides continuous quality assessment and intervention triggers. The key innovation of our approach lies in the dynamic orchestration mechanism (i.e., routing) that seamlessly coordinates between different functional modules based on ongoing task assessment, coupled with a comprehensive quality gate system that enables persistent monitoring, adaptive re-planning, and robust error recovery. Unlike existing systems that perform static task delegation, Agentic Lybic implements a feedback-driven execution model where performance is continuously monitored through multiple trigger mechanisms (periodic checks, stagnation detection, and success verification), enabling the system to adaptively adjust its strategy mid-execution. Our tiered reasoning approach introduces several novel contributions: (1) a state-aware orchestration framework that dynamically selects optimal execution strategies based on task characteristics and current system state, (2) a comprehensive quality gate system with multiple intervention triggers that enable proactive error handling and adaptive re-planning, (3) a specialized worker architecture that provides fine-grained control over different execution modalities while maintaining seamless coordination, and (4) an incremental clarification policy that systematically addresses visual ambiguity in GUI-dense environments. We evaluate Agentic Lybic on the challenging OSWorld benchmark, where our system achieves a new state-of-the-art success rate of 57.07% in 50 steps, representing a substantial improvement over existing methods including the recent CoAct-1 ( Song et al. 2025 ) (56.4%) and agent s2.5 ( Agashe et al. 2025a ) (54.2%). Beyond raw performance gains, our system demonstrates superior reliability in long-horizon scenarios and maintains efficiency through proactive quality control. Our results demonstrate that principled multi-agent orchestration with continuous quality assessment provides a more robust and scalable foundation for generalized desktop automation, opening new possibilities for autonomous task execution in complex computing environments. The tiered reasoning approach not only improves success rates but also provides a systematic framework for handling the inherent complexity and uncertainty of real-world desktop automation tasks. 2. Related Work 2.1. Screen Understanding and Visual Grounding A foundational challenge in GUI automation lies in accurately perceiving and grounding interface elements from raw pixel input. Early approaches focused on developing sophisticated screen parsing capabilities that could identify and locate interactive elements without relying on structured representations like DOM trees or accessibility hooks ( Lai et al. 2024 ) . OmniParser ( Lu et al. 2024 ; Yu et al. 2025 ) pioneered this direction by learning screen-parsing primitives for pure vision-based understanding, enhancing the ability of large models to generate accurate actions in interface regions through interactive icon detection and semantic element extraction. The grounding problem—mapping natural language instructions to actionable screen locations—has been addressed through several specialized systems. SeeClick introduced instruction-to-target grounding capabilities ( Cheng et al. 2024 ) , while Aria-UI ( Yang et al. 2024 ) and UGround ( Gou et al. 2024 ) extended this to universal GUI grounding across diverse interfaces. OS-Atlas (