Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization
Shan He, Runze WangZhuoyun DuHuiyu BaiZouying CaoYu Cheng, Bo Zheng [3pt] Future Living Lab of Alibaba[3pt] {shanhe.hs, yunze.wrz}@alibaba-inc.com · 2026
Read Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization by Shan He, Runze WangZhuoyun DuHuiyu BaiZouying CaoYu Cheng, Bo Zheng [3pt] Future Living Lab of Alibaba[3pt] {shanhe.hs, yunze.wrz}@alibaba-inc.com (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20714.txt.
Introduction
He and six colleagues introduce Textual Parameter Graph Optimization for multi-agent systems that learn to evolve. Existing methods focus on tuning, and optimizers never learn from experience. TPGO models the system as a Textual Parameter Graph of agents, tools, and workflows as nodes. Textual gradients, feedback from traces, pinpoint failures and suggest modifications. Its core, Group Relative Agent Optimization, learns from past experiences, so the system learns how to optimize itself. Experiments on GAIA and MCP-Universe show TPGO improves leading agent frameworks, with higher success rates.
Catalog record
| CREATOR | Shan He, Runze WangZhuoyun DuHuiyu BaiZouying CaoYu Cheng, Bo Zheng [3pt] Future Living Lab of Alibaba[3pt] {shanhe.hs, yunze.wrz}@alibaba-inc.com |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36107 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.20714
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization Shan He, Runze WangZhuoyun DuHuiyu BaiZouying CaoYu Cheng, Bo Zheng [3pt] Future Living Lab of Alibaba[3pt] {shanhe.hs, yunze.wrz}@alibaba-inc.com Abstract Designing and Optimizing multi-agent systems (MAS) is a complex, labor-intensive process of "Agent Engineering." Existing automatic optimization methods, primarily focused on flat prompt tuning , lack the structural awareness to debug the intricate web of interactions in MAS. More critically, these optimizers are static ; they do not learn from experience to improve their own optimization strategies. To address these gaps, we introduce Textual Parameter Graph Optimization (TPGO) , a framework that enables a multi-agent system to learn to evolve . TPGO first models the MAS as a Textual Parameter Graph (TPG) , where agents, tools, and workflows are modular, optimizable nodes. To guide evolution, we derive "textual gradients" , structured natural language feedback from execution traces, to pinpoint failures and suggest granular modifications. The core of our framework is Group Relative Agent Optimization (GRAO) , a novel meta-learning strategy that learns from historical optimization experiences. By analyzing past successes and failures, GRAO becomes progressively better at proposing effective updates, allowing the system to learn how to optimize itself . Extensive experiments on complex benchmarks like GAIA and MCP-Universe show that TPGO significantly enhances the performance of state-of-the-art agent frameworks, achieving higher success rates through automated, self-improving optimization . † † footnotetext: ∗ Equal contribution. † † footnotetext: † Corresponding author. 1 Introduction Figure 1: The pipelines of Traditional Agent Engineering and the proposed TPGO in multi-agent system optimization. The rapid advancement of Large Language Models (LLMs) has catalyzed a paradigm shift in artificial intelligence, moving from passive question-answering interfaces to autonomous agents capable of tool use and decision-making. This evolution has recently culminated in the development of Multi-Agent Systems (MAS) , where diverse agents collaborate to solve complex, multi-step problems ranging from software engineering to open-ended reasoning 8 ; 2 ; 3 . By decomposing intricate tasks into specialized sub-routines, MAS have demonstrated capabilities that significantly surpass solitary LLMs. However, the efficacy of these systems is critically dependent on the precise configuration of their textual components. An "Agent Engineer" must meticulously craft not only the system prompts for individual agents but also the descriptions of tools, the protocols for inter-agent communication, and the overarching workflow 9 ; 5 ; 12 . This process, often termed Agent Engineering , represents a high-dimensional and unstructured optimization challenge. As illustrated in Figure 1 (a), the opaque and non-deterministic nature of MAS interactions makes manual tuning a labor-intensive trial-and-error process that is difficult to scale and rarely converges to an optimal state. This establishes the automatic optimization of MAS as a crucial, yet unsolved, frontier. In real-world applications, this optimization challenge manifests in two primary forms. On the one hand, exploratory optimization addresses settings where the system must improve without a "golden" answer, relying on self-correction from execution feedback. On the other hand, imitative optimization seeks to align the system’s behavior with desired outcomes by learning from expert demonstrations or ideal solutions. To alleviate this manual burden, the first wave of research has explored Automatic Prompt Optimization (APO) 13 ; 15 . While promising, these methods fall short in addressing the unique complexities of MAS in two fundamental ways. First, they primarily focus on "flat" prompt optimization and lack the structural awareness required to navigate the intricate web of interactions within a multi-agent architecture. A system failure often stems not from a single flawed prompt, but from a subtle bug in a tool’s definition, a logical gap in the workflow, or a misaligned communication protocol, components that current APO methods are ill-equipped to identify and correct 1 . Second, and more fundamentally, existing optimizers are static ; they do not learn from the optimization process itself. They execute a search or apply a gradient-like update for a given problem, but they lack the mechanism to internalize past failures and successes to become better optimizers over time. This exposes a critical pain point: for MAS to achieve true autonomy, the optimization system itself must learn to evolve . This forms the core motivation for our work: to create an optimizer that not only refines the agent system but also refines its own optimization strategy through experience. To bridge these gaps, we propose Textual Parameter Graph Optimization (TPGO) , a novel framework that re-conceptualizes MAS optimization as a graph evolution problem , as depicted in Figure 1 (b). To address the structural challenge, we first formalize the system’s configuration as a Textual Parameter Graph (TPG) . In this representation, agents, tools, and logical units are modular nodes, and their interactions are directed edges. This structured view allows us to move beyond flat text editing to perform targeted, structural modifications on the system’s architecture. Crucially, to enable the optimizer to evolve, we introduce Group Relative Agent Optimization (GRAO) , a meta-learning mechanism that acts as the "brain" of the optimization process. By clustering historical error patterns and retrieving successful past optimization strategies, GRAO learns to generate more effective and targeted update proposals over time. It uses Textual Gradients , structured natural language feedback from execution traces, as its input signal, allowing it to reason about and correct semantic errors within the graph. In essence, GRAO empowers our framework to "learn how to optimize." Our contributions can be summarized as follows: • We propose TPGO , the first framework to treat multi-agent system optimization as a graph evolution problem , enabling both structural and semantic refinement of complex agent architectures. • We introduce the Textual Parameter Graph (TPG) , a structured representation that disentangles monolithic textual configurations into modular, individually optimizable semantic units. • We design Group Relative Agent Optimization (GRAO) , a novel self-evolving meta-optimization strategy that leverages historical experience to continuously improve the quality of its optimization proposals. • Extensive experiments demonstrate that our framework significantly enhances the performance and efficiency of state-of-the-art (SOTA) agent systems on complex benchmarks, achieving higher success rates through automated, evolutionary optimization . 2 Related Work 2.1 Language Agent Systems While single-agent systems (SAS) such as the ReAct framework 14 offer simplicity and efficiency, their capabilities are inherently limited. To overcome these limitations, LLM-based multi-agent systems (MAS) have emerged as a powerful paradigm for complex problem-solving. The prevailing design methodology involves decomposing a task into sub-tasks and assigning them to specialized agents that emulate collaborative human teams ( 8 ; 2 ) . This approach, however, introduces a vast design space with numerous critical choices that shape collaboration and behavior. For instance, at the system level, designers must define the collaboration structure, choosing from predefined workflows like hierarchical organizations ( 17 , e.g., MaCTG;) , iterative refinement, or multi-agent debate. At the agent level, their behavior is further guided by manually crafted personas ( 4 ) , which is crucial for achieving sophisticated outcomes beyond single-agent performance. Consequently, the overall system performance is highly sensitive to this vast array of design choices, including the composition of the agent team, the collaboration mechanism, the communication protocol, and the specific configuration of each agent’s persona and proactivity ( 16 ; 4 ) . 2.2 Automatic Prompt Optimization Recent efforts