SEMAG: Self-Evolutionary Multi-Agent Code Generation
Yulin Peng, Haowen Hou, Xinxin Zhu, Ying Tiffany He , F. Richard Yu · 2026
Read SEMAG: Self-Evolutionary Multi-Agent Code Generation by Yulin Peng, Haowen Hou, Xinxin Zhu, Ying Tiffany He , F. Richard Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15707.txt.
Introduction
Peng and co-authors present SEMAG, a self-evolving team system for code generation. Fixed workflows and hand-picked models adapt poorly as task difficulty shifts. SEMAG splits coding work into planning, coding, debugging, and discussion stages. Its agents can reach the newest models in real time. The backbone model upgrades itself when better options appear. The authors report new top Pass@1 scores across benchmarks. On CodeContests it beat prior methods by 3.3 percent at equal backbones. Self-picked backbones lifted the score to 52.6 percent.
Catalog record
| CREATOR | Yulin Peng, Haowen Hou, Xinxin Zhu, Ying Tiffany He , F. Richard Yu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33739 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.15707
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SEMAG: Self-Evolutionary Multi-Agent Code Generation Yulin Peng, Haowen Hou, Xinxin Zhu, Ying Tiffany He , F. Richard Yu Abstract Large Language Models (LLMs) have made significant progress in handling complex programming tasks. However, current methods rely on manual model selection and fixed workflows, which limit their ability to adapt to changing task complexities. To address this, we propose SEMAG, a Self-Evolutionary Multi-Agent code Generation framework that mimics human coding practices. It decomposes programming tasks into stages, including planning, coding, debugging, and discussion, while adapting workflows to task difficulty. Its self-evolutionary agents can access the latest models in real time and automatically upgrade the backbone model. SEMAG sets new state-of-the-art Pass@1 accuracy across benchmarks. Using identical backbone models, SEMAG outperforms prior methods by 3.3% on CodeContests. When augmented with self-evolutionary model selection that automatically identifies optimal backbones, SEMAG reaches 52.6%, showcasing both framework effectiveness and adaptability to evolving LLM capabilities. 1 Introduction Large Language Models (LLMs) have demonstrated substantial progress in code generation and completion, driven by large-scale pretraining on diverse codebases. The GPT series 1 ; 15 , CodeLLaMA-2 29 , Qwen2.5-Coder 14 , and DeepSeek-v3 24 exhibit strong coding capabilities, unlocking new avenues for automated software development. In parallel, multi-agent frameworks and debugging-enhanced methodologies—such as planning-centric workflows 20 , self-debugging paradigms 8 , and collaborative agent systems 38 —have shown promising performance on standard benchmarks. Nonetheless, real-world scenarios present open-ended tasks, constrained computational budgets, and evolving specifications, revealing critical limitations in current approaches. First, frameworks such as Self-Debugging 8 , LDB 38 typically adopt a fixed reasoning depth . On simple tasks, they introduce unnecessarily complex workflows, leading to redundant computation and excessive token usage, while on difficult tasks, the shallow reasoning depth results in poor success rates. Although hierarchical prompting has been shown to mitigate unnecessary reasoning 6 , these approaches still lack a principled mechanism to adapt reasoning depth dynamically to task complexity. Figure 1: Overview workflow of Self-Evolution Agents. Agents integrate insights from recent research, news, and community discussions, dynamically identify and deploy the most suitable models. Figure 2: Overview of SEMAG. (1) Self-Evolve: Agents dynamically select optimal backbone LLMs per task requirements. (2) Plan: Planning Agent creates solution plans validated by Plan Verifying Agent through I/O simulation. (3) Debug: Coding Agent generates code; upon failure, specialized agents (Embedding Trace, Code Explaining, Suggesting, Debugging) collaboratively refine using trace logs. (4) Debate: When debugging stalls, Debating Agents propose alternatives with Discriminating Agent selecting the optimal configuration. Second, current pipelines utilize a single debugging iteration. When initial outputs diverge significantly from the target, systems are prone to local minima. Though advanced reasoning paradigms such as Chain-of-Thought 34 , Tree-of-Thoughts 35 , and parallel candidate exploration 21 enhance complex reasoning, they lack explicit discussion–decision phases that aggregate diverse reasoning trajectories for improved synthesis. Third, most systems are tightly coupled to a single backbone model. Frameworks built on GPT 1 ; 15 , Gemini 31 ; 32 , or Claude 2 typically depend on a static model throughout execution. As task characteristics shift or new models emerge, backbone switching often requires manual intervention, limiting adaptability and scalability. To address these challenges, we propose SEMAG , a S elf- E volutionary M ulti- A gent code G eneration framework. Our contributions are summarized as follows: • Adaptive hierarchical prompting: We propose a dynamic strategy that adjusts reasoning depth based on task complexity. • Collaborative self-evolution: We introduce discussion–decision module enabling escape from local optima and adaptive backbone switching. • Empirical gains: Achieves state-of-the-art performance on seven benchmarks. With controlled backbone comparison, SEMAG improves 3.3% over the previous best method on CodeContests; with self-evolutionary model selection, it further reaches 52.6%. We evaluate SEMAG across seven text-to-code benchmarks, including four foundational datasets (HumanEval, MBPP, HumanEval-ET, MBPP-ET) and three competition-level benchmarks (APPS, LiveCode, CodeContests). Experimental results show that SEMAG achieves new state-of-the-art performance, including 98.8% Pass@1 7 ; 10 on HumanEval, 87.6% on MBPP, and 65.0% on LiveCode. Most notably, on the most challenging dataset CodeContests, SEMAG achieves 38.0% Pass@1 accuracy with GPT-4o (3.3% improvement over LPW under the same backbone). When augmented with self-evolutionary model selection that automatically identifies the optimal backbone, SEMAG further reaches 52.6%. These results demonstrate that SEMAG achieves superior performance and resource efficiency, while offering strong adaptability to evolving programming tasks. 2 Related Work 2.1 Traditional Approaches to Program Synthesis Program synthesis has a long-standing research foundation in artificial intelligence 33 ; 25 . Traditional methods leverage search strategies and data flow analysis 27 . Early efforts aimed to advance automatic programming and to identify viable approaches 5 ; 30 or explore large program spaces through domain-specific languages 28 ; 11 . These approaches struggle with generalization and scalability due to search space complexity. 2.2 Large Language Models for Code Synthesis Pretrained language models have enhanced code synthesis, with specialized models such as Qwen2.5-Coder 14 , CodeLLaMA-2 29 , Mistral 18 , and DeepSeek-v3 24 excelling in programming tasks. General-purpose models, including GPT 1 ; 15 , Gemini 31 ; 32 , and Claude 2 , also demonstrate robust code generation capabilities. However, these models still face challenges related to syntactic correctness, semantic alignment, generation robustness, and version conflicts. As a result, more refined control and evaluation mechanisms for code generation are necessary. 2.3 Prompting and Debugging Techniques Researchers have proposed various prompting and debugging techniques to improve code generation. Prompting strategies generally fall into three categories: retrieval-based 16 , planning-based 36 , and debugging-based 8 approaches. These aim to guide LLMs in decomposing complex tasks into manageable parts through step-by-step reasoning. Techniques such as Chain-of-Thought 34 , Tree-of-Thoughts 35 , and cumulative reasoning mimic human problem-solving paths, significantly enhancing model performance on complex tasks 39 ; 37 . More advanced methods simulate the software development process by constructing multiple candidate programs and exploring the solution space in parallel 21 ; 3 . Debugging systems such as Self-Debugging 8 and LDB 38 iteratively refine code using model explanations, execution, and human feedback. However, their effectiveness decreases when the initial code diverges from the intended function. To improve generation quality with limited supervision, some methods break down the coding task by incorporating visible test cases, step-by-step verification 13 ; 22 ; 26 , and natural language instructions to improve controllability and alignment. Previous methods either fix reasoning depth—wasting compute on simple tasks and underperforming on hard ones—or rely on a single LLM, limiting recovery from failures. SEMAG tackles both with three mechanisms: a hierarchical controller that scales from one-shot to multi-step planning based on feedback; a discussion–decision phase where agents critique and merge solutions to avoid local optima; and an automatic model selector that switches to a more capable backbone, boosting Pass@1 accuracy as difficulty rises. 3 Method We present a hierarchical multi-agent framework for code synthesis that adapts to task complexity through progressive refinement levels, coupled with a self-evolution mechanism for dynamic model selection. The overview of SEMAG is shown in Figure 2 . 3.1 Problem Formulation We define a code generation task as \mathcal{T}=(P,S,\mathcal{C}) where P\in\mathcal{P} is problem description, S=\{(x_{i},y_{i})\}_{i=1}^{n} are input-output examples, and \mathcal{C} is the program space. The core agent operations are: \begin{split}\text{CODER}&:\mathcal{P}\times\mathcal{S}\times\Pi\times\Theta\rightarrow\mathcal{C},\\ \text{PLANNER}&:\mathcal{P}\times\mathcal{S}\rightarrow\Pi,\\ \text{VERIFIER}&:\Pi\times\mathcal{P}\times\mathcal{S}\rightarrow{0,1}\times\Pi\times\mathcal{L},\\ \text{DEBUGGER}&:\mathcal{C}\times\Sigma\rightarrow\mathcal{C}\end{split} (1) where \Pi is the plan space, \Theta parameters, \mathcal{L}