CoRe-Code : Collaborative Reinforcement Learning for Code Generation

Zhihao Dou, Qinjian Zhao, Zhongwei Wan, Xiaoyu Xia, Sumon Biswas · 2026

Read CoRe-Code : Collaborative Reinforcement Learning for Code Generation by Zhihao Dou, Qinjian Zhao, Zhongwei Wan, Xiaoyu Xia, Sumon Biswas (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24812.txt.

Introduction

Zhihao Dou and co-authors improve multi-agent code generation with Collaborative Reinforcement Code, a framework for role-specialized language-model agents. Most code methods decode step by step without global planning, yielding locally coherent but globally weak solutions such as failing tests or slow code. CoRe-Code uses a simple Planner-Coder setup: the Planner drafts high-level plans and the Coder turns them into code. A collaboration-aware reinforcement learning stage sharpens specialization and alignment. Tests show it beats many baselines.

Catalog record

CREATORZhihao Dou, Qinjian Zhao, Zhongwei Wan, Xiaoyu Xia, Sumon Biswas
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78425
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24812

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CoRe-Code : Collaborative Reinforcement Learning for Code Generation Zhihao Dou, Qinjian Zhao, Zhongwei Wan, Xiaoyu Xia, Sumon Biswas Abstract Large language models (LLMs) have achieved strong performance in code generation, but most methods rely on autoregressive decoding without global planning, often leading to locally coherent yet globally suboptimal solutions (e.g., failing test cases or inefficient complexity). While recent approaches such as Chain-of-Thought (CoT) and multi-agent systems (MAS) introduce planning, their limited role specialization and coordination hinder performance on complex tasks. To address the challenges of coordination and specialization in multi-agent code generation, we propose Co llaborative Re inforcement Code ( CoRe-Code ), a framework for role-specialized LLM agents that enhances inter-agent coordination to generate more accurate and efficient code. CoRe-Code adopts a simple Planner–Coder paradigm, where the Planner produces high-level plans and the Coder executes them to generate code. We further introduce a collaboration-aware reinforcement learning stage based on Group Relative Policy Optimization (GRPO) to enhance role specialization and alignment. Experiments show that CoRe-Code outperforms a wide range of existing RL-based and multi-agent methods. In addition, we demonstrate that CoRe-Code can generalize to other multi-agent frameworks (e.g., Retrieval and Debugging agents), highlighting its flexibility and scalability. We evaluate CoRe-Code on multiple benchmarks of varying difficulty using three base models. Compared to existing baselines, the results show consistent improvements in accuracy, while also achieving higher efficiency in terms of execution time and memory usage, demonstrating the effectiveness and practicality of CoRe-Code . 1 Introduction With the rapid development of large language models (LLMs) in recent years, LLM-powered code generation methods have shown remarkable capabilities in a wide range of code generation tasks ( 37 ; 11 ; 28 ; 32 ; 2 ) . LLMs with advanced reasoning abilities, including DeepSeek ( 16 ; 17 ) , LLaMA ( 53 ) , Qwen ( 21 ) , and GPT ( 52 ) , have achieved notable results on a wide range of code generation benchmarks. During the generation process, these models typically follow an autoregressive decoding strategy, predicting the next token one step at a time based on the previously generated tokens. This sequential generation mechanism can be viewed as a token-level Markov process ( 38 ; 54 ; 65 ; 9 ) . However, such a generation mechanism may inherently lack global planning. As a result, it often produces outputs that, while locally coherent, are suboptimal solutions from a global perspective. ((a)) ((b)) ((c)) ((d)) Figure 1 : (a) CoT for code generation. (b) CoRe-Code for code generation. (c) Comparison with different multi-agent systems for code generation. (d) Collaboration Gain values across different models. Both (c) and (d) use Qwen2.5-7B-Coder-Instruct as the base model. More examples can be found in Appendix E.4 . To address this issue and improve the accuracy of code generation, prior studies have proposed the Chain-of-Thought (CoT) method ( 59 ) , which introduces intermediate reasoning steps or pseudocode before generation to enhance planning and reduce errors. However, CoT struggles with complex, multi-faceted tasks. To overcome this limitation, researchers have further explored multi-agent systems (MAS) based on large language models, where different agents (e.g., requirements engineers, programmers, and testers) collaborate to accomplish end-to-end software development workflows ( 22 ; 36 ) . However, we observe that existing multi-agent approaches often yield only marginal improvements in code generation performance as shown in Fig 1(c) . To understand this phenomenon, we analyze the role of collaboration in multi-agent systems. In typical frameworks, the final code is generated by a Coder Agent, while other agents provide intermediate guidance or feedback. We therefore ask: how much do these auxiliary agents actually help the Coder Agent? To quantify this effect, we introduce collaboration gain (CG) in Definition 1 , a metric that measures the effective contribution of auxiliary agents to the Coder Agent’s performance. As illustrated in Fig 1(d) , existing multi-agent approaches often fail to produce substantial collaboration gains, and in some cases may even lead to negative gains. This observation indicates that enhancing the collaboration capability among agents remains a critical challenge. To address these challenges, we propose CoRe-Code , a Co llaboration-aware Re inforcement learning framework for multi-agent Code generation, designed to enhance cooperation among agents during code synthesis. Existing studies on multi-agent reinforcement learning for code generation remain largely underexplored, leaving substantial room for exploring how RL can improve collaboration among specialized coding agents. We study this problem under the representative Planner–Coder paradigm ( 40 ; 19 ) , where a Planner agent produces a high-level solution plan and a Coder agent translates it into executable code. To make planning more concrete and actionable, we introduce algorithmic thoughts , a structured representation that decomposes algorithmic reasoning into input–output definition, linear progression, conditional logic, and iteration, inspired by prior work ( 31 ; 30 ; 8 ) . Nevertheless, directly optimizing the Planner is challenging, as intermediate plan quality is difficult to verify in isolation. Existing alternatives, such as reward model-based supervision ( 67 ; 70 ; 10 ) and off-policy training from collected planning trajectories ( 35 ; 46 ) , remain unreliable: the former may cause proxy-reward mismatch, where plan-level scores fail to align with final code correctness or executability ( 67 ; 12 ) , while the latter may suffer from distribution shift between collected plans and current Planner–Coder interaction dynamics. This motivates a verifiable optimization framework for training the Planner. We therefore use downstream code-execution results as verifiable rewards, following Reinforcement Learning with Verifiable Rewards (RLVR), and adopt Group Relative Policy Optimization (GRPO) ( 50 ) for policy optimization. Based on this design, we propose Collaborative GRPO , a collaboration-aware RL framework that trains the Planner and Coder with role-specific objectives while coordinating their learning through shared execution-based feedback. Consequently, the execution results of the Coder serve as indirect yet verifiable feedback for the Planner’s outputs. This enables the Planner to learn plans that better support downstream code synthesis, while encouraging the Coder to better align with and execute the generated plans. To the best of our knowledge, CoRe-Code represents one of the first verifier-guided RL-based multi-agent frameworks for code generation, leveraging verifiable execution feedback to improve collaboration among role-specialized agents. Furthermore, we experimentally show that CoRe-Code is not limited to the Planner–Coder paradigm. It can be further extended to optimize other agents, such as the Retrieval Agent and Debugging Agent, demonstrating its generality and flexibility across diverse multi-agent code generation frameworks. 2 Related work Due to space limitations, additional related work and preliminary knowledge of GRPO are presented in the Appendix D . 2.1 LLM-based code generation The automatic generation of code or completion of snippets from natural language using LLMs has gained much attention ( 17 ; 59 ; 74 ; 22 ; 25 ; 23 ; 36 ; 73 ; 71 ; 48 ; 18 ) , improving efficiency and reducing human error ( 20 ; 13 ) . Models such as GPT-4o ( 1 ) , ChatGLM ( 14 ) , CODEX ( 44 ) , Qwen ( 21 ) , DeepSeek ( 16 ) , and CodeGen ( 42 ) show strong code generation and understanding abilities, achieving SOTA results on MBPP ( 3 ) and HumanEval ( 7

Read the complete text