DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution

Shengda Fan, Xuyan Ye, Yankai Lin · 2026

Read DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution by Shengda Fan, Xuyan Ye, Yankai Lin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-13761.txt.

Introduction

Shengda Fan, Xuyan Ye, and Yankai Lin work on AI systems that improve themselves through self-play in 2026. In self-play, one model poses questions and an other solves them. Past setups face two flaws. The questioner's goals shift as the solver's feedback changes. And the solver learns from its own pseudo-labels, which can bake in errors.

Their method, DARC, splits the training into a decoupled, asymmetric curriculum. Separate paths for questioner and solver aim to calm the training. Stable self-play could let models grow with less human data. That path draws much hope, and much need for care.

Catalog record

CREATORShengda Fan, Xuyan Ye, Yankai Lin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-13761

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution Shengda Fan, Xuyan Ye, Yankai Lin Abstract Self-play with large language models has emerged as a promising paradigm for achieving self-improving artificial intelligence. However, existing self-play frameworks often suffer from optimization instability, due to (i) non-stationary objectives induced by solver-dependent reward feedback for the Questioner , and (ii) bootstrapping errors from self-generated pseudo-labels used to supervise the Solver . To mitigate these challenges, we introduce DARC ( D ecoupled A symmetric R easoning C urriculum), a two-stage framework that stabilizes the self-evolution process. First, we train the Questioner to synthesize difficulty-calibrated questions, conditioned on explicit difficulty levels and external corpora. Second, we train the Solver with an asymmetric self-distillation mechanism, where a document-augmented teacher generates high-quality pseudo-labels to supervise the student Solver that lacks document access. Empirical results demonstrate that DARC is model-agnostic, yielding an average improvement of 10.9 points across nine reasoning benchmarks and three backbone models. Moreover, DARC consistently outperforms all baselines and approaches the performance of fully supervised models without relying on human annotations. The code is available at https://github.com/RUCBM/DARC . 1 Introduction Self-improving artificial intelligence, which enables models to autonomously refine their capabilities without human intervention, is widely viewed as an important step toward more general and potentially superhuman intelligence 26 . While large language models (LLMs) have demonstrated remarkable progress in complex reasoning tasks 20 ; 18 , their success largely relies on extensive human supervision. However, as the availability of high-quality human-annotated data approaches its limit, this scarcity becomes a fundamental bottleneck for further scaling. Consequently, developing reliable and efficient self-evolution mechanisms that operate independently of human data has emerged as a pivotal research frontier. Figure 1: Comparison between DARC and previous coupled self-play methods. A widely adopted approach for LLM self-evolution is self-play, which establishes a co-evolutionary loop between a Questioner that proposes tasks and a Solver that attempts to solve them 29 ; 30 . In this paradigm, the Questioner optimizes a difficulty-calibration reward based on the Solver ’s current performance, while the Solver is updated on pseudo-labeled data constructed from the tasks generated by the Questioner . Despite its appeal, such tightly coupled systems are prone to premature plateaus or even performance collapse ( 10 ) . We argue that this instability stems from two inherent issues in the coupled design: non-stationary optimization targets and self-confirmation bias . As shown in the upper part of Figure 1 , the Questioner is continually optimized against a moving target determined by the evolving Solver . Its learned difficulty signal can become stale or even misaligned after each Solver update, inducing oscillatory shifts in task difficulty and unstable optimization dynamics. In parallel, the Solver is trained on self-generated pseudo-labels that are inevitably noisy, so errors can be propagated and amplified across iterations. To address the coupled instability in self-play, we propose D ecoupled A symmetric R easoning C urriculum ( DARC ), which restructures self-evolution into two decoupled and sequential stages: Questioner training and Solver training, as illustrated in the lower part of Figure 1 . In Stage 1, instead of chasing a moving Solver boundary, we train the Questioner with objective supervision from explicit difficulty levels and external documents, enabling it to generate corpus-grounded questions that are calibrated to a specified difficulty. This decoupling removes the dependence of question generation on the Solver ’s real-time performance, thereby mitigating non-stationary optimization targets. In Stage 2, we replace self-training on noisy pseudo-labels with an asymmetric self-distillation scheme. Concretely, we introduce a privileged teacher Solver that has access to the source document and performs majority voting to construct pseudo-labels, and distill its outputs into a student Solver that receives only the question as input. This asymmetric design (document-augmented teacher vs. question-only student) reduces label noise and alleviates self-confirmation bias, yielding a more stable and reliable learning signal for Solver improvement. Our experiments demonstrate that DARC is model-agnostic, consistently improving the reasoning abilities of both Qwen-based 35 and LLaMA-based 33 backbones. Notably, applying DARC improves the average accuracy by 10.9 points over the base models, outperforming label-free self-evolving methods R-Zero 10 and Absolute-Zero 39 , corpus-grounded self-play method SPICE 17 and weakly supervised self-play methods R-Few 36 . Notably, DARC approaches the performance of General-Reasoner 21 , which is trained on the full 232K WebInstruct dataset 37 , despite using no human annotations. Beyond empirical gains, we further demonstrate several key properties of DARC : (i) it does not merely memorize the training corpus; (ii) the difficulty rankings are solver-independent; and (iii) the decoupled Questioner learns a curriculum improving the performance of heterogeneous Solver backbones. 2 Related Work Reinforcement Learning for LLM Reasoning. Reinforcement learning (RL) is pivotal for advancing LLM reasoning, but a key bottleneck lies in how to obtain reliable supervision signals. A primary paradigm employs external verifiers ( 7 ) to provide rule-based rewards, but this approach is largely restricted to deterministic fields like mathematics and code. Alternatively, self-supervision methods derive signals via maximizing confidence ( 24 ) or getting pseudo-labels by majority voting ( 40 ) , yet they remain dependent on curated question sets. In contrast, DARC transcends these limitations by operating on raw corpora, generating both tasks and supervision signals in a self-evolving manner. Self-Play for LLM Self-Evolution. Self-play is a widely adopted paradigm for enabling self-evolution in LLMs 3 . One line of research focuses on code-centric self-play, where execution results provide grounded supervision and enable stable co-evolution between a coder and a verifier ( 39 ; 31 ; 15 ) . A parallel line extends self-play to general reasoning via a questioner–solver framework, in which a Questioner proposes tasks and a Solver learns from self-generated supervision ( 10 ; 17 ; 2 ) . Unlike code-centric settings that can rely on executable feedback for supervision, general-reasoning self-play often suffers from non-stationary rewards and unstable optimization. In contrast, our approach stabilizes self-evolving optimization by decoupling training and introducing asymmetric self-distillation. Data Synthesis for LLM Training. Synthetic data generation is pivotal for scaling reasoning supervision. Bootstrapping frameworks, including STaR ( 38 ) , ReST ( 6 ) , and WizardLM 34 , expand datasets by iteratively evolving instructions from seed exemplars. Meanwhile, corpus-driven pipelines like WebInstruct ( 37 ) and General-Reasoner ( 21 ) synthesize question-answer pairs by leveraging web mining or model priors to broaden domain coverage. While these approaches effectively scale data quantity or diversity, they often rely on superior external models (e.g., GPT-4o, Gemini) to guarantee data quality. In contrast, DARC dispenses with external teacher models by deriving both task difficulty and supervision signals internally, enabling stable self-evolution directly from unlabeled corpora. 3 Methodology Figure 2: Illustration of the two-stage DARC framework. In the first stage (the upper half), the Questioner learns to generate questions matching specified difficulty \tau with a difficulty-anchored reward. In the second stage (the lower half), the Solver is trained on an offline curriculum with an answer correctness reward. In this section, we present the DARC framework, which decomposes LLM self-evolution into two sequential stages. In the first stage (Section 3.1 ), the Questioner is trained for controllable question generation based on explicit difficulty levels and an external corpus. In the second stage (Section 3.2 ),

Read the complete text