GASP: Guided Asymmetric Self-Play For Coding LLMs

Swadesh Jana Cansu Sancaktar Tomáš Daniš Georg Martius, Antonio Orvieto Pavel Kolev · 2026

Read GASP: Guided Asymmetric Self-Play For Coding LLMs by Swadesh Jana Cansu Sancaktar Tomáš Daniš Georg Martius, Antonio Orvieto Pavel Kolev (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15957.txt.

Introduction

Jana and co-authors ground coding self-play with real hard questions as goalposts. Teacher-student self-play can drift toward hard but useless problems. Their GASP method anchors training on real-data goalpost questions. The teacher first builds an easier variant of a goalpost. It then builds a harder variant of that easier form. The ladder slowly closes the gap to the goalpost. Pass-at-20 on LiveCodeBench rose 2.5 percent over unguided play. Some goalpost questions fall that no baseline could solve.

Catalog record

CREATORSwadesh Jana Cansu Sancaktar Tomáš Daniš Georg Martius, Antonio Orvieto Pavel Kolev
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55161
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.15957

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GASP: Guided Asymmetric Self-Play For Coding LLMs Swadesh Jana Cansu Sancaktar Tomáš Daniš Georg Martius, Antonio Orvieto Pavel Kolev Abstract Asymmetric self-play has emerged as a promising paradigm for post-training large language models, where a teacher continually generates questions for a student to solve at the edge of the student’s learnability. Although these methods promise open-ended data generation bootstrapped from no human data, they suffer from one major problem: not all problems that are hard to solve are interesting or informative to improve the overall capabilities of the model. Current asymmetric self-play methods are goal-agnostic with no real grounding. We propose G uided A symmetric S elf- P lay (GASP), where grounding is provided by real-data goalpost questions that are identified to pose a hard exploration challenge to the model. During self-play, the teacher first generates an easier variant of a hard question, and then a harder variant of that easier question, with the goal of gradually closing the gap to the goalpost throughout training. Doing so, we improve pass@20 on LiveCodeBench (LCB) by 2.5% over unguided asymmetric self-play, and through the curriculum constructed by the teacher, we manage to solve hard goalpost questions that remain out of reach for all baselines. Figure 1: Overview of GASP . Self-play is guided by hard real-data coding questions that a standard RLVR run fails to solve. We refer to this subset as our goalpost questions. The teacher first generates an easy variant ( lemma ) of this goalpost and then a harder variant ( lift ), producing progressively more challenging questions that push the model’s knowledge boundary. 1 Introduction Asymmetric self-play aims to keep pushing the student model’s abilities by generating problems at the frontier of learning. While such methods have recently shown promise for large language model (LLM) post-training ( Zhao et al. 2025 ) , they quickly run into a fundamental bottleneck: not all hard problems at the agent’s frontier are interesting or help unlock experiences that improve generalization on the downstream tasks we actually care about. We argue that the teacher needs explicit guidance, i.e. some distant goalpost, to better ground the interestingness of the problems it generates. In the prevailing reinforcement learning (RL) post-training paradigm, problems are typically sampled uniformly at random or from a fixed curriculum defined over a static dataset in a given environment ( Moonshot AI 2025 ) . In practice, a non-trivial portion of this dataset remains consistently unsolved because it poses a hard exploration challenge. We hypothesize that these genuinely hard, real-data questions, which are known to be difficult yet relevant, form ideal goalposts for guided self-play. In this work, we set out to investigate: 1. Can we design a guided self-play algorithm that makes meaningful progress on a designated set of hard questions during RL training? In other words, does steering self-play toward these very hard problems as goalposts actually work? 2. Does such guidance lead the teacher to propose better (i.e., more relevant and informative) problems overall, thereby improving performance on downstream tasks? We propose G uided A symmetric S elf- P lay ( GASP ), where the teacher is guided by a set of hard real-data questions \mathcal{H} , namely our goalposts. Given such a target h\in\mathcal{H} , the teacher is prompted to generate an easier instance \ell_{0} , which we refer to as the lemma , that aims to preserve the high-level motif of h . We accept a lemma only if it is learnable but non-trivial for the student, ensuring that it still provides a clear learning signal unlike the goalpost. We then prompt the teacher to generate a harder instance \ell_{1} using \ell_{0} as a scaffold; we refer to \ell_{1} as the lift question. In this way, \ell_{0} and \ell_{1} serve as stepping stones that build a curriculum. Our ultimate goal is to generate questions at the student’s learnability frontier that remain relevant and useful for improving the model’s coding capabilities, measured through performance on well-known benchmarks. We evaluate GASP on LiveCodeBench (LCB) and show that goalpost guidance improves downstream coding performance. In particular, GASP consistently outperforms unguided asymmetric self-play ( AZR ) and is competitive with standard reinforcement learning with verifiable rewards (RLVR) trained on real-data, with improvements most pronounced at larger k . Beyond benchmark gains, GASP makes progress on the goalpost set itself: as training proceeds, some goalpost questions in \mathcal{H} that remain unsolved by all baselines become solvable. 2 Related Work Asymmetric self-play. Asymmetric self-play has been explored in games ( Sukhbaatar et al. 2017 ) and robotics ( OpenAI et al. 2021 ) . More recently, this paradigm has also been applied to RL post-training with LLMs. Poesia et al. 2024 and Dong & Ma 2025 use teacher–student training for theorem proving. Ye et al. 2024 applies similar ideas for alignment, and Liu et al. 2025 and Zhang et al. 2025 apply self-play to math and general reasoning. Most notably and closest to our setting, Absolute Zero ( AZR ) employs asymmetric self-play in the coding domain. Kuba et al. 2025 also tackles asymmetric self-play in instruction-following, math and coding domains, and Teodorescu et al. 2023 is among early works proposing self-play for coding puzzles, though they only train the teacher. Recent work by Yu et al. 2025 also proposes guided self-play for mathematical and general reasoning. However, their grounding relies on few-shot anchoring to a small labeled dataset, rather than steering toward a designated hard set of real-data goalposts as in GASP . Finally, Wei et al. 2025 proposes self-play for software agents, where an LLM is trained to inject and repair software bugs of increasing complexity in real-world codebases, providing an alternative form of grounding. Unsupervised environment design and automated curricula. A related line of work studies how to automatically generate and schedule training tasks given a parameterized task space. Parker-Holder et al. 2022 propose ACCEL , which mutates previously generated levels of the environment and prioritizes levels with high student regret, i.e. a gap between the current agent and an optimal agent, to push learning in grid-like game environments. Rutherford et al. 2024 show that commonly used regret approximations can correlate more with success rate than true regret, and propose alternative signals based on student learnability. This perspective is closely related to the teacher objective used in GASP , which encourages generating questions that are challenging yet solvable. Meta-learning. Sundaram et al. 2026 propose SOAR , a meta-learning approach that targets hard exploration in RLVR by grounding teacher training in a hard real-data subset in the math domain. At each iteration, the teacher generates synthetic questions for the student, and is rewarded based on the student’s measured improvement on the hard subset (used only through this improvement signal). In contrast, GASP does not reward the teacher directly for improving on goalpost questions; solving goalposts is instead a byproduct of learning from lemma–lift stepping stones. A practical limitation of the meta-learning objective is that the improvement-based reward can be sparse or noisy when target questions are extremely hard, and it requires repeated inner-loop training and evaluation to obtain the teacher signal. 3 Method We exclusively work in the coding domain, and focus on LiveCodeBench (LCB) ( Jain et al. 2024

Read the complete text