SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale

Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, Alexander Golubev · 2026

Read SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale by Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, Alexander Golubev (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23866.txt.

Description

Ibragim Badertdinov and three co-authors introduce SWE-rebench V2, a language-agnostic pipeline for harvesting real software engineering tasks to train coding agents. The 2026 paper addresses a bottleneck. Reinforcement learning for such agents needs many tasks with reproducible setups and trustworthy tests, which existing collections supply only in limited scale and language range.

The pipeline builds repository-specific install and test steps with an interactive setup agent, then filters unsound tasks with an ensemble of language-model judges checked against human-verified annotations. It yields tens of thousands of tasks across many languages and thousands of repositories, with prebuilt images for repeatable runs, plus a larger set with install steps, fail-to-pass tests, and metadata. A diagnostic study across languages and models validates the collection and flags confounders such as overly strict tests.

Catalog record

CREATORIbragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, Alexander Golubev
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-23866

Browse the full library · About the Tower of Babel