SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale
Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, Alexander Golubev · 2026
Read SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale by Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, Alexander Golubev (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-23866.txt.
Description
Ibragim Badertdinov and three co-authors introduce SWE-rebench V2, a language-agnostic pipeline for harvesting real software engineering tasks to train coding agents. The 2026 paper addresses a bottleneck. Reinforcement learning for such agents needs many tasks with reproducible setups and trustworthy tests, which existing collections supply only in limited scale and language range.
The pipeline builds repository-specific install and test steps with an interactive setup agent, then filters unsound tasks with an ensemble of language-model judges checked against human-verified annotations. It yields tens of thousands of tasks across many languages and thousands of repositories, with prebuilt images for repeatable runs, plus a larger set with install steps, fail-to-pass tests, and metadata. A diagnostic study across languages and models validates the collection and flags confounders such as overly strict tests.
Catalog record
| CREATOR | Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, Alexander Golubev |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-23866