CodeClash: Benchmarking Goal-Oriented Software Engineering
John Yang* 1, Kilian Lieret* 2, Joyce Yang 3, Carlos E. Jimenez 2, 2 Princeton University, 3 Cornell University · 2025
Read CodeClash: Benchmarking Goal-Oriented Software Engineering by John Yang* 1, Kilian Lieret* 2, Joyce Yang 3, Carlos E. Jimenez 2, 2 Princeton University, 3 Cornell University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00839.txt.
Introduction
John Yang and colleagues introduce CodeClash, a benchmark that tests language models the way real software work happens: chasing broad goals over many rounds, not fixing one named bug. Models compete in tournaments, editing their code in one phase and then facing rival codebases head to head in arenas scored on goals like resource gain or survival.
Across many tournaments with eight models in six arenas, the team finds wide differences in development style but shared weaknesses. Models reason poorly about long term strategy, and their codebases grow messy and redundant over time. The results suggest current models handle isolated coding tasks far better than the open ended, self directed upkeep that real projects demand.
Catalog record
| CREATOR | John Yang* 1, Kilian Lieret* 2, Joyce Yang 3, Carlos E. Jimenez 2, 2 Princeton University, 3 Cornell University |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 168 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-00839