CodeArena: A Collective Evaluation Platform for LLM Code Generation
Mingzhe Du, Anh Tuan Luu, Bin Ji, Xiaobao Wu, Dong Huang, Terry Yue Zhuo, Qian Liu, See-Kiong Ng · 2025
Read CodeArena: A Collective Evaluation Platform for LLM Code Generation by Mingzhe Du, Anh Tuan Luu, Bin Ji, Xiaobao Wu, Dong Huang, Terry Yue Zhuo, Qian Liu, See-Kiong Ng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01295.txt.
Description
Du leads a team introducing CodeArena, a shared platform for judging code-generating language models, in a 2025 paper. Static benchmarks leak into training data and scores drift. Many evaluation systems are also closed or hard to automate.
CodeArena scores each model against the whole field at once, recalibrating as models perform. That collective scoring aims to cancel biases from leaked benchmarks. All submitted solutions and test cases stay openly available. Automation-friendly interfaces plug evaluation into developer workflows. The authors present openness as the guard against stale scores. The platform targets timely, accurate comparison of coding models. It is offered as community infrastructure for the field.
Catalog record
| CREATOR | Mingzhe Du, Anh Tuan Luu, Bin Ji, Xiaobao Wu, Dong Huang, Terry Yue Zhuo, Qian Liu, See-Kiong Ng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01295