CodeMEnv : Benchmarking Large Language Models on Code Migration
Keyuan Cheng, Xudong Shen, Yihao Yang, Tengyue Wang, Yang Cao, Muhammad Asif Ali, Hanbin Wang, Lijie Hu, Di Wang · 2025
Read CodeMEnv : Benchmarking Large Language Models on Code Migration by Keyuan Cheng, Xudong Shen, Yihao Yang, Tengyue Wang, Yang Cao, Muhammad Asif Ali, Hanbin Wang, Lijie Hu, Di Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00894.txt.
Description
Keyuan Cheng and eight co-authors introduce a 2025 test set for judging how well language models move code between software settings. Moving code so it runs under a new version or setup is common work, yet it had no shared yardstick. Their test set, CodeMEnv, fills that gap.
CodeMEnv holds 922 tasks drawn from 19 Python and Java packages. It checks three skills: spotting functions that clash with a set version, noting changes in how a function is defined, and rewriting code for the target setup. Seven models scored a mean pass rate near 27 percent, with the best near 44 percent.
Catalog record
| CREATOR | Keyuan Cheng, Xudong Shen, Yihao Yang, Tengyue Wang, Yang Cao, Muhammad Asif Ali, Hanbin Wang, Lijie Hu, Di Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00894