JMigBench: A Benchmark for Evaluating LLMs on Source Code Migration (Java 8 to Java 11) Conference: 1st Workshop on Code Translation, Transformation, and Modernization ; April 12–18, 2026; Rio de Janeiro, Brazil 1st Workshop on Code Translation, Transformation, and Modernization (ReCode ’26), April
Nishil Amin, Zhiwei Fei, Xiang Li, Justyna Petke, He Ye · 2026
Read JMigBench: A Benchmark for Evaluating LLMs on Source Code Migration (Java 8 to Java 11) Conference: 1st Workshop on Code Translation, Transformation, and Modernization ; April 12–18, 2026; Rio de Janeiro, Brazil 1st Workshop on Code Translation, Transformation, and Modernization (ReCode ’26), April by Nishil Amin, Zhiwei Fei, Xiang Li, Justyna Petke, He Ye (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-09930.txt.
Description
Upgrading old Java code by hand is slow, repetitive work, and language models promise to speed it up. A 2026 paper led by Nishil Amin, with four co-authors, builds JMigBench, a test for moving functions from Java 8 to Java 11. An early collection of function pairs proved too noisy, so the team rebuilt the set around eight groups of deprecated interfaces.
The team tested Mistral Codestral with measures of wording, meaning, and migration correctness. The model handled simple one-to-one swaps with middling success, matching the reference migration in about one case in nine, but stumbled on harder cases such as CORBA or JAX-WS. The verdict is partial help: the model can trim routine effort, but it cannot yet replace a human on this benchmark.
Catalog record
| CREATOR | Nishil Amin, Zhiwei Fei, Xiang Li, Justyna Petke, He Ye |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-09930