JMigBench: A Benchmark for Evaluating LLMs on Source Code Migration (Java 8 to Java 11) Conference: 1st Workshop on Code Translation, Transformation, and Modernization ; April 12–18, 2026; Rio de Janeiro, Brazil 1st Workshop on Code Translation, Transformation, and Modernization (ReCode ’26), April

Nishil Amin, Zhiwei Fei, Xiang Li, Justyna Petke, He Ye · 2026

Read JMigBench: A Benchmark for Evaluating LLMs on Source Code Migration (Java 8 to Java 11) Conference: 1st Workshop on Code Translation, Transformation, and Modernization ; April 12–18, 2026; Rio de Janeiro, Brazil 1st Workshop on Code Translation, Transformation, and Modernization (ReCode ’26), April by Nishil Amin, Zhiwei Fei, Xiang Li, Justyna Petke, He Ye (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-09930.txt.

Description

Upgrading old Java code by hand is slow, repetitive work, and language models promise to speed it up. A 2026 paper led by Nishil Amin, with four co-authors, builds JMigBench, a test for moving functions from Java 8 to Java 11. An early collection of function pairs proved too noisy, so the team rebuilt the set around eight groups of deprecated interfaces.

The team tested Mistral Codestral with measures of wording, meaning, and migration correctness. The model handled simple one-to-one swaps with middling success, matching the reference migration in about one case in nine, but stumbled on harder cases such as CORBA or JAX-WS. The verdict is partial help: the model can trim routine effort, but it cannot yet replace a human on this benchmark.

Catalog record

CREATORNishil Amin, Zhiwei Fei, Xiang Li, Justyna Petke, He Ye
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-09930

Browse the full library · About the Tower of Babel