UniCode: A Framework for Generating High-Quality Competitive Coding Problems

Xinyue Zheng, Haowei Lin, Shaofei Cai, Zilong Zheng, Yitao Liang · 2025

Read UniCode: A Framework for Generating High-Quality Competitive Coding Problems by Xinyue Zheng, Haowei Lin, Shaofei Cai, Zilong Zheng, Yitao Liang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17868.txt.

Description

Xinyue Zheng and five co-authors argue that coding tests for large language models often flatter the models. Fixed test problems leak into training data, and models can pass by memory rather than reason. Their UniCode system turns seed problems into harder changed versions and builds tests for them on its own.

Scores of top models fell sharply on the changed problems, mostly from weak planning and scaling, not from syntax slips. Models also slid back to memorized answers for the seed problems. Readers who judge coding models get a sterner test and a warning about how much reported skill is recall.

Catalog record

CREATORXinyue Zheng, Haowei Lin, Shaofei Cai, Zilong Zheng, Yitao Liang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-17868

Browse the full library · About the Tower of Babel