OpenCodeReasoning: Advancing Data Distillation for Competitive Coding

Wasi Uddin Ahmad, Sean Narenthiran, Somshubra Majumdar, Aleksander Ficek · 2025

Read OpenCodeReasoning: Advancing Data Distillation for Competitive Coding by Wasi Uddin Ahmad, Sean Narenthiran, Somshubra Majumdar, Aleksander Ficek (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-01943.txt.

Description

Wasi Uddin Ahmad and co-authors published this machine learning paper in 2025. Reasoning models have transformed competitive coding, and a popular recipe distills their skill into smaller student models. Much of that progress hides in private datasets. The team instead built a large, open supervised fine-tuning dataset for coding reasoning, and trained student models of several sizes on it alone.

Their distilled models scored 61.8 percent on LiveCodeBench and 24.6 percent on CodeContests, beating alternatives trained with reinforcement learning. The analysis held surprises. Filtering training data by running the code and keeping only correct solutions actually hurt accuracy. Diversity of instructions mattered more than verified correctness. The authors also studied how efficiently the students use tokens and what reasoning patterns they adopt. Datasets and models are being released openly.

Catalog record

CREATORWasi Uddin Ahmad, Sean Narenthiran, Somshubra Majumdar, Aleksander Ficek
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-01943

Browse the full library · About the Tower of Babel