Idea First, Code Later: Disentangling Problem Solving from Code Generation in Evaluating LLMs for Competitive Programming
Sama Hadhoud Alaa Elsetohy Frederikus Hudi, [0.2em] Jan Christian Blaise Cruz Steven Halim Alham Fikri Aji, [0.6em] MBZUAI · 2026
Read Idea First, Code Later: Disentangling Problem Solving from Code Generation in Evaluating LLMs for Competitive Programming by Sama Hadhoud Alaa Elsetohy Frederikus Hudi, [0.2em] Jan Christian Blaise Cruz Steven Halim Alham Fikri Aji, [0.6em] MBZUAI (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-11332.txt.
Introduction
Sama Hadhoud and six coauthors split problem solving from code writing. Their 2026 preprint tests language models on competitive programming. Models now solve many such problems end to end. But a single score hides two unlike skills in the score.
One skill is the plan: the algorithm that cracks the task. The other is the code: turning the plan into a program. The team centers plain-language editorials in the test. Editorials state the plan in words, with no code at all. Solving and grading both pass through that plan layer. That split shows if a model fails on thought or on syntax. Contest-style tasks are at heart problem-solving tasks, they argue.
Catalog record
| CREATOR | Sama Hadhoud Alaa Elsetohy Frederikus Hudi, [0.2em] Jan Christian Blaise Cruz Steven Halim Alham Fikri Aji, [0.6em] MBZUAI |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 122 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-11332