When Prompt Under-Specification Improves Code Correctness: An Exploratory Study of Prompt Wording and Structure Effects on LLM-Based Code Generation

Amal Akli, Mike Papadakis, Maxime Cordy, Yves Le Traon · 2026

Read When Prompt Under-Specification Improves Code Correctness: An Exploratory Study of Prompt Wording and Structure Effects on LLM-Based Code Generation by Amal Akli, Mike Papadakis, Maxime Cordy, Yves Le Traon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-24712.txt.

Description

Akli, Papadakis, Cordy and Le Traon ask when vague prompts help rather than hurt code generation. Earlier work on minimal benchmarks found that under-specification sharply cuts correctness. The authors test ten models on HumanEval and the richer LiveCodeBench. On LiveCodeBench, the same vague-prompt changes have near-zero net effect, because redundancy across descriptions, constraints, examples and input-output conventions cushions them. Surprisingly, some prompt changes improve correctness on the richer benchmark.

Catalog record

CREATORAmal Akli, Mike Papadakis, Maxime Cordy, Yves Le Traon
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53792
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.24712

Browse the full library · About the Tower of Babel