Task Abstention for Large Language Models in Code Generation

2026

Read Task Abstention for Large Language Models in Code Generation (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-17029.txt.

Description

Zhou's team studies task abstention for code-generating language models: deciding when a model should refuse a task to avoid error. Their calibrated rule, grounded in multiple hypothesis testing, judges output consistency through code run outcomes, handling syntax variety among same-meaning programs without oracle test cases or outside databases. They prove a rigorous, distribution-free guarantee on refusal calls. Benchmarks with several open-source code models show their method spots and refuses error-prone tasks more accurately and efficiently than current techniques, giving safer, more robust code generation.

Catalog record

YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50011
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.17029

Browse the full library · About the Tower of Babel