Assessing Correctness in LLM-Based Code Generation via Uncertainty Estimation

Arindam Sharma, Cristina David · 2025

Read Assessing Correctness in LLM-Based Code Generation via Uncertainty Estimation by Arindam Sharma, Cristina David (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11620.txt.

Description

Arindam Sharma and Cristina David used doubt scores to vet computer code in two thousand twenty-five. Language models now write code, but much of it is subtly wrong. Checking every output by hand defeats the purpose. The pair asked whether model uncertainty flags bad code. They adapted two doubt measures from text generation research. Code has strict meaning, so they added a logic check.

Their check compares code behavior using symbolic execution. Doubt scores lined up strongly with actual correctness. A simpler version of one measure worked nearly as well. It assumes the model's answers spread evenly by default. The team then built a refusal policy on the scores. When doubt runs high, the model stays silent. That policy cut wrong outputs sharply in their tests.

Catalog record

CREATORArindam Sharma, Cristina David
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-11620

Browse the full library · About the Tower of Babel