Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revision

Hong Yi Lin, Chunhua Liu, Haoyu Gao, Patanamon Thongtanunam, Christoph Treude · 2026

Read Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revision by Hong Yi Lin, Chunhua Liu, Haoyu Gao, Patanamon Thongtanunam, Christoph Treude (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-06723.txt.

Description

Lin and four colleagues calibrate confidence for language models revising code. Developers lean on capable yet imperfect models, and calibrated confidence guides accept-or-reject calls. Global Platt-scaling of sequence-level scores works in many generative tasks but proves shaky for code revision, where correctness hinges on local edits and miscalibration varies by sample. Their fine-grained approach applies local Platt-scaling to three fine-grained scores. Across three tasks and 14 models, fine-grained scores cut calibration error across more intervals, with global scaling on top amplifying the gain.

Catalog record

CREATORHong Yi Lin, Chunhua Liu, Haoyu Gao, Patanamon Thongtanunam, Christoph Treude
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE104671
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.06723

Browse the full library · About the Tower of Babel