Debugging Without Error Messages

Audrey Salmon, Katie Hammer, Eddie Antonio Santos, Brett A. Becker · 2025

Read Debugging Without Error Messages by Audrey Salmon, Katie Hammer, Eddie Antonio Santos, Brett A. Becker (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-05706.txt.

Description

Salmon, Hammer, Santos, and Becker studied in twenty twenty-five how a language model explains programming errors when the compiler's own message is withheld. Error messages from compilers often confuse beginners with jargon or misleading hints. Models can write clearer explanations, but they usually receive the original message as context.

The team gave GPT-3.5 only the faulty code and asked for explanations. They tried several prompting strategies, including showing one example and fine-tuning, to improve the results. They report baseline scores for how well the explanations gave useful feedback. They also show how each strategy changed quality. The authors aim to help teachers judge how students may use AI tools when stuck on errors.

Catalog record

CREATORAudrey Salmon, Katie Hammer, Eddie Antonio Santos, Brett A. Becker
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-05706

Browse the full library · About the Tower of Babel