Debugging Without Error Messages
Audrey Salmon, Katie Hammer, Eddie Antonio Santos, Brett A. Becker · 2025
Read Debugging Without Error Messages by Audrey Salmon, Katie Hammer, Eddie Antonio Santos, Brett A. Becker (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-05706.txt.
Description
Salmon, Hammer, Santos, and Becker studied in twenty twenty-five how a language model explains programming errors when the compiler's own message is withheld. Error messages from compilers often confuse beginners with jargon or misleading hints. Models can write clearer explanations, but they usually receive the original message as context.
The team gave GPT-3.5 only the faulty code and asked for explanations. They tried several prompting strategies, including showing one example and fine-tuning, to improve the results. They report baseline scores for how well the explanations gave useful feedback. They also show how each strategy changed quality. The authors aim to help teachers judge how students may use AI tools when stuck on errors.
Catalog record
| CREATOR | Audrey Salmon, Katie Hammer, Eddie Antonio Santos, Brett A. Becker |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-05706