Smaller Models, Unexpected Costs: Trade-offs in LLM Quantization for Automated Program Repair
Fernando Vallecillos-Ruiz 1 , Giordano d’Aloisio 2 , Max Hort 1 , Luca Traini 2 , Antinisca Di Marco 2 , Leon Moonen 1 · 2026
Read Smaller Models, Unexpected Costs: Trade-offs in LLM Quantization for Automated Program Repair by Fernando Vallecillos-Ruiz 1 , Giordano d’Aloisio 2 , Max Hort 1 , Luca Traini 2 , Antinisca Di Marco 2 , Leon Moonen 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-27205.txt.
Description
Vallecillos and colleagues test whether shrinking language models for bug fixing is as cheap as it looks. Quantization stores model weights at lower precision, slashing memory needs. Most reports judge it by benchmark scores alone, hiding behavioural shifts and overheads. Their test bed is automated program repair, a demanding software engineering task. Thirteen quantization setups, varying bit width, method, and target parts, run across six models and two repair benchmarks in this empirical study of 2026.
Shrunken and full models repair largely different problem sets, with little overlap, despite similar totals. Memory footprints fall by up to eighty-five percent of the original size in the best cases. Yet inference time and energy use both rise, blamed on poor hardware utilisation. Nearly half the setups are strictly dominated, beaten on every axis by some rival. No single method wins; architecture and task complexity steer the trade-offs.
Catalog record
| CREATOR | Fernando Vallecillos-Ruiz 1 , Giordano d’Aloisio 2 , Max Hort 1 , Luca Traini 2 , Antinisca Di Marco 2 , Leon Moonen 1 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.27205