Subjective Code Preferences in Experts and Large Language Models

Anna Mokhova, Subhabrata Dutta, Iryna Gurevych, Simone Balloccu · 2026

Read Subjective Code Preferences in Experts and Large Language Models by Anna Mokhova, Subhabrata Dutta, Iryna Gurevych, Simone Balloccu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25296.txt.

Description

Anna Mokhova and colleagues study a neglected side of coding help: personal taste in code style. They define four taste axes, covering complexity, commenting, modularity, and readability, and gather about 3,000 paired Python snippets rated by 73 experts. When 13 language models judge the same task in words and then as code, many flip their preference. The steadiest models show a positional bias, favoring whichever option appears in a certain slot. Compared with people, model ratings skew polarized, and one leading model leans on outside assumptions and fragile reasoning.

Catalog record

CREATORAnna Mokhova, Subhabrata Dutta, Iryna Gurevych, Simone Balloccu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64144
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.25296

Browse the full library · About the Tower of Babel