How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning

William Orwig, Roger E. Beaty · 2026

Read How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning by William Orwig, Roger E. Beaty (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29672.txt.

Description

Orwig and Beaty ask whether multimodal large language models can judge visual creativity with no training on human ratings. Scoring the originality of images is a long-standing challenge. Automated scoring works in the verbal domain, yet visual evaluation remains an open question.

Six multimodal models rate nearly one thousand AI-generated images and fifteen hundred hand-drawn sketches, all previously scored by human raters. Every model aligns substantially with human ratings on both datasets in the zero-shot setting, meaning no examples or fine-tuning were given. A second study analyses the step-by-step reasoning of three models on the same material. That reasoning shows what models attend to and how they balance originality against quality. Reasoning, however, does not improve alignment with human ratings. The authors release an open scoring app implementing the pipeline.

Catalog record

CREATORWilliam Orwig, Roger E. Beaty
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.29672

Browse the full library · About the Tower of Babel