ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation
Tencent Hunyuan Team · 2025
Read ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation by Tencent Hunyuan Team (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04952.txt.
Description
Language models now write web pages that move and respond, but standard coding tests only check whether the logic runs, not whether the result looks or feels right. Chenchen Zhang and co-authors built ArtifactsBench to close that gap. Their system renders each model's output, screenshots its behaviour over time, and hands the pictures plus the code to a multimodal judge model armed with a per-task checklist.
The benchmark holds 1,825 varied tasks, and more than thirty leading models were run through it. The automated rankings matched a popular human-preference arena ninety-four percent of the time, and agreed with human experts on over ninety percent of paired comparisons. Generalist models often beat coding specialists. Everything is open-sourced for the community.
Catalog record
| CREATOR | Tencent Hunyuan Team |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-04952