CANVAS : A Benchmark for Vision-Language Models on Tool-Based User Interface Design
Daeheon Jeong, Seoyeon Byun, Kihoon Son, Dae Hyun Kim, Juho Kim · 2025
Read CANVAS : A Benchmark for Vision-Language Models on Tool-Based User Interface Design by Daeheon Jeong, Seoyeon Byun, Kihoon Son, Dae Hyun Kim, Juho Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-20737.txt.
Description
Daeheon Jeong and colleagues introduce in 2025 CANVAS, the first benchmark for whether vision-language models can do interface design by actually operating design software. Designers refine screens step by step in tools like Figma, and new models can invoke such tools directly, but existing benchmarks only grade generated code, not tool use. In CANVAS the model works through tool calls, such as creating a rectangle as a button background, inside design software.
The benchmark holds 598 tasks drawn from 3,327 human-made mobile interfaces across 30 function categories, from onboarding to messaging. One task type is replication, rebuilding a whole screen from a reference image. The other is modification, changing one part of an existing screen. Models are scored on structural, compositional, and semantic similarity plus component-level attributes. Leading models issued more strategic tool calls and produced better designs, and the authors catalogue the common error patterns the rest fell into.
Catalog record
| CREATOR | Daeheon Jeong, Seoyeon Byun, Kihoon Son, Dae Hyun Kim, Juho Kim |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 116 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-20737