Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification

Zehai He, Wenyi Hong, Zhen Yang, Ziyang Pan, Mingdao Liu, Xiaotao Gu, Jie Tang · 2026

Read Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification by Zehai He, Wenyi Hong, Zhen Yang, Ziyang Pan, Mingdao Liu, Xiaotao Gu, Jie Tang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26648.txt.

Introduction

He and co-authors introduce Vision2Web, a benchmark for building websites from visual designs with coding agents. The tasks scale from static page copying to interactive multi-page sites and full-stack builds. The set holds 193 tasks in 16 groups, drawn from real websites, with 918 prototype images and 1,255 test cases. Scoring pairs a GUI agent verifier with a vision-language judge under a workflow-based scheme. Even leading models show wide gaps at every level, and full-stack work still defeats them often.

Catalog record

CREATORZehai He, Wenyi Hong, Zhen Yang, Ziyang Pan, Mingdao Liu, Xiaotao Gu, Jie Tang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56548
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.26648

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification Zehai He, Wenyi Hong, Zhen Yang, Ziyang Pan, Mingdao Liu, Xiaotao Gu, Jie Tang Abstract Recent advances in large language models have improved the capabilities of coding agents, yet systematic evaluation of complex, end-to-end website development remains limited. To address this gap, we introduce Vision2Web, a hierarchical benchmark for visual website development, spanning from static UI-to-code generation, interactive multi-page frontend reproduction, to long-horizon full-stack website development. The benchmark is constructed from real-world websites and comprises a total of 193 tasks across 16 categories, with 918 prototype images and 1,255 test cases. To support flexible, thorough and reliable evaluation, we propose workflow-based agent verification paradigm based on two complementary components: a GUI agent verifier and a VLM-based judge. We evaluate multiple visual language models instantiated under different coding-agent frameworks, revealing substantial performance gaps at all task levels, with state-of-the-art models still struggling on full-stack development. Project page: https://vision2web-bench.github.io/ . \icmlauthornotes ∗ Equal contribution † Corresponding author. Work was done when ZH, WH, ZY, ML interned at Zhipu AI. Figure 1 : Overview of Vision2Web, a hierarchical benchmark for visual website development. Tasks span three levels—static webpages, interactive frontends, and full-stack websites—requiring agents to integrate visual prototypes with textual specifications. Evaluation is performed via a workflow-based agent verification paradigm, measuring functional correctness and visual fidelity. 1 Introduction The reasoning and coding capabilities of autonomous software agents have been significantly advanced through the development of large language models (LLMs) ( Dong et al. 2025 ) . Equipped with these models, contemporary coding agents are capable of performing end-to-end software development tasks encompassing system design, data processing, code generation, and project debugging. Despite these advances, existing evaluation benchmarks for coding agents remain fundamentally limited in scope and rigor: • Limited task formulation. Prominent benchmarks such as SWE Bench and its variants ( Jimenez et al. 2023 ; Yang et al. 2024 ) focus on incremental, issue-driven code edits, capturing localized development skills but failing to evaluate holistic, end-to-end software engineering capabilities. • Misaligned multimodal coverage. While recent text-only benchmarks, including VIBE Bench ( MiniMax 2025 ) and WebGen Bench ( Lu et al. 2025 ) , have begun exploring end-to-end development scenarios, multimodal benchmarks remain largely restricted to static webpage reproduction tasks such as Design2Code ( Si et al. 2025 ) . • Insufficient verification mechanisms. Despite initial attempts toward end-to-end development ( Lu et al. 2025 ) , reliably and reproducibly assessing complex interactions and long-horizon system outcomes remains challenging, due to underspecified task definitions and insufficiently constrained verification procedures. Table 1: Comparison of Existing Benchmarks for Software Engineer and Web Development Tasks. Benchmark # Tasks # Test Cases # Prototypes # Multimodal Task Types SWE-Bench Multimodal 617 - - ✓ Issue fixing Design2Code 484 - 484 ✓ Single page UI-to-Code WebGenBench 101 647 - ✗ Website generation Vision2Web 193 1255 918 ✓ Full-stack website development To address these gaps, we introduce Vision2Web , a hierarchical benchmark that enables autonomous evaluation of multimodal coding agents on visual website development via agent verification. As a task formulation, website development naturally satisfies these requirements: it spans the full software lifecycle and requires coordinated understanding of visual prototypes, textual requirements, and codebases, making it an ideal testbed for evaluating long-horizon multimodal agent intelligence. Overall, Vision2Web is designed around three core principles: Capability disentanglement. To enable explicit failure attribution across development stages, Vision2Web organizes tasks into three progressively harder levels— static webpage generation , interactive frontend development , and full-stack website construction —with each level building upon the previous one, enabling systematic diagnosis of agent capabilities from fine-grained visual understanding to holistic system construction. Verifiable task construction. Rather than relying on underspecified synthetic tasks, Vision2Web is curated from publicly accessible websites through a rigorous multi-stage pipeline that integrates large-scale data collection, automated filtering and agent-assisted annotation. The resulting benchmark comprises 193 website development tasks with 1255 test cases spanning four major website categories and 16 subcategories, closely reflecting the diversity of real-world websites. Reliable automated evaluation. Automated assessment of end-to-end software systems remains challenging due to implementation diversity. Accordingly, Vision2Web adopts a workflow-based agent verification paradigm in which GUI agents execute expert-designed test workflows encoding multi-step, interdependent functionalities, while a dedicated VLM-based judge quantitatively evaluates visual fidelity against UI prototypes. This coordinated design enables reproducible and objective evaluation without sacrificing the flexibility of agent-based interactions. Our experiments reveal notable gaps in the capabilities of state-of-the-art coding agents across all three levels, highlighting limitations in cross-modal reasoning, long-horizon task planning, and multi-page coordination. These insights provide a foundation for future research in advancing agent reasoning and software development performance. In summary, Vision2Web offers three major contributions: • Hierarchical Task Design: A hierarchical task formulation that systematically disentangles agent capabilities across stages of visual website development. • Realistic Multimodal Data: A large-scale benchmark grounded in real-world websites with explicit specifications, enabling evaluation under realistic multimodal constraints. • Workflow-Based Agent Verification: A reproducible, implementation-agnostic evaluation paradigm that combines structured workflows with agent execution to assess both functional correctness and visual fidelity in end-to-end website development. 2 Overview of Vision2Web This section provides an overview of Vision2Web, including its task formulation dataset construction and overall statistics. 2.1 Task Formulation Visual website development encompasses a range of capabilities, from interpreting UI prototypes to managing interaction-driven application states and page transitions, ultimately delivering full-stack system delivery. To systematically evaluate these competencies, we formalize the coding task via a three-level hierarchical framework, with each level targeting a distinct set of critical skills. Level 1: Static Webpage. This level evaluates models’ ability to interpret UIs and generate executable code in a device-responsive setting. Each task provides prototype images of the same webpage across desktop, tablet, and mobile, with resolution specifications. Models must produce a single static webpage that faithfully reproduces layout, visual content, and styling at each resolution. Level 2: Interactive Frontend. At this level, inputs include multiple prototype images and text describing inter-page logical relationships. Models must generate a fully interactive multi-page frontend that preserves structural consistency and coherent navigation flows, assessing the ability to reason across pages and organize components in a multimodal context. Level 3: Full-Stack Website. This level simulates realistic engineering scenarios, providing structured requirement documents alongside prototype images. Agents are expected to interpret requirements, manage complex application states, perform integrated debugging, and deliver cohesive full-stack systems, evaluating comprehensive end-to-end software engineering capabilities. All tasks include a multimedia resource library with images, icons, videos and fonts to simulate realistic development. Moreover, each task is defined with specific and unambiguous requirements. This hierarchical design enables Vision2Web to systematically assess model capabilities across all stages of visual website development. 2.2 Dataset Construction To ensure high-quality, contamination-free evaluation data, Vision2Web is constructed through a multi-stage pipeline that refines large-scale web corpora into realistic, well-defined tasks suitable for systematic evaluation. All test tasks are sourced exclusively from the C4 ( Raffel et al. 2020 ) validation set to avoid potential leakage from popular websites, and we apply the following three-stage filtering pipeline to guarantee dataset quality and diversity: Structural Assessment. Following principles adapted from the Design2Code ( Si et al. 2025 ) benchmark, DOM-level properties, including HTML tag distribution, DOM tree depth, and token

Read the complete text