VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing
Xiaoyan Su, Peijie Dong, Zhenheng Tang, Song Tang, Yuyao Zhai, Kaitao Lin, Liang Chen, Yuhang Gai, Yuyu Luo, Qiang Wang, Xiaowen Chu · 2026
Read VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing by Xiaoyan Su, Peijie Dong, Zhenheng Tang, Song Tang, Yuyao Zhai, Kaitao Lin, Liang Chen, Yuhang Gai, Yuyu Luo, Qiang Wang, Xiaowen Chu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15677.txt.
Introduction
Xiaoyan Su and ten colleagues launch VCG-Bench, a benchmark for visual diagram tasks. Current vision-language models handle structured diagrams poorly, since pixel synthesis limits editability and fidelity. The team proposes a Diagram-as-Code paradigm with symbolic logic, using mxGraph XML for diagram generation and editing. VCG-Bench holds 1,449 diverse diagrams across 6 domains and 15 sub-domains. It joins generation (vision-to-code) with editability (code-to-code) under a tailored evaluation protocol with multi-dimensional metrics. Tests show top models still struggle with structured fidelity and instruction compliance.
Catalog record
| CREATOR | Xiaoyan Su, Peijie Dong, Zhenheng Tang, Song Tang, Yuyao Zhai, Kaitao Lin, Liang Chen, Yuhang Gai, Yuyu Luo, Qiang Wang, Xiaowen Chu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 115413 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.15677
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing Xiaoyan Su, Peijie Dong, Zhenheng Tang, Song Tang, Yuyao Zhai, Kaitao Lin, Liang Chen, Yuhang Gai, Yuyu Luo, Qiang Wang, Xiaowen Chu Abstract Despite the rapid advancements in Vision-Language Models (VLMs), a critical gap remains in their ability to handle structured, controllable diagrammatic tasks essential for professional workflows. Existing methods predominantly rely on pixel-based synthesis, which operates in probabilistic pixel spaces and is inherently limited in editability and fidelity. Instead, we propose a new “Diagram-as-Code” paradigm with symbolic logic that leverages mxGraph Extensible Markup Language (XML) for precise diagram generation and editing. We present VCG-Bench , a unified benchmark for visual-centric mxGraph tasks. VCG-Bench comprises: (1) a taxonomized dataset of 1,449 diverse diagrams spanning 6 domains and 15 sub-domains, (2) a paradigm definition that integrates Generation (Vision-to-Code) and Editability (Code-to-Code), (3) a Tailored Evaluation Protocol employing multi-dimensional metrics such as mxGraph Execution Success Rate, Style Consistency Score (SCS), etc. Experimental results highlight the challenges faced by current State-of-the-Art (SOTA) VLMs in structured fidelity and instruction compliance, reflecting their vision and reasoning capabilities. † † affiliationnotice: Equal contribution 1 Introduction Vision-Language Models (VLMs) ( 42 ; 15 ; 59 ; 10 ; 31 ; 56 ; 3 ) have demonstrated impressive capabilities in general multimodal tasks, spanning visual question answering ( 2 ) to open-ended image generation ( 51 ; 46 ) . Through high-capacity architectures and massive-scale multimodal pre-training, modern VLMs have achieved near-human performance in scene description, optical character recognition (OCR), and cross-modal reasoning. Yet, moving beyond natural images to abstract, symbolic, and structured visual information remains a key frontier. This gap is particularly evident for diagrams such as flowcharts, system architectures, scientific figures, and UI mockups—which are pervasive in professional settings ( 45 ) . Unlike natural images, diagrams are designed to encode precise semantics through composition, layout, and discrete symbols, requiring models to capture structure rather than appearance. Figure 1 : Comparison of diagrammatic tasks. VCG utilizes symbolic mxGraph XML for precise generation and editing, overcoming structural drift in pixel-based models. Table 1 : Comparative analysis of visual representation formats. mxGraph balances semantic richness with structured editability. Dimension mxGraph SVG HTML/CSS PPT Primitive Semantic Entities Vector Paths DOM Elements Prop. Objects (Nodes, Edges) (Lines, Curves) (Tags, Styles) (Slides, Shapes) Semantic Depth High Low Medium Low (Logical Topology) (Visual Geometry) (Structure) (Presentation) Editability Structured Unstructured Rigid Restricted (Topology Preserved) (Pixel/Vector) (Flow) (GUI-centric) Interpretability Open XML Open XML Open Text Prop./Binary Controllability High Low Medium Poor Modification Cost Low High Medium Low (Software Ops) (Path Recalc.) (DOM/Style) (GUI Manual) Figure 2 : Overview of the VCG-Bench framework. Unlike fragmented approaches (top), VCG-Bench (bottom) unifies Vision-to-Code Generation (Task 1) and Instruction-to-Patch Editing (Task 2). Utilizing symbolic mxGraph XML enables precise, low-cost modifications for professional workflows. We propose VCG-Bench , a unified benchmark designed to evaluate the end-to-end capability of VLMs in generating and editing mxGraph XML diagrams. VCG-Bench follows a “Data-Task-Evaluation” framework. (1) We construct a diverse dataset categorized into 6 major domains (Academic, Software, Business, etc.), covering complex, logic-intensive layouts. (2) We unify diagrammatic tasks into two complementary streams: Generation (creating valid mxGraph XML from standard images) and Editability (modifying existing structures based on instructions), with an Incremental Modification Strategy to enhance efficiency. (3) We establish a rigorous evaluation protocol beyond simple text matching, incorporating execution success rates, visual style consistency (SCS), and semantic fidelity checks via QA. Our contributions are summarized below: • We introduce VCG-Bench, the first benchmark bridging the gap between vision and editable structured generation. • We curate a high-quality, taxonomized dataset of 1,449 diagrams with provenance and structural diversity. • We propose a comprehensive evaluation protocol with novel metrics for executability, style consistency, and instruction following, benchmarking SOTA VLMs on diagrammatic tasks. To motivate this benchmark design, we first contrast two diagrammatic paradigms. For diagrammatic tasks, one of the main paradigms is “Diagram-as-Pixels” . It relies on rasterized pixel generation and editing pipelines (e.g., prompt + pixel-editing models such as GPT-Image and diffusion models). Although it can produce visually plausible results, it is inherently stochastic, unstructured, and hallucination-prone , because it operates on pixels rather than symbolic structure. Fig. 1 illustrates that this often leads to blurry lines, text distortion, scale/alignment errors, and even hallucinated elements (e.g., inserting an unintended “CDN”). These issues conflict with practical diagram editing workflows that require: (1) vector-accurate outputs (e.g., SVG or mxGraph ), (2) low-cost incremental modifications , and (3) frequent iterative edits without degradation. To address these limitations, we adopt a “Diagram-as-Code” paradigm, which is more controllable, structured, and deterministic . Instead of editing pixels, we use VLMs (e.g., Gemini ( 15 ) , Claude ( 1 ) , and GPT ( 43 ) ) to convert raster figures into structured representations. Guided by user instructions, the system performs edits via Visual Controllable Generation (VCG) with symbolic operations , enabling effective collaboration between LLMs and human users while preserving precise, editable diagram structure—producing clean, high-fidelity output figures (Fig. 1 ). Notably, the Diagram-as-Code paradigm can be integrated with pixel-level generators to produce more aesthetically pleasing, stylized renderings. For “Diagram-as-Code” , Tab. 1 compares dominant representation formats ( mxGraph , SVG , HTML/CSS , PPT ). SVG is largely geometry-centric with limited semantics, HTML/CSS offers only moderate structural control, and PPT is GUI-centric with restricted programmatic editability. By contrast, mxGraph models diagrams as semantic entities (nodes/edges), providing high semantic depth , structured editability , and high AI controllability through an interpretable open XML representation, making it well-suited for AI-driven reasoning and precise modification. (See App. D.2 for further discussion.) Figure 3 : Overview of the VCG-Bench data generation framework. Subfigure (a) illustrates the end-to-end pipeline for Task 1, from raw web scraping to structured XML-based rendering. Subfigure (b) details the Task 2 pipeline, which focuses on generating editing-based reasoning tasks derived from Task 1. Fig. 3 details the data generation pipeline for VCG-Bench and shows the systematic process from data collection to structured evaluation. App. B provides detailed descriptions of the data synthesis pipeline, including Stage 1 (Image-to- mxGraph XML) and Stage 2 (Instruction Synthesis). 2 Related Work Our work advances the domain of multimodal code generation by focusing on the mxGraph format. We situate it in four strands of related literature: chart and scientific plot generation, UI and symbolic graphics, instruction-driven image editing, and evaluation methodologies. Chart and Scientific Plot Generation. Diagrammatic reasoning has been advanced by VQA benchmarks (e.g., AI2D ( 23 ) , ChartQA ( 37 ) , FigureQA ( 22 ) , MMBench ( 35 ) ) and chart-understanding evaluations ( CharXiv ( 57 ) , MMSCI ( 28 ) , AIBench ( 29 ) ). Recent work on executable code includes ChartMimic ( 65 ; 41 ) , Plot2Code ( 60 ) , PlotCraft ( 70 ) , MMCode ( 26 ) , Flow2Code ( 17 ) , and RealChart2Code ( 69 ) ; most of this line remains centered on rasterized, data-driven visualizations. Interactive visualization systems such as HAIChart ( 62 ) study human-AI paired chart creation. Related to scientific diagrams, Draw with Thought ( 11 ) reconstructs static scientific diagrams into editable mxGraph XML. VCG-Bench provides a multi-domain Draw.io /