ChartE 3 : A Comprehensive Benchmark for End-to-End Chart Editing
Shuo Li, Jiajun Sun, Zhekai Wang, Xiaoran Fan, Hui Li, Dingwen Yang, Zhiheng Xi, Yijun Wang, Zifei Shan, Tao Gui, Qi Zhang, Xuanjing Huang · 2026
Read ChartE 3 : A Comprehensive Benchmark for End-to-End Chart Editing by Shuo Li, Jiajun Sun, Zhekai Wang, Xiaoran Fan, Hui Li, Dingwen Yang, Zhiheng Xi, Yijun Wang, Zifei Shan, Tao Gui, Qi Zhang, Xuanjing Huang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21694.txt.
Introduction
Shuo Li, Jiajun Sun, and twelve co-authors build a benchmark for editing charts end to end from user intent. Charts are a basic format for structured data analysis. Editing one on request sounds simple but demands fine control and global consistency at once.
A changed label must not break the scale, and a new color scheme must stay readable. Their benchmark grades systems on full editing tasks, not toy tweaks. End-to-end skill here would save analysts hours of fiddly manual work. The benchmark also exposes where current AI tools still mangle the structure they were asked to preserve.
Catalog record
| CREATOR | Shuo Li, Jiajun Sun, Zhekai Wang, Xiaoran Fan, Hui Li, Dingwen Yang, Zhiheng Xi, Yijun Wang, Zifei Shan, Tao Gui, Qi Zhang, Xuanjing Huang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-21694
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ChartE 3 : A Comprehensive Benchmark for End-to-End Chart Editing Shuo Li, Jiajun Sun, Zhekai Wang, Xiaoran Fan, Hui Li, Dingwen Yang, Zhiheng Xi, Yijun Wang, Zifei Shan, Tao Gui, Qi Zhang, Xuanjing Huang Abstract Charts are a fundamental visualization format for structured data analysis. Enabling end-to-end chart editing according to user intent is of great practical value, yet remains challenging due to the need for both fine-grained control and global structural consistency. Most existing approaches adopt pipeline-based designs, where natural language or code serves as an intermediate representation, limiting their ability to faithfully execute complex edits. We introduce ChartE 3 , an E nd-to- E nd Chart E diting benchmark that directly evaluates models without relying on intermediate natural language programs or code-level supervision. ChartE 3 focuses on two complementary editing dimensions: local editing, which involves fine-grained appearance changes such as font or color adjustments, and global editing, which requires holistic, data-centric transformations including data filtering and trend line addition. ChartE 3 contains over 1,200 high-quality samples constructed via a well-designed data pipeline with human curation. Each sample is provided as a triplet of a chart image, its underlying code, and a multimodal editing instruction, enabling evaluation from both objective and subjective perspectives. Extensive benchmarking of state-of-the-art multimodal large language models reveals substantial performance gaps, particularly on global editing tasks, highlighting critical limitations in current end-to-end chart editing capabilities. † † affiliationnotice: Equal contribution 1 Introduction Charts are one of the most widely used visualization formats for presenting structured data in scientific research, business analysis, and decision-making workflows. Unlike natural images, charts encode precise numerical values, semantic structures, and visual conventions, making their correct interpretation and manipulation essential for downstream data analysis tasks. As multimodal large language models (MLLMs) ( 23 ) continue to advance in visual understanding and reasoning, chart understanding has emerged as a critical testbed for evaluating their ability to jointly reason over visual and structured information. Table 1 : Comparison of our proposed benchmark with existing chart generation benchmarks. While prior benchmarks mainly target captioning, QA, or chart-to-code generation, they offer limited support for end-to-end chart editing. Chart E 3 provides broader coverage of charts and places stronger emphasis on global editing tasks with reference target images, enabling more reliable evaluation of complex chart edits. Name Output Format Image Source Chart Types w/ Ref. Image Local Editing Global Editing Conversion Order Filtering Reference ChartCraft ( 42 ) Json Syn. 5 ✓ ✓ ✗ ✗ ✓ ✗ Plot2Code ( 38 ) Code Syn. 6 ✓ ✗ ✗ ✗ ✗ ✗ ChartX ( 40 ) Code Syn. 18 ✓ ✓ – ✗ ✗ ✗ ✗ AcademiaChart ( 48 ) Code Real 6 ✓ ✗ ✗ ✗ ✗ ✗ ChartM 3 ( 44 ) Code Syn. 10 ✓ ✓ ✓ ✗ ✗ ✗ ChartEdit ( 49 ) Code Real 19 ✓ ✓ ✓ ✗ ✗ ✗ Chart E 3 (Ours.) Figure Syn.&Real 10/42 ✓ ✓ ✓ ✓ ✓ ✓ Recent progress in chart-related research can be broadly categorized into two main lines of work. The first line focuses on chart understanding, including tasks such as chart classification, data extraction, question answering, and numerical reasoning over plotted values. These studies ( 22 ; 41 ; 40 ) aim to recover the semantics and underlying data encoded in chart images, and have led to substantial improvements in models’ abilities to interpret structured visual information. The second line of work addresses chart editing through code-mediated pipelines. In this paradigm, models first translate user intent into chart-specific code or program representations—such as plotting scripts or configuration files—which are then executed by a rendering engine to produce the edited chart image ( 49 ; 44 ; 38 ; 4 ; 13 ) . This formulation naturally leverages the structured nature of charts and has shown promising results in constrained editing scenarios. However, such approaches primarily evaluate a model’s ability to generate syntactically and semantically correct code, rather than its capacity to perform true end-to-end chart editing, where an input chart is directly transformed into an edited output. Moreover, the reliance on intermediate code representations decouples visual perception from editing execution, making it difficult to assess how well models jointly reason over visual appearance, data semantics, and editing intent. As a result, the end-to-end editing capability of multimodal models remains largely unexplored and insufficiently evaluated as shown in Table 1 . To address these limitations, we introduce ChartE 3 , an end-to-end Chart Editing benchmark designed to systematically evaluate chart editing capabilities of multimodal models under a true image-to-image setting. Unlike prior benchmarks that operate through intermediate code representations (e.g., ChartM3 ( 44 ) , ChartX ( 40 ) ), where editing instructions explicitly reference code variables or rendering patterns that are inaccessible to image editing models, ChartE 3 focuses on fully image-based editing. It evaluates whether models can directly manipulate visual charts while preserving perceptual fidelity, semantic correctness, and structural consistency. ChartE 3 is constructed through a carefully designed five-stage data pipeline to ensure both diversity and annotation quality. Specifically, we first collect raw chart data from diverse sources and chart types. We then perform image diversity filtering to reduce redundancy and ensure broad coverage of visual layouts and styles. Based on the filtered charts, we obtain paired chart–code representations, which serve as a reliable reference for subsequent edit generation. Using these paired representations, we generate edited chart data that reflect a wide range of realistic editing intents. Finally, we apply automatic filtering followed by human verification to remove invalid or ambiguous samples and to ensure the correctness and consistency of annotations. The benchmark is organized around two major categories of editing tasks: local editing and global editing. Local editing focuses on fine-grained, appearance-level modifications, such as adjusting fonts, colors, or annotations, while global editing requires holistic, data-centric transformations, including data filtering, aggregation, and the addition of derived visual elements. These two categories are further divided into 12 fine-grained task types, capturing a broad spectrum of chart editing tasks. In total, ChartE 3 contains over 800 unique chart images and more than 1,200 annotated editing samples. We employ objective metrics, including SSIM and CLIP-based similarity, to measure visual consistency, and complement them with GPT-based subjective scoring to assess semantic correctness and edit faithfulness. Using this benchmark, we conduct extensive evaluations of both open-source and closed-source multimodal models. Our results reveal several notable findings: closed-source models often outperform open-source counterparts on end-to-end chart editing; local editing tasks are consistently easier than global, data-level edits; and error analysis shows that many failures stem from limitations in text generation and instruction grounding, even when visual understanding is correct. In summary, our main contributions are: • We introduce a new paradigm for end-to-end chart editing, emphasizing direct transformation without intermediate code or program representations; • We present ChartE 3 , a comprehensive benchmark for evaluating end-to-end chart editing across diverse local and global editing tasks; • We perform extensive benchmarking and analysis of current multimodal models, uncovering systematic strengths, weaknesses, and failure modes that point to future research directions in end-to-end chart editing. 2 Related Work Multimodal Large Language Models Multimodal large language models (MLLMs) have recently attracted significant attention as a unified framework for visual–language understanding,