ChartComplete: A Taxonomy-based Inclusive Chart Dataset

Ahmad Mustapha, Charbel Toumieh, Mariette Awad · 2026

Read ChartComplete: A Taxonomy-based Inclusive Chart Dataset by Ahmad Mustapha, Charbel Toumieh, Mariette Awad (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-10462.txt.

Introduction

Ahmad Mustapha, Charbel Toumieh, and Mariette Awad built a chart set to test sight and text models. Chart sense is a fast growth field. Models now read bars, lines, and pies with good grades. To judge them, labs need test sets that span chart kinds in full.

The set is named ChartComplete. It is built on a taxonomy, a sorted tree of chart types. That tree makes sure no main type is missed. Each type is well stocked in the set. Labs can then see which chart kinds trip a model. The work gives the field a more whole test base.

Catalog record

CREATORAhmad Mustapha, Charbel Toumieh, Mariette Awad
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE16 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-10462

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ChartComplete: A Taxonomy-based Inclusive Chart Dataset Ahmad Mustapha, Charbel Toumieh, Mariette Awad Abstract With advancements in deep learning (DL) and computer vision techniques, the field of chart understanding is evolving rapidly. In particular, multi-modal large language models (MLLMs) are proving to be efficient and accurate in understanding charts. To accurately measure the performance of MLLMs, the research community has developed multiple datasets to serve as benchmarks. By examining these datasets, we found that they are all limited to a small set of chart types. To bridge this gap, we propose the ChartComplete dataset. The dataset is based on a chart taxonomy borrowed from the visualisation community, and it covers thirty different chart types. The dataset is a collection of classified chart images and doesn’t include a learning signal. We present the ChartComplete dataset as is to the community to build upon it. Keywords: Chart, Dataset, Chart Taxonomy, Chart Classification Background Artificial intelligence systems are rapidly becoming smarter and capable of handling more complex tasks. One of these complex tasks is understanding charts. In particular, ChartQA is a research domain where machines are designed to have a visual understanding of chart elements. In ChartQA, a machine is handed a question and a chart, and the machine is supposed to answer the question about this image. To be successful in answering the question, the machine is required to perform a complex task that is composed of computer vision, natural language processing, and domain knowledge, all in one shot. A handful of benchmarks have been proposed to test machine learning models’ capabilities in handling ChartQA questions. However, the majority of these benchmarks are limited to a few types of charts, such as bar, pie, line, and scatter charts. To truly test a model’s capabilities in ChartQA, the benchmarks should be more inclusive and include more types of chart. Ideally, models should support common chart types. To bridge this gap, we present the ChartComplete dataset. ChartComplete includes 30 different types of chart, ranging from the common bar charts to the less common charts, such as parallel coordinates charts, including special charts such as box and whisker charts and choropleth map charts. Unlike existing benchmarks, ChartComplete was based on a chart taxonomy that was borrowed from the visualization taxonomy. Figure 1: The visual feature space of ChartComplete images. The features are extracted using Google VIT. The space is projected using T-SNE. One of the first works on the topic was published under the name FigureSeer [ 10 ] . Following the trace of FigureSeer but with scalability in mind, FigureQA [ 4 ] was proposed. FigureQA is the first large-scale dataset that addresses the ChartQA question. Both the data for the charts and the charts themselves were generated by the authors. To solve the problem of unnatural data labels, PlotQA [ 9 ] was proposed. Unlike FigureQA, the chart data were collected from different sources. The charts themselves were generated. As a sequel in this line of research, ChartQA [ 8 ] was proposed. ChartQA collected charts and their data from online resources, making it the first dataset to have real-life-looking charts with real-life data. Other data sets include Chart-to-text [ 6 ] , OpenCQA [ 5 ] , and ChartFC [ 1 ] . All of these data sets and benchmarks presented above addressed only a handful of types of charts. Those charts were mainly bar charts, horizontal and vertical, grouped and stacked, line charts, and pie charts. Table 1 summarizes the chart types used by different benchmarks. Table 1: Summary of the chart types covered by existing datasets Dataset Supported Charts FigureSeer Graph Plot, Flow Chart, Scatter Plot, Bar Plot FigureQA Line Plot, Dot-Line Plot, Vertical Bar Plot, Horizontal Bar Plot, Pie Plot PlotQA Bar Plot, Scatter Plot, Line Plot ChartQA Horizontal Bar Plot, Vertical Bar Plot, Stacked Bar Plot, Pie Chart, Line Plot Chart-to-Text Bar Plot, Grouped Bar Plot, Stacked Bar Plot, Line Plot, Area Plot, Scatter Plot, Pie Plot OpenCQA Bar Plot, Grouped Bar Plot, Stacked Bar Plot, Line Plot, Area Plot, Scatter Plot, Pie Plot ChartFC Bar Plot Figure 2: The Centered Kernel Alignment (CKA) values between different chart types features. The features are extracted using Google ViT. The higher the CKA value the more the features are similar. A current limitation of ChartComplete is that it serves solely as a dataset of classified chart images. It presently lacks a direct training signal, such as those used for summarization or question answering tasks. We plan to address this in future work and actively encourage the community to contribute to its development in these areas. Collection Methods and Design The collection process of the ChartComplete dataset was based on a modified version of Borkin’s taxonomy of chart types [ 2 ] . The taxonomy has categories of charts and terms, or chart types in each category. We have thirty different chart types. For each chart type, we collected fifty images. The collection process was partially manual and partially automated, utilising scraping. Table 2 shows the counts and the percentage of manually collected images vs scraped ones. Table 2: Summary of collected and scraped charts images in the ChartComplete dataset Type Count Percentage (%) Collected 951 63.4 Scraped 549 36.6 Table 3 presents the taxonomy on which we based our work. It is a modified version of Borkin’s taxonomy [ 2 ] . As aforementioned, we introduced some changes to the original Borkin taxonomy. First, we introduced the ”stacked” variant of different charts. We define the ”stacked” chart as a chart that represents several data columns in the same plot. Several lines in a line chart, for example. They usually have several legends. The naming convention was inspired by the common ”stacked bar chart”. In the literature, such charts are differentiated from simpler charts by using the simple/complex prefixes [ 5 ] . For clarity, we decided to use the vanilla/stacked prefixes. We introduced the stacked class to four chart types: area, line, scatter, and bubble charts. The charts that represent only one data column are referred to as ”vanilla” charts. Second, we removed from the taxonomy the less common chart types like the Venn diagram, the circular bar chart, or the contour line chart. Third, we added combination charts, like the bar and line charts. Table 3 represents the entire taxonomy. Table 3: The modified Borkin taxonomy used to build ChartComplete Category Terms Area Vanilla, Overlapped, Stacked, Proportional Bar Vanilla, Grouped, Waterfall Circle Donut, Pie Diagram Flow, Sankey, Timeline Distribution Curve, Histogram, Box and Whisker Matrix Heat Map Line Vanilla, Stacked, Radar, Surface, Parallel Coordinates Map Choropleth, Contour Point Scatter, Stacked Scatter, Bubble, Stacked Bubble Text Word Cloud Tree Tree Map Combination Bar and Line Chart To collect the ChartComplete dataset we utilized the following sources: • Source 1: Statista 1 1 1 www.statista.com/chartoftheday/ website, which is a global data and business intelligence platform with an extensive collection of statistics, reports, and insights on over 80,000 topics from 22,500 sources in 170 industries. We scraped all the charts listed on the chart of the day webpage on the 27th of January 2025. • Source 2: Our World in Data 2 2 2 www.ourworldindata.org website, which is a project of the Global

Read the complete text