AnnoGram: An Annotative Grammar of Graphics Extension
\authororcid Md Dilshadur Rahman0009-0008-5467-615X, \authororcid Md Rahat-uz- Zaman0000-0001-6728-7569, \authororcid Andrew McNutt 0000-0001-8255-4258, and \authororcid Paul Rosen 0000-0002-0873-9518 · 2025
Read AnnoGram: An Annotative Grammar of Graphics Extension by \authororcid Md Dilshadur Rahman0009-0008-5467-615X, \authororcid Md Rahat-uz- Zaman0000-0001-6728-7569, \authororcid Andrew McNutt 0000-0001-8255-4258, and \authororcid Paul Rosen 0000-0002-0873-9518 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-04236.txt.
Introduction
AnnoGram, a chart note system built into a chart grammar, is set out by Md Dilshadur Rahman and co-authors in a 2025 arXiv paper. Notes make charts speak, yet most tools treat them as hand-placed extras. Such notes are hard to reuse and sit loose from the chart rules below. The team makes notes first-class parts with set targets, types, and place rules.
A test build extends the Vega-Lite chart tool. Stacked against eight tools, the build showed more range and less author work. Notes can move with the chart and keep their sense. The frame suits reports that update often.
Catalog record
| CREATOR | \authororcid Md Dilshadur Rahman0009-0008-5467-615X, \authororcid Md Rahat-uz- Zaman0000-0001-6728-7569, \authororcid Andrew McNutt 0000-0001-8255-4258, and \authororcid Paul Rosen 0000-0002-0873-9518 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-04236
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AnnoGram: An Annotative Grammar of Graphics Extension \authororcid Md Dilshadur Rahman0009-0008-5467-615X, \authororcid Md Rahat-uz- Zaman0000-0001-6728-7569, \authororcid Andrew McNutt 0000-0001-8255-4258, and \authororcid Paul Rosen 0000-0002-0873-9518 Abstract Annotations are central to effective data communication, yet most visualization tools treat them as secondary constructs—manually defined, difficult to reuse, and loosely coupled to the underlying visualization grammar. We propose a declarative extension to Wilkinson’s Grammar of Graphics that reifies annotations as first-class design elements, enabling structured specification of annotation targets, types, and positioning strategies. To demonstrate the utility of our approach, we develop a prototype extension called Vega-Lite Annotation. Through comparison with eight existing tools, we show that our approach enhances expressiveness, reduces authoring effort, and enables portable, semantically integrated annotation workflows. \onlineid 1199 \vgtccategory Research \vgtcpapertype Theoretical & Empirical \authorfooter Josiah Carberry is with Brown University. E-mail: [email protected] Ed Grimley is with Grimley Widgets, Inc. E-mail: [email protected]. Martha Stewart is with Martha Stewart Enterprises at Microsoft Research. E-mail: [email protected]. \teaser (A) An annotated Washington Post visualization [ 36 ] recreated using a proof-of-concept implementation of our AnnoGram extension to the Grammar of Graphics, (B) is shown alongside an excerpt of the specification. (C) the implementation supports diverse annotation types (text, connector, enclosure, indicator, etc.) across various chart types. keywords annotation, visualization grammar 1 Introduction Annotations are textual and graphical elements embedded in visualizations to draw attention, highlight salient features, provide context, and guide interpretation [ 22 , 27 , 30 ] . They improve memorability, comprehension, and recall [ 7 , 3 , 4 , 1 ] , and help externalize insights during analysis [ 15 , 34 , 19 ] . Annotations are employed in visualizations across numerous domains, including scientific research, business analytics, healthcare, education, journalism, and public policy. Despite their ubiquity, current programmatic tools do not treat annotations as first-class elements, making annotations difficult to reuse across different designs and datasets and impeding their computational analysis (such as for recommendation). Most visualization platforms treat annotations as secondary overlays, offering only ad hoc mechanisms loosely connected to underlying data and encodings. For example, Vega-Lite [ 32 ] lacks a dedicated annotation abstraction; users must instead define separate data sources and specify annotation marks using basic graphical primitives such as points, lines, and text. This approach is verbose, error-prone, and complicates the maintenance and updating of annotations as the data or design evolves. Similarly, D3.js [ 5 ] affords low-level graphical control but requires users to imperatively construct, position, and manage annotations, tightly binding them to specific chart implementations and reducing reusability. Although tools such as d3-annotation offer reusable task-specific components, their non-declarative design limits flexibility and generalization across visualization types. The situation is similar for other programmatic tools. As a result, many practitioners export charts to tools such as PowerPoint to add annotations manually [ 23 , 2 ] , severing links to data, removing interactivity, and limiting portability and automation. To address these challenges, we propose AnnoGram, an extension to Wilkinson’s Grammar of Graphics [ 39 ] (GoG) that elevates annotation to a first-class design element—decomplecting it from GoG’s notion of guides and secondary datasets, and making it a sibling to scales and geometries. Our extension supports declarative specification of annotation types, targets, positioning strategies, and styles. This support reduces manual positioning, enables data-driven placement, and promotes reusability, allowing annotations to transfer across charts as needs evolve. As a demonstration, we construct a proof-of-concept implementation that extends Vega-Lite [ 32 ] . By extending this DSL, rather than re-implementing it, we can reuse the full Vega rendering pipeline while still making use of it as a GoG petri dish (echoing Zong, Pollock, et al. [ 42 ] ). We evaluate the extension using a heuristic-based comparison with existing tools. Through this work, we show that annotations can be first-class components of visualization grammars that enabling consistent, reusable workflows. 2 Related Work Our work extends prior work on tool support for visualization annotations as well as on Grammar of Graphics [ 39 ] . GoG provides an expressive conceptual framework for describing a wide array of statistical graphics and has been used to structure tools like ggplot2 [ 38 ] and Vega-Lite [ 32 ] . The central observation in this model is that specifying charts via their types (e.g., scatterplots or bar charts) complects different ideas (e.g., notions of scale and encoding) that, when reified, support expressive construction of a broad range of charts, both novel and familiar. Yet, this model is not perfect and omits important structures for some tasks. For instance, accessibility [ 21 ] or uncertainty [ 25 ] , or as we consider, annotation. Notably, GoG can support each of these tasks in an unmodified form (cf. Wilkinson’s uncertainty chapter [ 39 ] ); however, it encounters the so-called Turing tarpit , a situation in programming language design in which “everything is possible but nothing of interest is easy” [ 24 ] . This situation is particularly evident for annotation, where current GoG-based tools require verbose and indirect encodings. We develop an extension that privileges a specific task of interest (annotation in our case) that, based on usage [ 26 ] , is believed to be important. Recent efforts have extended Vega-Lite’s declarative grammar to support underrepresented design concerns—an approach our work follows for annotations. For instance, Cicero [ 16 ] is a responsive design grammar that modifies Vega-Lite to enable layout adaptation across devices. Animated Vega-Lite [ 42 ] extends the Vega-Lite’s compiler with a temporal encoding channel to support animation, treating time as a first-class visual variable. Tactile Vega-Lite [ 8 ] augments the grammar with tactile-specific abstractions—such as braille, textures, and spatial layout controls—to support blind and low-vision readers. DXR [ 35 ] uses an extension of Vega-Lite syntax to support authoring immersive 3D visualizations. We similarly extend the Vega-Lite grammar but focus on the underexplored dimension of annotation, reifying it as a core design element. In doing so, our work complements these prior extensions by addressing a different yet equally foundational aspect of visualization design. Beyond grammar-based approaches, many tools support annotations [ 40 , 12 , 13 , 9 ] , albeit in limited ways [ 26 ] . For example, ChartAccent [ 31 ] maps data to marks using rules. Contextifier [ 14 ] automatically generates annotations from text for time series charts. Although these systems support annotations, they commonly treat them as external overlays detached from the underlying visualization model, lacking the modularity, portability, and integration commonly found in data-driven workflows. Our work enables annotations to be specified, reasoned about, and rendered alongside core visualization constructs. Some prior work has also explored structured annotations for reasoning [ 41 ] and narrative generation [ 6 ] , treating annotations as integral to analysis or storytelling. Our work complements these by formalizing annotation structure for declarative specification and reuse. 3 Extension Design Our primary contribution is AnnoGram (summarized in Figure 1 ), an extension to GoG that elevates annotations to the same level as other primary GoG components, such as geometries or scales. Unlike other extensions, ours is complicated by the issue that annotations require information from across the specification-assembly-display pipeline (a concept underpinning the organization of GoG [ 39 ]