Athanor : A u th oring A ction Modification-based Interactions on Static Visualizations via N atural Language

\authororcid Can Liu0000-0002-1175-0734, Jaeuk Lee, Tianhe Chen, Zhibang Jiang, \authororcid Xiaolin Wen0000-0002-8562-7640 and \authororcid Yong Wang0000-0002-0092-0793 · 2026

Read Athanor : A u th oring A ction Modification-based Interactions on Static Visualizations via N atural Language by \authororcid Can Liu0000-0002-1175-0734, Jaeuk Lee, Tianhe Chen, Zhibang Jiang, \authororcid Xiaolin Wen0000-0002-8562-7640 and \authororcid Yong Wang0000-0002-0092-0793 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-17736.txt.

Introduction

Can Liu and co-authors present Athanor, a tool to make static charts interactive by voice or text. Interactivity is crucial for good data visualization. Static charts, though, often lack source code and data. Rebuilding them takes long effort even when code exists. Athanor lets users describe changes in natural language. It applies action modifications to the chart as shown.

A working tool would rescue vast chart archives. News, reports, and papers hold millions of static figures. Users could probe them without the author's files. That widens access for analysts and the public. The work treats the image itself as the interface. Language becomes the bridge to interaction.

Catalog record

CREATOR\authororcid Can Liu0000-0002-1175-0734, Jaeuk Lee, Tianhe Chen, Zhibang Jiang, \authororcid Xiaolin Wen0000-0002-8562-7640 and \authororcid Yong Wang0000-0002-0092-0793
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-17736

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Athanor : A u th oring A ction Modification-based Interactions on Static Visualizations via N atural Language \authororcid Can Liu0000-0002-1175-0734, Jaeuk Lee, Tianhe Chen, Zhibang Jiang, \authororcid Xiaolin Wen0000-0002-8562-7640 and \authororcid Yong Wang0000-0002-0092-0793 Abstract Interactivity is crucial for effective data visualizations. However, it is often challenging to implement interactions for existing static visualizations, since the underlying code and data for existing static visualizations are often not available, and it also takes significant time and effort to enable interactions for them even if the original code and data are available. To fill this gap, we propose Athanor , a novel approach to transform existing static visualizations into interactive ones using multimodal large language models (MLLMs) and natural language instructions. Our approach introduces three key innovations: (1) an action-modification interaction design space that maps visualization interactions into user actions and corresponding adjustments, (2) a multi-agent requirement analyzer that translates natural language instructions into an actionable operational space, and (3) a visualization abstraction transformer that converts static visualizations into flexible and interactive representations regardless of their underlying implementation. Athanor allows users to effortlessly author interactions through natural language instructions, eliminating the need for programming. We conducted two case studies and in-depth interviews with target users to evaluate our approach. The results demonstrate the effectiveness and usability of our approach in allowing users to conveniently enable flexible interactions for static visualizations. \onlineid 5086 \ieeedoi xx.xxxx/xxxx.201x.xxxxxxx \vgtccategory Research \vgtcpapertype please specify \authorfooter C. Liu, T. Chen, X. Wen, and Y. Wang are with Nanyang Technological University. E-mail: [email protected], {chen1594, xiaolin004}@e.ntu.edu.sg, [email protected]. Y. Wang is the corresponding author. J. Lee is with Ajou University. E-mail: [email protected]. Z. Jiang is with Providence Health & Services. E-mail: [email protected]. \teaser Given a static chart (a), Athanor supports enabling interactions for it. The static chart has been enhanced with buttons to modify chart types (transforming (a) to (b) and (d) to (e)), the ability to rearrange the visual elements to stack or group ((b) to (c) and (e) to (f)), rescaling on the x-axis ((c) to (d)), and hover functionality to display a detailed tooltip (f). After authoring, the static line chart can support interactive features that enable users to perform in-depth analysis. keywords User interfaces, interaction authoring, interactive visualization, multi-agent 1 Introduction Interactivity adds a new dimension to visualizations, enabling users to dynamically explore data, uncover hidden patterns, and tailor views to their specific needs. For instance, interactive visualization software like Tableau and Power BI enable real-time filtering and zooming, revealing insights that are typically inaccessible in static visualizations. However, despite the power of these interactive visualization tools, static visualizations remain prevalent in published outputs [ 56 ] , typically presenting data from a fixed perspective with underlying data and code usually inaccessible to viewers. This static nature limits users’ ability to adjust focus, compare different time periods, and segment data, thereby hindering users from gaining deeper data insights. Interactivity can significantly enhance the utility of static visualizations, yet integrating it into static formats is far from being straightforward and easy [ 48 ] . There are three major challenges that have limited the adoption of interactive visualizations: • C1. Interaction Complexity: The interaction space is vast as it encompasses multidimensional design space ranging from different actions (e.g., clicking, hovering, dragging) to various analytical intents (e.g., filtering, comparing, and hypothesis validation) [ 59 , 13 ] . It is often time-consuming to enable interactions for a static visualization. Even with access to the original code and data, modifying each visualization’s code to add interactive features requires substantial visualization expertise and implementation skills. • C2. User Requirement Diversity: The space of user-required interaction functionalities is enormous. Different users have distinct requirements based on their specific scenarios, tasks, and analysis goals. For example, data analysts may need complex filtering and comparison interactions, while casual users prefer simple highlighting and annotation features. This diversity in user requirements across different contexts makes it challenging to provide a standardized implementation of visualization interactions. • C3. Visualization Implementation Diversity: There are various types of visualizations (e.g., bar charts, area charts, line charts, etc.) and visualizations can be implemented using different toolkits, e.g., Vega-Lite [ 40 ] , ECharts [ 21 ] , and D3 [ 3 ] , resulting in substantially different code implementations. This heterogeneity makes it difficult to enable interactions for visualizations implemented by different visualization toolkits. Furthermore, many existing visualizations lack access to the underlying data and code, making it infeasible to support interactivity by extending the original implementation. To address these challenges, we propose Athanor , a novel approach that allows users to add interactions to existing static visualizations conveniently and efficiently based on users’ natural language inputs. Our approach comprises an action-modification interaction design space , a multi-agent requirement analyzer , and an implementation-agnostic representation translator . The action-modification interaction design space maps interactions to user actions and their corresponding visualization modifications , providing a structured framework to describe the potential interactions supported by a static visualization’s information. This design space specifies how user actions trigger specific visualization modifications, forming an actionable interaction model ( C1 ). The multi-agent requirement analyzer interprets users’ natural language requirements, transforming them into actionable specifications within the action-modification design space. Through a collaborative agent-based approach, it processes diverse inputs to precise operations by translating inputs into structured sequences of actions and modifications, correcting parsing errors, and providing guidance to refine unclear requests ( C2 ). The visualization abstraction translator converts existing visualizations into a flexible abstraction independent of the underlying implementation ( C3 ). The translator leverages multimodal large language models (MLLMs) to parse existing visualizations, representing the results through a constraint model [ 27 ] , which is an implementation-agnostic representation for existing visualizations. Athanor can effectively parse the content of existing visualizations in SVG format, especially the most popular charts (e.g., bar charts, line charts, scatter plots, and area charts), and then directly enable interactivity on them, which can maintain the visual consistency between the interactive visualizations and the original static visualizations. To demonstrate the effectiveness of Athanor , we first present two case studies to illustrate how users can conveniently enable interactions for a given visualization with the help of our approach. Also, we conducted in-depth user interviews with 11 participants. The feedback indicates that our action-modification design space can cover various user requirements, and Athanor allows users to effectively add interactions to static visualizations with natural language inputs. The contributions of this work are as follows: • We propose Athanor that consists of three major components: action-modification interaction design space, multi-agent requirement analyzer, and visualization abstraction translator. It allows users to conveniently author flexible interactions for static visualizations via natural language interaction. • We present two case studies and conduct in-depth user interviews to demonstrate the effectiveness and usefulness of Athanor . A gallery of interaction-enhanced visualizations created by using our approach can be found at https://authorvisinter.github.io . 2 Related Work Our work provides a natural language interface (NLI) for authoring interactions on static visualizations, which relates to visualization interaction, NLI for visualization, and visualization enhancement. 2.1 Visualization Interaction Visualization interaction encompasses various ways users engage with and manipulate visual representations of data. We treat a visualization interaction as user

Read the complete text