Agentic Visualization: Extracting Agent-based Design Patterns from Visualization Systems Journal: IEEE Computer Graphics & Applications

Vaishali Dhanoa, Anton Wolter, Gabriela Molina León, Hans-Jörg Schulz, Niklas Elmqvist · 2025

Read Agentic Visualization: Extracting Agent-based Design Patterns from Visualization Systems Journal: IEEE Computer Graphics & Applications by Vaishali Dhanoa, Anton Wolter, Gabriela Molina León, Hans-Jörg Schulz, Niklas Elmqvist (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19101.txt.

Introduction

Vaishali Dhanoa and co-authors read today's visualization tools through the lens of autonomous agents in a 2025 paper. Large language models have made software agents practical, and data visualization is next in line. The field has always kept a human in the loop of making sense of data, so handing work to agents raises hard questions about autonomy and control.

The authors re-examine existing systems that already automate parts of analysis, from suggesting charts to drafting insights, and reinterpret them as agents. From that survey they extract design patterns: the roles agents can play, how they talk to users and each other, and how work is coordinated. The patterns aim to amplify analysts, not replace them. The piece offers a shared vocabulary for building tools where people keep insight and command.

Catalog record

CREATORVaishali Dhanoa, Anton Wolter, Gabriela Molina León, Hans-Jörg Schulz, Niklas Elmqvist
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-19101

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Agentic Visualization: Extracting Agent-based Design Patterns from Visualization Systems Journal: IEEE Computer Graphics & Applications Vaishali Dhanoa, Anton Wolter, Gabriela Molina León, Hans-Jörg Schulz, Niklas Elmqvist Abstract Autonomous agents powered by Large Language Models are transforming AI, creating an imperative for the visualization field to embrace agentic frameworks. However, our field’s focus on a human in the sensemaking loop raises critical questions about autonomy, delegation, and coordination for such agentic visualization that preserve human agency while amplifying analytical capabilities. This paper addresses these questions by reinterpreting existing visualization systems with semi-automated or fully automatic AI components through an agentic lens. Based on this analysis, we extract a collection of design patterns for agentic visualization, including agentic roles, communication and coordination. These patterns provide a foundation for future agentic visualization systems that effectively harness AI agents while maintaining human insight and control. \jtitle Agentic Visualization: Extracting Agent-based Design Patterns from Visualization Systems \sptitle SPECIAL ISSUE † † footnotetext: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. \chapteri Autonomous agents—software entities that perceive their environment and take goal-directed actions—have been a cornerstone of artificial intelligence research for decades [ 22 ] . The emergence of Large Language Models (LLMs) has dramatically accelerated this paradigm, enabling general-purpose agentic AI systems capable of sophisticated planning, reasoning, and autonomous action in complex domains [ 20 ] . Given how previous waves of AI innovation have transformed visualization, this agentic turn will inevitably reshape our field. Yet visualization differs fundamentally from other AI application domains through its emphasis on human-centered reasoning, support for open-ended sensemaking, and placement of human analysts at the center of the analytical process. This creates an inherent tension between autonomous agent capabilities and visualization’s human-centered focus, raising questions about agency, delegation, coordination, and responsibility [ 6 ] . How can we design visualization systems that leverage agentic capabilities while preserving human insight and analytical control? What visualization tasks can we confidently delegate to agents? To address these questions, we introduce the concept of agentic visualization (AV): interactive visual analysis systems that incorporate autonomous AI agents while maintaining human agency in the analytical process. Rather than starting from scratch, we analyze existing visualization systems with automatic or semi-automatic capabilities—sometimes described as mixed-initiative systems [ 7 ] —reinterpreting them as early forms of agentic visualization. By examining these systems through an agentic lens, we identify recurring structures that effectively coordinate automated components and human analysts. We formalize these structures as design patterns for agentic visualization that balance automation and analytical control. Design patterns, originating in architecture [ 1 ] and software engineering [ 5 ] , provide a structured vocabulary for describing reusable solutions to common challenges. This approach aligns with human-centered AI principles that emphasize human values, interpretability, and agency in AI-augmented systems [ 2 , 13 ] . We validate our proposed design patterns by identifying their presence across diverse published visualization systems. Each pattern is structured with a descriptive name, the problem it addresses, its solution structure, implementation examples, and associated tradeoffs. To demonstrate practical application, we present a novel example scenario that combines multiple patterns to create a dynamic visualization dashboard surveying an active scientific field where agents with different roles, communication, and coordination mechanisms work together to update progressively. This paper makes the following contributions: (1) The concept of agentic visualization as a framework for designing and analyzing systems that incorporate autonomous agents while preserving human-centered analytical processes; (2) a collection of design patterns for agentic visualization extracted from a systematic analysis of prior literature on automated and semi-automated visualization systems; and (3) a demonstration showing how these patterns can be applied to a novel analytical scenario, including their composition into higher-order solutions. Autonomous agents—software entities that perceive their environment and take goal-directed actions—have been a cornerstone of artificial intelligence research since the field’s inception in the 1950s [ 22 ] . The early reactive agents of the 1990s emphasized real-time interaction over complex reasoning, exemplified by Rodney Brooks’ subsumption architecture [ 17 ] , followed by multi-agent systems and cognitive architectures that aimed to model distributed problem-solving and human cognition respectively [ 18 ] . The current resurgence of agent technology, powered by Large Language Models (LLMs), has dramatically accelerated this paradigm, enabling general-purpose agentic AI systems capable of sophisticated planning, reasoning, and autonomous action in complex domains [ 20 ] . This represents a significant shift in capabilities but confronts persistent challenges that historical agent systems also faced: poor generalization across domains, scalability issues with complex tasks, inadequate coordination mechanisms, brittle performance in unexpected situations, and ethical concerns. These limitations highlight an inherent tension in agent design that was famously debated by Ben Shneiderman and Pattie Maes at IUI ’97 and CHI ’97, where Shneiderman advocated for direct manipulation interfaces that “affords the user control and predictability,” while Maes argued for delegation to software agents that could learn user preferences and act on their behalf [ 21 ] . This debate continues to shape the balance between computational agency and human control in interactive systems. Addressing these challenges requires more than just technical improvements to agent capabilities. As Shah and White argue, successful agent systems need to generate sufficient value for users, provide adaptable personalization, establish trustworthiness, achieve social acceptability, and develop standardization protocols. Aridor and Lange’s work on design patterns for agent systems provides a useful foundation for capturing common solutions to recurring problems in agent design [ 16 ] . Their patterns are conceptually divided into three classes: • Traveling patterns: Agents moving and navigating in distributed environments; • Task patterns: Accomplishing agent objectives and distributing workloads; and • Interaction patterns: Governing communication and coordination between agents. Recent work has extended this approach to foundation model-based agents, with researchers developing catalogs of architectural patterns that address challenges such as hallucinations, explainability, and accountability [ 19 ] . RELATED WORK The notion of agents lies on a spectrum of varying degrees of their inherent agency over their own actions. On one end of that spectrum, agents are explicitly called upon by the user and given the autonomy to perform tasks under changing and unanticipated conditions without a user’s oversight. This “autonomous systems” approach is embodied in early visualization research dating back to the 1990s. It can be found in concepts like Smart Particles that are sent out into the data space, travel through it, and in their wake leave behind visual elements that make the data space visible as the swarm of particles wanders across it [ 10 ] . Another example is the idea of Design Agents with different strategies and abilities and which by interacting with each other, yield visualizations as an emergent joint outcome that automatically adjusts to underlying data changes as the agents react to them [ 8 ] . The next step on that spectrum are agents that continuously work alongside the users without being explicitly triggered. They silently observe the users’ actions and offer unobtrusive help, possibly even extrapolating their needs and speculatively carrying out anticipated tasks ahead of time. This echoes ideas such as Mark Weiser’s Calm Computing , sometimes likened

Read the complete text