SASAV: S elf-Directed A gent for S cientific A nalysis and V isualization

\authororcid Jianxin Sun0000-0002-9627-9397, \authororcid David Lenz0000-0002-2587-2783, \authororcid Tom Peterka0000-0002-0525-3205, and \authororcid Hongfeng Yu0000-0002-0596-8227 · 2026

Read SASAV: S elf-Directed A gent for S cientific A nalysis and V isualization by \authororcid Jianxin Sun0000-0002-9627-9397, \authororcid David Lenz0000-0002-2587-2783, \authororcid Tom Peterka0000-0002-0525-3205, and \authororcid Hongfeng Yu0000-0002-0596-8227 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03406.txt.

Introduction

Sun and colleagues present SASAV, the first fully autonomous agent for scientific data analysis and display without expert prompting or human-in-the-loop feedback. The multi-agent system profiles data on its own, retrieves context-aware knowledge, and explores display settings through reasoning. By removing reactive, data-dependent guidance, SASAV aims to scale scientific discovery and innovation.

Catalog record

CREATOR\authororcid Jianxin Sun0000-0002-9627-9397, \authororcid David Lenz0000-0002-2587-2783, \authororcid Tom Peterka0000-0002-0525-3205, and \authororcid Hongfeng Yu0000-0002-0596-8227
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52661
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.03406

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SASAV: S elf-Directed A gent for S cientific A nalysis and V isualization \authororcid Jianxin Sun0000-0002-9627-9397, \authororcid David Lenz0000-0002-2587-2783, \authororcid Tom Peterka0000-0002-0525-3205, and \authororcid Hongfeng Yu0000-0002-0596-8227 Abstract With recent advances in frontier multimodal large language models (MLLMs) for data understanding and visual reasoning, the role of LLMs has evolved from passive LLM-as-an-interface to proactive LLM-as-a-judge, enabling deeper integration into the scientific data analysis and visualization pipelines. However, existing scientific visualization agents still rely on domain experts to provide prior knowledge for specific datasets or visualization-oriented objective functions to guide the workflow through iterative feedback. This reactive, data-dependent, human-in-the-loop (HITL) paradigm is time-consuming and does not scale effectively to large-scale scientific data. In this work, we propose a Self-Directed Agent for Scientific Analysis and Visualization (SASAV), the first fully autonomous AI agent to perform scientific data analysis and generate insightful visualizations without any external prompting or HITL feedback. SASAV is a multi-agent system that automatically orchestrates data exploration workflows through our proposed components, including automated data profiling, context-aware knowledge retrieval, and reasoning-driven visualization parameter exploration, while supporting downstream interactive visualization tasks. This work establishes a foundational building block for the future AI for Science to accelerate scientific discovery and innovation at scale. \onlineid 0 \vgtccategory Research \vgtcpapertype please specify \authorfooter Jianxin Sun and Hongfeng Yu are with the University of Nebraska-Lincoln. E-mail: [email protected], [email protected]. David Lenz and Tom Peterka are with Argonne National Laboratory. E-mail: [email protected], [email protected]. \teaser SASAV workflow to automatically generate visualization results for AbdomenAtlas 1.0 Mini datasets: (a) Direct volume rendering of the input volume; (b) Isosurface rendering of evenly sampled isovalues; (c) Isosurface rendering with color mapping suggested by the semantic analyzer and color transfer function designer; (d) Isosurface rendering with opacity mapping suggested by the opacity transfer function designer; (e) Isosurface rendering from the best viewpoint; (f) The animation rendering from the best viewpoint trajectory. Both the best view and the best exploratory trajectory are suggested by the view selection. Introduction keywords AI agent, Scientific visualization, LLM, AI for science Scientific data analysis involves processing and interpreting raw data to extract meaningful patterns, relationships, and insights, while scientific visualization transforms that data into visual representations that humans can easily understand. Scientific visualization is a critical analytical tool that enables domain scientists to uncover meaningful features in complex datasets across a wide range of fields, including medical imaging, geophysics, meteorology, materials science, and physical simulations. Traditional scientific visualization systems are grounded in human-centered design to facilitate data exploration and support interactive workflows, leveraging techniques such as visual encoding, dimensionality reduction, level-of-detail (LOD) control, feature extraction, and transfer function design [ 42 , 11 ] . Recent agentic AI systems demonstrate strong potential for intuitive interaction, data understanding, and visual reasoning by leveraging rapidly advancing frontier multimodal large language models (MLLMs) [ 12 ] . Figure 1 : Evolution of AI for Science Existing Scientific Visualization (SciVis) agents basically function as reactive tools, relying on human-in-the-loop (HITL) input to provide prior knowledge of the target dataset and heuristic to direct data exploration. SciVis code generation LLM assistants, such as ChatVis [ 32 ] , rely on users to provide a complete and specific description of visualization parameters, including the viewpoints, transfer functions (TFs), lighting, and visualization method, in order to converge through an iterative image metric-based self-correction. For interactive visualization of scientific data, LLM-based visualization approaches, such as NLI4VolVis [ 3 ] and ParaView-MCP [ 23 ] , require a clear visualization task description and continuous feedback from domain expert to guide the data exploration. Those iterative trial-and-error feedback loops with HITL is time-consuming, subjective, and infeasible when handling large-scale scientific data [ 31 ] with extensive visualization parameters. MLLMs have demonstrated strong capabilities in image understanding and analysis, with perceptual abilities that closely align with human visual assessment [ 7 , 8 , 48 ] . Recent work on visualization parameter optimization seeks to automate the search for key visualization parameters, such as the color and opacity transfer function, by employing an LLM-as-a-judge [ 16 ] to provide feedback on a predefined objective function, leveraging the visual reasoning capability of the frontier MLLMs. AVA [ 22 ] provides an automatic adjustment of the opacity TF according to the feedback from MLLMs, which evaluate the recognizability and clarity of intermediate renderings. Similarly, IntuiTF [ 45 ] introduces a framework that leverages MLLMs to define a loss metric for downstream evolutionary optimization, enabling the automatic exploration and evaluation of transfer functions for volume rendering. However, those methods suffer from the following issues: 1) Both AVA and IntuiTF remain semi-autonomous, as they still depend on users to specify clear visualization objectives: AVA requires a textual prompt, such as action instructions or visual perception prompts, while IntuiTF requires user text or visual intent. 2) There are two types of scientific data: empirical data and simulated data. Empirical data are scientific data measured or reconstructed from real-world objects, originating from physical reality with rich semantic meaning. Simulated data are scientific data generated by computational models that simulate physical, chemical, or biological processes. Existing methods are more effective for empirical data, as it is easier for users to provide semantically clear prompts to MLLMs, whereas simulated data is often difficult to interpret due to its blurry, distribution-like appearance. 3) Existing TF refinement is carried out in an extremely large and high-dimensional search space through a single-threaded iterative loop that queries MLLMs for ad hoc evaluations [ 32 , 23 , 3 ] , leading to slow convergence and limited generalizability. 4) The visualization–perception–action loop adopted in existing methods tends to converge to local optima, leading to incomplete visualizations that miss important regions of interest and compromise the information richness [ 24 ] . All the aforementioned issues limit the usability and scalability of MLLM-based agentic workflow for scientific visualization tasks, particularly when dealing with real-world, large-scale scientific data. In this work, we propose SASAV, a self-directed agent for scientific analysis and visualization, which, to the best of our knowledge, is the first fully autonomous and zero-prompting agentic workflow for scientific visualization. SASAV provides a unified solution for visualizing various types of scientific data, where prior knowledge is not available to provide a specific task description. Our solution leverages the visual reasoning and comparison capabilities of the frontier MLLMs, enabling comprehensive, systematic, and efficient parameter optimization for visualization. We employ a multi-agent workflow with parallel and batched MLLM inference to enable efficient exploration of visualization parameters. SASAV consists of 4 key workflow components (data profiling, knowledge retrieval, TFs suggestion, and view selection) and directly generates 3 visualization forms (static visualization image, animation, and interactive visualization) to best help domain experts to accelerate scientific discovery. Our work is not designed to replace existing interactive visualizations, but rather to provide a comprehensive and informative starting point that facilitates more efficient exploration, ultimately accelerating scientific discovery and innovation. We see our work as one of the foundational building blocks for establishing the level 3 (fully agentic discovery) stage during the evolution of AI for Science [ 46 ] , which is shown in Fig. 1 . The demonstration, evaluation, and source code are available at https://selfdirectedscivisagent.github.io . The main contributions of the proposed fully autonomous self-directed AI

Read the complete text