Semantic Interaction for Narrative Map Sensemaking: An Insight-based Evaluation
Brian Felipe Keith-Norambuena, Fausto German, Eric Krokos, Sarah Joseph, Chris North · 2026
Read Semantic Interaction for Narrative Map Sensemaking: An Insight-based Evaluation by Brian Felipe Keith-Norambuena, Fausto German, Eric Krokos, Sarah Joseph, Chris North (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-29651.txt.
Introduction
Keith-Norambuena and co-authors test semantic interaction for sense-making with narrative maps. Analysts steer AI models by moving items directly in a visual display. The study ran 33 people under three setups. One setup was a timeline, one a plain map, and one a map with semantic interaction. Map users found more insights than timeline users. The interactive group had the highest mean score. Two usage styles appeared: corrective moves and additive ones. Users also refined models with less parameter tuning.
Catalog record
| CREATOR | Brian Felipe Keith-Norambuena, Fausto German, Eric Krokos, Sarah Joseph, Chris North |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33626 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.29651
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Semantic Interaction for Narrative Map Sensemaking: An Insight-based Evaluation Brian Felipe Keith-Norambuena, Fausto German, Eric Krokos, Sarah Joseph, Chris North Abstract Semantic interaction (SI) enables analysts to incorporate their cognitive processes into AI models through direct manipulation of visualizations. While SI frameworks for narrative extraction have been proposed, empirical evaluations of their effectiveness remain limited. This paper presents a user study that evaluates SI for narrative map sensemaking, involving 33 participants under three conditions: a timeline baseline, a basic narrative map, and an interactive narrative map with SI capabilities. The results show that the map-based prototypes yielded more insights than the timeline baseline, with the SI-enabled condition reaching statistical significance and the basic map condition trending in the same direction. The SI-enabled condition showed the highest mean performance; differences between the map conditions were not statistically significant but showed large effect sizes ( d > 0.8 d>0.8 ), suggesting that the study was underpowered to detect them. Qualitative analysis identified two distinct SI approaches—corrective and additive—that enable analysts to impose quality judgments and organizational structure on extracted narratives. We also find that SI users achieved comparable exploration breadth with less parameter manipulation, suggesting that SI serves as an alternative pathway for model refinement. This work provides empirical evidence that map-based representations outperform timelines for narrative sensemaking, along with qualitative insights into how analysts use SI for narrative refinement. [orcid=0000-0001-5734-8962, [email protected] ] *1 [orcid=0009-0005-0954-4578 ] [] [] [orcid=0000-0002-8786-7103 ] keywords Semantic Interaction ,Narrative Extraction ,Narrative Visualization ,Sensemaking ,Visual Analytics † † copyrightyear: 2026 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: In: R. Campos, A. Jorge, A. Jatowt, S. Bhatia, M. Litvak (eds.): Proceedings of the Text2Story’26 Workshop, Delft (The Netherlands), 29-March-2026 † † address: Universidad Católica del Norte, Av. Angamos 0610, Antofagasta, 1270709, Chile † † address: Virginia Tech, 620 Drillfield Drive, Blacksburg, VA, 24061, USA † † address: U.S. Government, Washington, D.C. 20500, USA † † corresp: Corresponding author. 1 Introduction Visual analytics (VA) seeks to support the analysis of large and complex data sets through interactive exploration 5 . VA systems assist users in their sensemaking process by allowing them to interact with statistical and artificial intelligence (AI) models through visualizations to explore connections and hypotheses in the data 8 . Semantic interaction (SI) 8 ; 4 enables users to incorporate their cognitive processes into the underlying computational models through intuitive interactions with visualizations. Rather than requiring direct parameter manipulation, SI allows analysts to perform natural interactions that are interpreted and translated into model updates. This approach has shown promise in various VA domains, and recent work has developed SI frameworks specifically for narrative map extraction 17 . Narrative maps—graph-based representations that capture connections between events in a story—are a useful tool for narrative sensemaking 18 . However, while SI frameworks for narrative maps have been proposed, empirical evaluations of their effectiveness in supporting real sensemaking tasks remain limited. Understanding how analysts actually use SI capabilities, what strategies they develop and whether SI leads to better analytical outcomes requires user studies. In this paper, we present a user study that evaluates semantic interaction for narrative map sensemaking. We compare an SI-enabled prototype with baseline conditions using 33 participants examining narratives from the 2021 Cuban protests. Our study employs an insight-based evaluation methodology 21 to assess how effectively SI supports the sensemaking process. Our contributions include: (1) empirical evidence that map-based visualizations significantly outperform timeline baselines, with suggestive but not statistically significant additional benefits from SI; (2) identification of distinct SI strategies—corrective and additive—that analysts employ for narrative refinement; (3) analysis of SI’s effect on exploration behavior, suggesting it serves as an alternative to parameter-based refinement; and (4) practical insights for SI system design derived from observed user strategies and perceptions. The rest of this paper is organized as follows. We review related work in Section 2 and describe the SI-enabled narrative map system in Section 3 . Section 4 presents the design of the user study, followed by the results in Section 5 . We discuss implications and limitations in Section 6 and conclude in Section 7 . 2 Related Work In this work, we combine two lines of research: narrative maps , which computationally extract graph-based story representations from document collections, and semantic interaction , which allows analysts to refine underlying models by directly manipulating visualizations. We review each in turn, along with the evaluation methodology we adopt. Narrative Extraction and Visualization. Narratives can be defined as systems of interrelated storylines that share coherent themes 12 . Computational narrative extraction typically relies on event-based models using discrete structures such as timelines or directed acyclic graphs (DAGs) 16 . Narrative maps 18 represent narratives as DAGs where nodes are events and edges represent narrative connections, similar to other narrative representations 24 ; 2 . Previous work has established the utility of narrative maps for sensemaking tasks 15 ; 17 . However, empirical evaluations comparing interactive narrative map systems with baselines in realistic sensemaking scenarios remain limited. Semantic Interaction. SI 8 enables users to communicate analytical intent through intuitive visualization interactions rather than direct parameter manipulation. SI has been applied in document analysis 8 , spatialization 4 , high-dimensional data exploration 7 , and text analytics 23 ; 13 . Keith et al. 17 proposed a mixed multi-model SI framework specifically for narrative maps, addressing challenges of combining discrete graph structures with continuous embedding spaces. Our work builds on this framework by providing empirical evaluation of its effectiveness. Evaluating Visual Analytics Systems. Insight-based evaluation 22 ; 21 assesses VA systems by measuring the insights analysts generate rather than task completion metrics. This approach better captures the effectiveness of the system for exploratory sensemaking 11 and accommodates diverse analytical strategies 1 . We adopt this methodology to evaluate the effectiveness of SI for narrative sensemaking. 3 System Description We evaluate a narrative map system with semantic interaction capabilities, built upon the mixed multi-model SI framework proposed by Keith et al. 17 . This section briefly describes the key components of the system that are relevant to understanding the user study. Narrative Extraction. Our work falls within the event-based narrative extraction paradigm 16 , where each document represents a single event and a narrative is a temporally ordered, thematically coherent sequence of such document-events. The system extracts narrative maps from news article collections following the approach of Keith and Mitra 18 . Articles are represented using sentence-level Transformer embeddings (all-MiniLM-L6-v2) aggregated to produce document-level representations, projected to 2D using Uniform Manifold Approximation and Projection (UMAP) 20 for visualization, and clustered using Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) 19 to identify topical groups. A linear programming optimization then constructs the narrative map as a directed acyclic graph by maximizing coherence—a measure combining content similarity (via embedding cosine similarity) and topical relatedness (based on cluster assignments)—subject to coverage constraints that ensure that the narrative spans diverse topics. Users can adjust three parameters: map size (expected length of the main story), coverage (topic diversity), and temporal sensitivity (preference for temporally proximate connections). Semantic Interaction Capabilities. The SI-enabled prototype allows analysts to influence the underlying