Chasing Meaning and/or Insight? A Survey on Evaluation Practices at the Intersection of Visualization and the Humanities Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Facto

Alejandro Benito-Santos, Florian Windhager, Aida Horaniet Ibañez, Rabea Kleymann, Alfie Abdul-Rahman, Eva Mayr · 2026

Read Chasing Meaning and/or Insight? A Survey on Evaluation Practices at the Intersection of Visualization and the Humanities Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Facto by Alejandro Benito-Santos, Florian Windhager, Aida Horaniet Ibañez, Rabea Kleymann, Alfie Abdul-Rahman, Eva Mayr (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20464.txt.

Introduction

Alejandro Benito-Santos and five colleagues survey how research where visualization meets the humanities is evaluated. The field, called VIS*H, is marked by a tension between chasing analytical insight and interpretive meaning. Standard evaluation frameworks judge tools by analytical utility and communicative power.

That yardstick can mismatch the interpretive aims of humanities scholarship, which does not always seek a single right reading. The authors survey evaluation practices across this borderland. Their map can help both sides judge work by goals the work actually has, rather than by imported standards. The field is young, and its rules are still being set by the work now in print.

Catalog record

CREATORAlejandro Benito-Santos, Florian Windhager, Aida Horaniet Ibañez, Rabea Kleymann, Alfie Abdul-Rahman, Eva Mayr
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE110 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-20464

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Chasing Meaning and/or Insight? A Survey on Evaluation Practices at the Intersection of Visualization and the Humanities Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain DOI: 10.1145/3772318.3793150 ISBN: 979-8-4007-2278-3/2026/04 CCS: Human-centered computing Visualization design and evaluation methods CCS: Human-centered computing Visualization theory, concepts and paradigms CCS: Human-centered computing Empirical studies in visualization CCS: Human-centered computing HCI design and evaluation methods CCS: General and reference Surveys and overviews CCS: Applied computing Arts and humanities Alejandro Benito-Santos, Florian Windhager, Aida Horaniet Ibañez, Rabea Kleymann, Alfie Abdul-Rahman, Eva Mayr Abstract . The intersection of visualization and the humanities (VIS*H) is marked by a tension between chasing analytical “insight” and interpretive “meaning.” The effectiveness of visualization techniques hinges on established evaluation frameworks that assess both analytical utility and communicative efficacy, creating a potential mismatch with the non-positivist, interpretive aims of humanities scholarship. To examine how this tension manifests in practice, we systematically surveyed 171 VIS*H design studies to analyze their evaluation workflows and rigor according to standard practice. Our findings reveal recurring flaws, such as an over-reliance on monomethod approaches, and show that higher-quality evaluations emerge from workflows that effectively triangulate diverse evidence. From these findings, we derive recommendations to refine quality and validation criteria for humanities visualizations, and juxtapose them to ongoing critical debates in the field, ultimately arguing for a paradigm shift that can reconcile the advantages of established validation techniques with the interpretive depth required for humanistic inquiry. † † cc-license: by 1. Introduction Digital methods have entered the working repertoire of the arts and humanities, and visualization has become one of their most prominent expressions. 1 1 1 In the absence of an unbiased shorthand for this field (cf. ( 23 ) ), we will refer to it as VIS*H—an acronym that accommodates multiple operationalizations and constellations within a tripolar community of practice, spanning visualization research (VIS), digital humanities (DH), and traditional humanities (H). These include prominent unilateral delivery scenarios such as VIS \rightarrow DH and critical responses VIS \leftarrow H, but also more balanced constellations characterized by mutual learning cycles (VIS \rightleftarrows DH \rightleftarrows H) which represent the field’s inherent promise. This development represents a significant shift within the humanities, as it complements traditional language- and text-based reasoning methods with more formalized and analytical approaches, and also with visual and interactive means of exploring and interpreting complex datasets ( 35 ; 76 ) . Yet, at the core of this shift lies a fundamental tension: while the “insight” derived from pattern discovery or outlier identification is a core strength of visualization ( 111 ; 134 ) , the “meaning” central to the humanities arises from interpretive and discursive acts that require a careful and critical engagement with cultural materials, considering not only their explicit content but also the multiple possible readings related to their motivations and historical contexts ( 40 ; 90 ) —a process that poses a significant challenge for digital formalization ( 42 ; 31 ) . As such, evaluating these visualizations has become a critical area of research ( 23 ; 64 ) , necessitating both a survey of existing evaluation approaches and an exploration of the unique affordances and demands of humanities data and of related scholarly communities. This paper presents an extensive study on the current state of evaluation methods in visualizing humanities data, highlighting prevalent methodologies, but also mapping out future challenges and advancements for this rapidly expanding field. We consider the relevance of this topic to derive from the unique nature of humanities data, conceived here as digital representations that carry both observational and interpretive properties of cultural materials . These materials—and cultures as a whole—can be conceptualized as “human creations that define what is real and important” for those who create, preserve, or share them ( 85 , p.627) . Whether as texts, images, ideas, music, narratives, or movies, cultural materials carry both meaning and value , from small and mundane to collective and historical levels. From a sociological point of view, they cause and catalyze both social cohesion but also conflict by delighting, informing, and connecting collectives—and thus also distancing or antagonizing others ( 73 ; 69 ) . As highly diverse and complex subject matters—whether historical or contemporary—they document the diversity and development of human thinking, feeling, and striving—motivating their study and preservation due to their “aesthetic, historic, scientific, or social value” ( 70 ) . Consequently, datasets about such materials and topics often encompass complex conceptual, qualitative, and value-laden dimensions, in addition to temporal, spatial, or relational characteristics, which are relatively easy to process by formal analytical means. We consider this specific mixture of interpretive and formally tractable features---and related methodological tensions---to be a core reason why the VIS*H field appears as a notably challenging and promising ‘application field’, requiring innovative, context-specific, and (self-)critical approaches to visualization design. Applying standard datafication, formalization, and visualization methods to this field can significantly enhance the work of humanities experts (a stance we will refer to as the ‘‘discovery paradigm’’), but is also frequently seen as not enough (a perspective we label the ‘‘discursive paradigm’’): 2 2 2 We adopt these labels as provisional shorthand, with “discursive” referring to humanities discourse as the site where the core methodological movements of interpretation and critique intertwine (see Sec. 8 ). i) The Discovery Paradigm : The integration of visualization methods in humanities research is often lauded for facilitating a formally deeper and quantitatively more comprehensive understanding of cultural and historical datasets, democratizing access to complex information—from individual artifacts to large corpora—and for making them more comprehensible and engaging to a broader audience. The effectiveness, comprehensibility, and success of these visualizations, however, are contingent upon rigorous evaluation methods that can assess both their analytical utility (including factors like ease of use and task effectiveness) and their ability to communicate insights to diverse audiences. ii) The Discursive Paradigm : Visualizations have, however, also attracted a significant amount of ‘meta-evaluative’ commentary and critique in the humanities, scrutinizing their epistemological and methodological value at various levels. From the outset, debates have emerged around the question of whether visualizations can productively contribute to humanities scholarship in their current form (\mathrm{VIS}\rightarrow H) —including adaptations to the field through user-centered design (\mathrm{VIS}\rightleftarrows H) ---or whether they must undergo significant transformation, to address the substantial challenges 3 3 3 One of these challenges is the need to reconcile computational and visualization-based approaches with the dominant interpretive and discursive practices in the humanities. Other substantial challenges, as elaborated in Section 8 , involve representing the provenance and uncertainty inherent in cultural data and aligning visual designs with the critical and theoretical frameworks that guide humanistic inquiry. arising from its focus on cultural, semiotic, value-laden objects, topics, and materials (\mathrm{VIS}\leftarrow H) ( 23 ; 40 ) . As such, questions persist as to whether visualization and other digital methods should be considered a) as legitimate, core research methods, b) as ancillary, supporting instruments working orthogonally to established humanities methods, or c) as distractions, deformations, or ‘Trojan horses’ that should instead be avoided ( 31 ;

Read the complete text