Reflections on Traceability for Visualization Research Volume: 45 Issue: 3
J. Rogers 1 D. Akbaba 2 J. S. Brown 3 A. Lex 4,5 and M. Meyer 6 1 Idaho National Lab, USA 2 KTH Royal Institute of Technology, Sweden 3 University of Edinburgh, UK 4 Graz University of Technology, Austria 5 University of Utah, USA 6 Linköping University, Sweden · 2026
Read Reflections on Traceability for Visualization Research Volume: 45 Issue: 3 by J. Rogers 1 D. Akbaba 2 J. S. Brown 3 A. Lex 4,5 and M. Meyer 6 1 Idaho National Lab, USA 2 KTH Royal Institute of Technology, Sweden 3 University of Edinburgh, UK 4 Graz University of Technology, Austria 5 University of Utah, USA 6 Linköping University, Sweden (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14417.txt.
Introduction
Rogers and four colleagues argue that design-oriented visualization research needs traceability, not reproducibility, since insight grows through subjective, situated, iterative work. They define traceability as showing where contributions came from and how they grew, using rich artifacts from the design process. A joint self-study plus a tool called tRRRacer test three tenets: record abundant annotated artifacts, report curated threads of rationale and change, and read through views that retrace claims. The paper theorizes traceability and reflects on how to support it.
Catalog record
| CREATOR | J. Rogers 1 D. Akbaba 2 J. S. Brown 3 A. Lex 4,5 and M. Meyer 6 1 Idaho National Lab, USA 2 KTH Royal Institute of Technology, Sweden 3 University of Edinburgh, UK 4 Graz University of Technology, Austria 5 University of Utah, USA 6 Linköping University, Sweden |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58181 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.14417
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reflections on Traceability for Visualization Research Volume: 45 Issue: 3 J. Rogers 1 D. Akbaba 2 J. S. Brown 3 A. Lex 4,5 and M. Meyer 6 1 Idaho National Lab, USA 2 KTH Royal Institute of Technology, Sweden 3 University of Edinburgh, UK 4 Graz University of Technology, Austria 5 University of Utah, USA 6 Linköping University, Sweden Abstract Decades of advocacy for reproducibility and replication have advanced open, transparent practices in the sciences. However, traditional notions of reproducibility fit poorly with design-oriented visualization research, where insights emerge through subjective, situated, and iterative work. So how can we ensure rigor and transparency in processes that are inherently unreproducible? To introduce transparency in design-oriented research, we propose to focus on traceability: surfacing the origin and development of research contributions based on rich sets of artifacts documenting the design process. We investigated traceability through a collaborative autoethnographic reflection that builds on several years of work exploring ways to make design-oriented research transparent. This exploration includes an experiment to build a tool to support traceability, which we called tRRRacer. The tRRRacer tool provided a testbed for us to operationalize the three tenets of a traceable process: (1) Record abundant, annotated artifacts representative of research activities; (2) Report curated research threads that articulate rationale and evolution of the process, allowing others to (3) Read via interfaces that help retrace claims and assess plausibility. Reflecting on our experiences, we contribute a theorization of traceability and reflections on how we might support it. \SpecialIssuePaper \CGFccby ccs Human-centered computing Visualization design and evaluation methods † † year: 2026 † † year: 2026 † † editors: N. Andrienko, M. Chen, and B. Wang † † subject: Eurographics Conference on Visualization CGF Vol No † † orcid: 0000-0002-9568-469X † † orcid: 0000-0001-9419-3402 † † orcid: 0000-0001-5642-8346 † † orcid: 0000-0001-6930-5468 † † orcid: 0000-0002-8971-6245 1 Introduction Reproducibility and replication of empirical scientific research are prominent focuses and concerns in computer science [ Nat19 ] . Reproducing and replicating results provides a research community with the means to validate and trust their truthfulness. The visualization research community has championed reproducibility and replication for decades, producing a shift towards open research practices, more access to data, preregistration of studies, and an increased valuing of replication studies [ FF20 , BBD∗21 , NEDM18 ] . However, the field of visualization — like computer science more broadly — is increasingly methodologically diverse with a plurality of epistemic foundations and approaches. Today, many visualization methods of inquiry include subjective and reflective practices, such as design methods and qualitative analysis, making them ill-suited to traditional reproducibility and replication requirements. In considering subjective and reflective research, scholars in the social sciences and design research communities instead advocate for transparency and scrutinizability so that others can assess the credibility, plausibility, and rigor of the research results [ LG85 , Tra10 , ZF08 ] . But what does it mean for design-oriented visualization research to be transparent? How do we record and report reflective processes? What is — or could be — the evidence of iterative, subjective, and reflective research methods? And how do we make this evidence legible to others? In this work, we propose a theoretical framing called traceability to support the transparency and scrutinizability of non-reproducible visualization research. This framing is grounded in the concept of material traces from science and technology studies (STS), which theorizes that objects in the world are embedded with traces of their making. From this position, we articulate a set of concepts for visualization research — research artifacts and research threads — for making design and reflective work in research legible to others. This theoretical framing originates from a collaborative autoethnographic reflection [ LCB∗25 ] on a design experiment in which we created a tool for supporting traceability. Called tRRRacer, this tool was designed to support the core tasks of tracing: recording, reporting, and reading. Although our experiment did not result in a tool for public dissemination, it did provide us with new ways to think of how we might support traceability in the future. The core contribution of this work is a theoretical conceptualization of traceability intended as a generative framing for reasoning about recording and reporting visualization research. We ground this conceptualization in theories from STS as well as a design experiment in which we designed, developed, and tested a new tool for recording and reporting on visualization research. Reflecting on our successes and failures in the experiment, we propose speculative future directions to support the traceability of visualization research and question how much transparency is enough. We see this paper as part of a continuing discussion on visualization design, open practices, and rigor across an increasingly diverse set of visualization research methods. 2 Background The ability to reproduce and replicate scientific results is a hallmark of good science, providing a means to validate the truthfulness of studies and, ultimately, to establish confidence and trust to build upon others’ prior work. The inability to reproduce and replicate vast numbers of published scientific studies, however, has gained attention in both the scientific literature and popular media. In response to this replication crisis , the US government commissioned a study in 2017 to assess the reproducibility and replicability of scientific and engineering research. The study’s final report defines reproducibility as the ability of a second researcher to recompute original results through the “availability of data, code, and methods that make that recomputation possible” [ Nat19 ] . Replication is defined similarly, but using independently obtained data. The report acknowledges, however, that not all scientific research is replicable, recognizing that “reproducibility and replicability are not, in and of themselves, the end goals of science, nor are they the only way in which scientists gain confidence in new discoveries” [ Nat19 ] . A study by Raghupathi et al. looked at reproducibility characteristics across a corpus of computer science studies and found that the multidisciplinary nature of the field, along with the development and validation of artifacts in socio-technical contexts, contributes to problems with reproducibility [ RRR22 ] . A critical issue their study highlights is the subjective and largely undocumented nature of technology design and development: practices that are poorly understood and do not fit neatly into normative frameworks for verifying the validity of hypothesis-driven scientific research. Visualization researchers have a long history of advocating for making visualization research reproducible [ SFC07 , FF20 , BBD∗21 ] , as well as in developing provenance tools for supporting the reproducibility of visual data analysis [ XOW∗20 ] . Provenance tools focuses on addressing the questions what happened and when in an interactive visual analysis session [ RESC16 , CGL20 , GLG∗16a , SFC07 , SGP∗18 , CZGR09 ] . This work is important in the context of this paper for understanding current practices and tools for tracing the iterative work of data analysts and creating readable traces for others [ GLG∗16a , WAF22 ] . This work, however, focuses on making visual analysis processes transparent during the use of a visualization tool, while the work we present in this paper highlights the unique considerations and challenges of tracing the subjective and undocumented practices of visualization design . Visualization researchers have argued