The Shape of Agency: Designing for Personal Agency in Qualitative Data Analysis DOI: XXXXXXX.XXXXXXX Conference: ACM Conference on Human Factors in Computing Systems; April 26– May 1, 2025; Yokohama, Japan

Luka Ugaya Mazza, Plinio Morita, James R. Wallace · 2024

Read The Shape of Agency: Designing for Personal Agency in Qualitative Data Analysis DOI: XXXXXXX.XXXXXXX Conference: ACM Conference on Human Factors in Computing Systems; April 26– May 1, 2025; Yokohama, Japan by Luka Ugaya Mazza, Plinio Morita, James R. Wallace (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-14481.txt.

Introduction

Luka Ugaya Mazza, Plinio Morita, and James Wallace explore, in 2024, how to keep researchers in control when computers help analyze interview and text data. Computational thematic analysis can handle large collections, but many qualitative researchers cannot code and do not want to hand their work to an opaque system.

Using a design science approach over four phases, the team built a visualization prototype with health researchers. Users held wide-ranging needs and prized choice and freedom in their analysis. They wanted to feel ownership of the work, not replaced or blocked by artificial intelligence. The findings frame agency as a design goal in its own right.

Catalog record

CREATORLuka Ugaya Mazza, Plinio Morita, James R. Wallace
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-14481

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Shape of Agency: Designing for Personal Agency in Qualitative Data Analysis DOI: XXXXXXX.XXXXXXX Conference: ACM Conference on Human Factors in Computing Systems; April 26– May 1, 2025; Yokohama, Japan Luka Ugaya Mazza, Plinio Morita, James R. Wallace Abstract . Computational thematic analysis is rapidly emerging as a method of using large text corpora to understand the lived experience of people across the continuum of health care: patients, practitioners, and everyone in between. However, many qualitative researchers do not have the necessary programming skills to write machine learning code on their own, but also seek to maintain ownership, intimacy, and control over their analysis. In this work we explore the use of data visualizations to foster researcher agency and make computational thematic analysis more accessible to domain experts. We used a design science research approach to develop a datavis prototype over four phases: (1) problem comprehension, (2) specifying needs and requirements, (3) prototype development, and (4) feedback on the prototype. We show that qualitative researchers have a wide range of cognitive needs when conducting data analysis and place high importance upon choices and freedom, wanting to feel autonomy over their own research and not be replaced or hindered by AI. 1. Introduction Qualitative research enables researchers to understand the meanings, experiences, and views of participants, giving voice to the lived experiences of people. It provides a deep understanding of complex phenomena by exploring subjective experiences in rich detail ( Denzin and Lincoln 2011 ) and placing findings within their social, cultural, or situational contexts ( Creswell and Poth 2016 ) . Rice 1996 paints qualitative research as important for describing and explaining behaviour, understanding how people perceive their own health experience, and how to best design for those that will use it. It can be particularly important when seeking to identify new variables or concepts that might otherwise be overlooked, while capturing emotions and subjective perspectives—dimensions often neglected in purely numerical data ( Patton 2014 ) . However, qualitative analysis can also be labour-intensive, rigorous, and exhausting ( Jiang et al. 2021 ; Pope et al. 2000 ) . It can take weeks for researchers to manually curate and analyze data sets and this process is often complicated by a lack of available time ( Jiang et al. 2021 ; Feuston and Brubaker 2021 ) . These challenges are further exacerbated when considering the vast amounts of data available to researchers wishing to engage with online sources like social networking sites (e.g. ( Gauthier and Wallace 2022 ; Gauthier et al. 2022b ) ). When considered together, these challenges mean that qualitative researchers are often faced with a lack of agency when working with the large data sets available today from sources like social media. Computational thematic analysis seeks to combine established qualitative research methods (e.g., ( Braun and Clarke 2006 ; Boyatzis 1998 ) ) with emerging techniques from machine learning (e.g., ( Blei et al. 2003 ; Vaswani et al. 2017 ; Devlin et al. 2018 ) ). Human-computer interaction research has in turn sought to develop visual interfaces that make these techniques accessible to domain experts who may not have the programming expertise required to perform computational thematic analysis without technical support (e.g., ( Gauthier and Wallace 2022 ; Hong et al. 2022 ) ). However, recent findings suggest that domain experts are unlikely to use tools that they do not trust; they are hesitant to give up intimacy, ownership, and agency in their own analyses ( Jiang et al. 2021 ; Feuston and Brubaker 2021 ) . In this work we explore domain experts’ experiences around using data visualizations as a tool for knowledge translation for computational thematic analysis. Data visualizations can be useful when conducting larger scale research, since they allow for pattern recognition, data exploration, summarization, and external cognition ( Braga et al. 2020 ; Gomes et al. 2019 ; Cabitza et al. 2016 ; Kennedy and Engebretsen 2020 ; Jiang et al. 2021 ; Hong et al. 2022 ) . We engaged in design science research to understand their perceptions of personal agency — which is the positive feeling related to being given the space to act and see those actions shape the world ( Murray 1997 ; Eichner 2014 ) — when working with large data sets, and with computational tools and data visualizations to support their analyses. Our approach included an iterative design exercise which was informed by a close reading of the literature and qualitative data analysis software, interviews with domain experts, low-fidelity prototyping, and follow-up interviews with the same experts to provide feedback on those prototypes. Our results show that personal agency can be used as a design lens for software interfaces. Personal agency provides a novel way of understanding how specific design features can foster agency and thus assist in incorporating them into the interfaces that attend to users’ needs . It is particularly useful when considering human-AI interactions, where loss of agency has been shown to be a primary concern of end users ( Jiang et al. 2021 ) . In summary, we make the following contributions: (1) Show how personal agency as conceptualized by Eichner 2014 can be used as a design lens in HCI (2) Investigate qualitative researchers’ perceptions of how AI can support their work, particularly within the context of thematic analysis of large data sets. (3) We engaged in a co-design exercise in which qualitative researchers helped to design a series of novel, low-fidelity prototypes (4) We reflect on barriers and facilitators to the use of AI in qualitative research, and how qualitative researchers’ own perceptions of AI shifted during the co-design exercise. 2. Related Work The human-computer interaction research community has recently explored the use of social media as a resource to understand people. The insights generated can be quite valuable, for instance, the ability to post on social media anonymously has been found to support disclosure around sensitive topics, such as addiction and recovery ( Gauthier et al. 2022a ) , disordered eating ( Nova et al. 2022 ) , parenting issues ( Ammari et al. 2018 ) , intimate partner violence ( Rodriguez and Storer 2020 ) , and mental health ( Andalibi et al. 2016 ; Xu et al. 2023 ) . These disclosures can be absent in data collected using methods like interviews or surveys since people may hesitate to speak about sensitive or stigmatized issues for privacy reasons or fear of being judged ( Ammari et al. 2018 ) . However, a significant barrier to performing qualitative analysis on social media data is its scale, which requires researchers to sample data for topics of interest. While in the past, researchers have focused on randomized samples, modern computational techniques enable for purposive sampling over large data sets through unsupervised topic modelling ( Gauthier et al. 2022a ; Gauthier and Wallace 2022 ) . In exploring the use of these techniques, the human-computer interaction research community has noted ‘convergences’ ( Muller et al. 2016 ; Baumer et al. 2017 ) between qualitative and machine learning research: they are both grounded in data, interpretive, and iterative in nature. And in further pursuing these convergences, they have explored both technological advances

Read the complete text