DeTAILS: Deep Thematic Analysis with Iterative LLM Support DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06
Ansh Sharma, Karen Cochrane, James R. Wallace · 2025
Read DeTAILS: Deep Thematic Analysis with Iterative LLM Support DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 by Ansh Sharma, Karen Cochrane, James R. Wallace (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17575.txt.
Introduction
Sorting interview and discussion material into themes is core qualitative research work, but it is slow and hard to scale. Ansh Sharma, Karen Cochrane, and James R Wallace introduce DeTAILS, a toolkit that weaves large language model help into the classic stages of thematic analysis. The system assists with drafting codes, grouping them, and shaping themes, while the researcher stays in charge through feedback loops at each step.
Eighteen qualitative researchers tried the toolkit on social media discussion data. Sharma and his co-authors report that the AI supported outputs aligned strongly with what the researchers themselves settled on, while the workload felt lighter and the tool was rated highly useful. Participants said the system sped up their work and pushed them to reflect on the AI suggestions rather than accept them blindly.
Catalog record
| CREATOR | Ansh Sharma, Karen Cochrane, James R. Wallace |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-17575
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DeTAILS: Deep Thematic Analysis with Iterative LLM Support DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 Ansh Sharma, Karen Cochrane, James R. Wallace Abstract . Thematic analysis is widely used in qualitative research but can be difficult to scale because of its iterative, interpretive demands. We introduce DeTAILS, a toolkit that integrates large language model (LLM) assistance into a workflow inspired by Braun and Clarke’s thematic analysis framework. DeTAILS supports researchers in generating and refining codes, reviewing clusters, and synthesizing themes through interactive feedback loops designed to preserve analytic agency. We evaluated the system with 18 qualitative researchers analyzing Reddit data. Quantitative results showed strong alignment between LLM-supported outputs and participants’ refinements, alongside reduced workload and high perceived usefulness. Qualitatively, participants reported that DeTAILS accelerated analysis, prompted reflexive engagement with AI outputs, and fostered trust through transparency and control. We contribute: (1) an interactive human–LLM workflow for large-scale qualitative analysis, (2) empirical evidence of its feasibility and researcher experience, and (3) design implications for trustworthy AI-assisted qualitative research. 1. Introduction The growing scale of online datasets, such as social media posts, has created a pressing need for methods that can support efficient yet rigorous analysis ( Chandrasekar et al. 2024 ) . Qualitative research methods such as thematic analysis (TA) ( Braun and Clarke 2006 ) enable scholars to understand meanings, experiences, and perspectives, producing rich insights into complex social phenomena ( Denzin and Lincoln 2011 ) and situating findings within broader cultural or situational contexts ( Creswell and Poth 2016 ) . These approaches are particularly valuable for describing behaviour, exploring lived experience, and identifying concepts or variables that might otherwise be overlooked ( Rice 1996 ; Patton 2014 ) . However, despite these strengths, TA is time- and labour-intensive, and scaling it to large corpora presents practical challenges for consistency and depth ( Braun and Clarke 2021a ; Guest et al. 2012 ; Jiang et al. 2021 ) . Importantly, TA is not a single technique but a family of approaches ( Creswell and Poth 2016 ) . Reflexive Thematic Analysis (RTA), developed by Braun and Clarke, emphasizes the researcher’s interpretive role, the evolving and flexible nature of codes, and reflexivity as a way to acknowledge how analytic decisions shape findings ( Braun and Clarke 2019 ; Ayton 2023 ) . In contrast, codebook TA stabilizes codes into more structured frameworks, supporting consistency, rigour, and team-based analysis ( Boyatzis 1998 ; Roberts et al. 2019 ) . Both approaches are widely used, but they carry different commitments to reflexivity, flexibility, and replicability, which become particularly salient when scaling analysis to large datasets. Large language models (LLMs) have recently been proposed as tools to help meet these scaling challenges. Their ability to generate, classify, and summarize text makes them promising assistants for coding and theme development ( Deiner et al. 2024 ; Tai et al. 2024 ) . Yet existing AI-assisted qualitative tools often produce outputs that are generic or decontextualized, require extensive refinement, and operate through opaque processes that undermine trust ( Burrell 2016 ; Feuston and Brubaker 2021 ; Jiang et al. 2021 ) . Analysts frequently prefer manual methods when reflexivity and interpretive authority are central ( Braun and Clarke 2019 ) . As a result, despite the promise of LLMs, their adoption in qualitative research remains limited, with open questions around how to balance efficiency with transparency, trust, and interpretive depth. To address these challenges, we developed DeTAILS: Deep Thematic Analysis with Iterative LLM Support , a researcher-centred toolkit inspired by Braun and Clarke’s six phases of analysis. Although originally motivated by RTA, the scale of Reddit datasets required design choices such as editable but stabilized codebooks, which ultimately aligned the workflow more closely with codebook TA ( Boyatzis 1998 ; Roberts et al. 2019 ) . Reflecting on our evaluation, however, we observed that reflexivity remained central: participants frequently verified, critiqued, and reworked AI outputs, treating this engagement as an integral part of their analysis. Our evaluation with 18 qualitative researchers demonstrated both the potential and the tensions of this approach. DeTAILS substantially accelerated coding and theme development and was valued for its structured workflow and support for researcher control. At the same time, participants noted areas for improvement: reflexivity often centred on verifying AI outputs rather than interrogating analytic assumptions, novices at times over-trusted fluent outputs, and the workflow risked constraining interpretive flexibility. Through our own analysis of these interviews, using a codebook-style approach similar to what DeTAILS ultimately supported, we developed themes that informed a set of design recommendations for future iterations. These findings showcase the system’s promise while also highlighting what needs to change in order to better align with reflexive thematic analysis. This paper makes three contributions to HCI and qualitative research. First, it introduces DeTAILS, an interactive toolkit that integrates LLM support across thematic analysis phases while balancing acceleration with researcher control. Second, it provides a nuanced empirical account of how qualitative researchers of varying expertise engaged with LLM assistance, highlighting both efficiencies and tensions around agency, reflexivity, and trust. Third, it offers design implications for the next generation of AI-assisted qualitative tools, including opt-in AI involvement, iterative workflows that support both forward and backward reflection, and safeguards against over-trust. In doing so, this work addresses current gaps in how AI systems are applied to qualitative research, where existing tools often privilege efficiency over reflexivity, provide opaque or decontextualised outputs, and leave unanswered questions about trust and adoption. Our findings demonstrate both the potential and the limits of current approaches, and chart a path toward systems that scale qualitative analysis without eroding its interpretive commitments. 2. Related Work Thematic Analysis (TA) is a foundational qualitative method for identifying patterns of meaning (“themes”) within textual data ( Attride-Stirling 2001 ; Boyatzis 1998 ; Braun and Clarke 2006 ) . Reflexive TA (RTA), developed by Braun and Clarke ( Braun and Clarke 2019 ) , has become especially influential. In RTA, the researcher plays an active, interpretive role in theme development rather than applying a fixed codebook or pursuing inter-coder reliability ( Terry et al. 2017 ) . Codes and themes are not pre-defined but emerge through close engagement with the data and ongoing critical reflection ( Braun and Clarke 2006 ; Nowell et al. 2017 ; Saldana 2021 ) . Reflexivity is central: researchers acknowledge their subjectivity and positionality, recognising how analytic decisions shape knowledge production ( Lincoln et al. 1985 ; Morrow 2005 ) . This interpretive freedom allows for the generation of rich, situated insights but also demands rigour in documenting decisions and engaging critically with one’s own role in the analysis ( Finlay 2002 ; Lincoln et al. 1985 ; Nowell et al. 2017 ) . These qualities made RTA an attractive starting point for our work. We sought to design a system that could preserve interpretive depth and reflexivity while helping researchers handle larger, more complex corpora such as social media data. Yet RTA is inherently time- and labour-intensive, particularly at scale ( Guest et al. 2012 ; Nelson 2017 ; Nowell et al. 2017 ) . Researchers must develop deep