PVminerLLM: Structured Extraction of Patient Voice from Patient-Generated Text using Large Language Models

Samah Fodeh, Linhai Ma, Ganesh Puthiaraju, Srivani Talakokkul, Afshan Khan, Ashley Hagaman, Sarah Lowe, Aimee Roundtree · 2026

Read PVminerLLM: Structured Extraction of Patient Voice from Patient-Generated Text using Large Language Models by Samah Fodeh, Linhai Ma, Ganesh Puthiaraju, Srivani Talakokkul, Afshan Khan, Ashley Hagaman, Sarah Lowe, Aimee Roundtree (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05776.txt.

Introduction

Fodeh and colleagues build a benchmark and a fine-tuned language model for pulling structured patient voice from patient-written text. Patient accounts carry lived experience, social context, and care engagement that shape adherence, coordination, and equity, yet such signals seldom arrive in structured form. Their system, PVminerLLM, beats prompt-based baselines across several datasets and model sizes, reaching up to 83.82 percent F1 on code prediction. Strong results persist even with smaller models, showing that dependable extraction need not demand extreme scale, with code and models slated for public release.

Catalog record

CREATORSamah Fodeh, Linhai Ma, Ganesh Puthiaraju, Srivani Talakokkul, Afshan Khan, Ashley Hagaman, Sarah Lowe, Aimee Roundtree
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69244
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.05776

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PVminerLLM: Structured Extraction of Patient Voice from Patient-Generated Text using Large Language Models Samah Fodeh, Linhai Ma, Ganesh Puthiaraju, Srivani Talakokkul, Afshan Khan, Ashley Hagaman, Sarah Lowe, Aimee Roundtree Abstract Motivation: Patient-generated text contains critical information about patients’ lived experiences, social circumstances, and engagement in care, including factors that strongly influence adherence, care coordination, and health equity. However, these patient voice signals are rarely available in structured form, limiting their use in patient-centered outcomes research and clinical quality improvement. Reliable extraction of such information is therefore essential for understanding and addressing non-clinical drivers of health outcomes at scale. Results: We introduce PVminer, a benchmark for structured extraction of patient voice, and propose PVminerLLM, a supervised fine-tuned large language model tailored to this task. Across multiple datasets and model sizes, PVminerLLM substantially outperforms prompt-based baselines, achieving up to 83.82% F1 for Code prediction, 80.74% F1 for Sub-code prediction, and 87.03% F1 for evidence Span extraction. Notably, strong performance is achieved even with smaller models, demonstrating that reliable patient voice extraction is feasible without extreme model scale. These results enable scalable analysis of social and experiential signals embedded in patient-generated text. Availability and Implementation: Code, evaluation scripts, and trained LLMs will be released publicly. Annotated datasets will be made available upon request for research use. keywords Large Language Models, Supervised Fine-Tuning, Medical Annotation, Patient-Generated Text, Clinical NLP 1 Introduction Figure 1: Overview of the PVminer Framework’s pipeline: Codebook Development \rightarrow Annotation \rightarrow Prompt Engineering \rightarrow Supervised Fine-tuning Patient-generated data, including secure messages, survey responses, and interview narratives, provide a rare and direct window into patients’ lived experiences outside traditional clinical encounters Howie et al. 2014 ; Huba and Zhang 2012 ; Shapiro et al. 2012 ; Tiase et al. 2020 . Unlike structured clinical records, these texts capture how individuals articulate their needs, constraints, emotions, and expectations in their own words. Collectively, such expressions constitute the patient voice, a socially embedded signal that reflects not only clinical concerns but also broader social, environmental, and relational contexts that shape health outcomes. These contexts include social determinants of health, such as housing instability, and financial insecurity, as well as patients’ engagement, preferences, and participation in managing their care Amineh and Asl 2015 ; Bakhtin 1994 ; Gherlone 2016 ; Fodeh et al. 2026a ; Fodeh et al. 2026b ; Fodeh et al. 2026c . Despite its importance, the patient voice remains largely underexplored in patient-centered outcomes research and health services research. When patient-generated text is analyzed, it is often reduced to narrow, isolated categories or surface-level entities, obscuring the complex and overlapping social realities embedded in everyday language. For example, a single patient message can simultaneously convey medical uncertainty, emotional distress, social constraints, and preferences regarding care decisions. Yet most computational approaches fail to preserve this richness, instead focusing on a limited subset of social factors Roter et al. 2008 ; Ye et al. 2010 ; Rubin et al. 2012 ; Blei and Lafferty 2009 ; Chang et al. 2009 ; Mcauliffe and Blei 2008 ; Wang and Blei 2011 ; Blei and Lafferty 2006 ; Blei 2012 ; Wallach 2006 ; Mikolov et al. 2013 ; Le and Mikolov 2014 ; Graves and Schmidhuber 2005 . As a result, critical social signals that influence adherence, care coordination, partnership and shared decision making, and equity are systematically underrepresented in large-scale analyses Alpert et al. 2020 ; Alpert et al. 2019 ; Raisa et al. 2023 . For conditions such as mental health and substance use disorders, where much of care occurs outside traditional clinical settings and depends heavily on sustained patient engagement, outcomes hinge not only on access to therapy or medication but also on patients’ ability to manage stigma, social stressors, and fluctuating motivation in daily life. Evidence shows that treatment adherence and continuity of care in depression, anxiety, and opioid use disorder are strongly influenced by psychosocial factors, such as housing, food, financial insecurity, stress, and perceived social support. López-Paterna et al. 2025 ; Seaberg et al. 2025 Scalable methods that can reliably extract these signals from unstructured text are therefore essential for understanding treatment effectiveness and for designing responsive interventions to patients’ needs. However, manual abstraction of the patient voice from textual data and its transformation into a structured, accessible format is laborious and expensive Roter et al. 1997 ; Roter et al. 2006 ; Sulieman et al. 2017 ; Cronin et al. 2015 ; Cronin et al. 2017 ; Wallace et al. 2013 ; Roter et al. 2008 ; Roter 1997 . Existing machine learning methods Kharrazi et al. 2018 ; Hollister et al. 2017 ; Richard et al. 2015 ; Dalton-Locke et al. 2020 ; Senior et al. 2020 ; Chen et al. 2019 ; Wang et al. 2018a ; Wang et al. 2018b ; Wang et al. 2022a ; Wang et al. 2022b ; Wang et al. 2025a ; Wang et al. 2025b offer scalability but have primarily focused on clinical notes in electronic health records rather than unstructured patient-generated text, and typically target only a limited set of social domains. In this study, we introduce the PVminer framework, which formalizes the patient voice annotation as schema-constrained structured prediction from unstructured patient-generated text. The task requires extracting hierarchical labels with the main domains including: partnership and building rapport; shared decision-making; socioemotional support; and social determinants of health, and associated evidence Spans. PVminer reflects intrinsic properties of patient-generated data, including highly unstructured language, a wide range of social factors, overlapping categories, severe label imbalance, and a small number of semantically critical tokens that determine annotation correctness. To provide an initial solution using state-of-the-art scalable techniques, we first implement a dedicated prompt engineering approach and benchmark a range of instruction-tuned large language models spanning model sizes from 1.5B to 70B parameters under zero-shot and few-shot settings. Carefully engineered prompts enable these models to capture coarse semantic signals and yield measurable performance improvements. However, they frequently produce poorly structured, verbose, or truncated outputs. This behavior leads to substantial precision–recall gaps, indicating that prompting alone is insufficient for reliable patient voice extraction and motivating task-specific model adaptation. To address these limitations, we introduce PVminerLLM, a suite of supervised fine-tuned language models specialized for the PVminer task. By adapting instruction-tuned models to the PVminer schema, PVminerLLM enforces structured, schema-valid outputs and achieves strong performance across hierarchical labels and evidence Spans. Our results demonstrate that supervised fine-tuning offers a scalable and effective approach for high-fidelity extraction of socially and clinically meaningful signals from patient-generated text, enabling downstream analysis of the patient voice at scale. The contributions of this work are as follows. • We propose the PVminer Framework, a structured prediction formulation for extracting patient voice from patient-generated text that captures hierarchical labels, and evidence Spans. • We propose a systematically designed prompt engineering approach and provide a benchmark of instruction-tuned large language models ranging from 1.5B to 70B parameters under zero-shot and few-shot settings, highlighting the limitations of prompt-based approaches for schema-constrained patient voice extraction. • We developed PVminerLLM, a set of supervised fine-tuned large language models with different sizes that achieve strong performance on the PVminer task. These LLMs offer a practical and effective solution for extracting socially meaningful patient voice signals

Read the complete text