Topic-Aware Summarization of Lived Health Care Experiences: Large Language Model Evaluation Study
Maneesh Bilalpur, Megan E Hamm, Young Ji Lee, Natasha G Norman, Kathleen M Mctigue, Yanshan Wang, Arriel Benis · 2026
Read Topic-Aware Summarization of Lived Health Care Experiences: Large Language Model Evaluation Study by Maneesh Bilalpur, Megan E Hamm, Young Ji Lee, Natasha G Norman, Kathleen M Mctigue, Yanshan Wang, Arriel Benis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/PMC13258062.txt.
Description
Maneesh Bilalpur and colleagues present a 2026 evaluation study using large language models, AI text systems, to summarize people's lived health care experiences. The source material was fifty automatically transcribed spoken stories from African American patients, caregivers, and health care providers. A topic detection method grouped the stories into themes. A layered summarization approach then summarized stories topic by topic, and the GPT-4 model rated the summaries, checked by two domain experts.
Speech transcription reached a Levenshtein score of six percent, a measure of text errors. Twenty-six topics were found, including health behaviors, dealings with medical teams, caregiving, and symptom management. GPT-4 rated the topic summaries as free of fabrication, accurate, complete, and useful. Its ratings agreed moderately to highly with the experts, with scores of 0.65 or higher.
Catalog record
| CREATOR | Maneesh Bilalpur, Megan E Hamm, Young Ji Lee, Natasha G Norman, Kathleen M Mctigue, Yanshan Wang, Arriel Benis |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41132 |
| LICENSE | https://creativecommons.org/licenses/by/4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |