Leveraging Open-Source Large Language Models for Data Augmentation in Hospital Staff Surveys: Mixed Methods Study

Carl Ehrett, Sudeep Hegde, Kwame Andre, Dixizi Liu, Timothy Wilson, Blake Lesselroth · 2024

Read Leveraging Open-Source Large Language Models for Data Augmentation in Hospital Staff Surveys: Mixed Methods Study by Carl Ehrett, Sudeep Hegde, Kwame Andre, Dixizi Liu, Timothy Wilson, Blake Lesselroth (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-11590755.txt.

Description

Small survey datasets are hard to classify well, and hospitals cannot always ship staff comments to commercial AI services. Carl Ehrett and co-authors tested in 2024 whether open-source language models could manufacture extra training text. Their material was survey narratives from frontline radiology staff describing how they adapted during the first COVID-19 wave.

Four generative models produced synthetic responses in the same style; three separate classifiers then sorted real and synthetic text into themes. The best recipe paired a LLaMA 7B generator, set to a middling randomness, with one hundred synthetic cases and a RoBERTa classifier, reaching an average accuracy score of 0.87. The authors conclude that open models can lift classifier performance on small health datasets while keeping data in-house. They also flag privacy and ethics questions for further work.

Catalog record

CREATORCarl Ehrett, Sudeep Hegde, Kwame Andre, Dixizi Liu, Timothy Wilson, Blake Lesselroth
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC11590755

Browse the full library · About the Tower of Babel