Zero-shot generation of synthetic neurosurgical data with large language models

Austin A. Barr, Eddie Guo, Emre Sezgin · 2025

Read Zero-shot generation of synthetic neurosurgical data with large language models by Austin A. Barr, Eddie Guo, Emre Sezgin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09566.txt.

Introduction

Austin Barr, Eddie Guo, and Emre Sezgin test whether a large language model can make fake brain surgery data that is good enough to study. Real patient data are hard to get, thanks to privacy rules, small samples, and the work needed to scrub names and dates.

They ask the model to make data cold, with no tuning and no sight of real records, and they compare it with a known fake data method. The model data matched real data well on means, shares, and links between fields. A classifier trained on it did as well as one trained on the rival data, and no real records were copied.

Catalog record

CREATORAustin A. Barr, Eddie Guo, Emre Sezgin
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-09566

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Zero-shot generation of synthetic neurosurgical data with large language models Austin A. Barr, Eddie Guo, Emre Sezgin Abstract Objective : Use of neurosurgical data for clinical research and machine learning (ML) model development is often limited by data availability, sample size, and regulatory constraints. Synthetic data offers a potential solution to challenges associated with accessing and using real-world data (RWD). This study aims to evaluate the capability of zero-shot generation of synthetic neurosurgical data with a large language model (LLM), GPT-4o, by benchmarking data fidelity, utility, and privacy with the conditional tabular generative adversarial network (CTGAN). Methods : A prompt instructed GPT-4o to generate synthetic data based on a plain-language description of the univariate and bivariate statistical properties of a real-world open-access neurosurgical dataset, which included 139 older adults who underwent neurosurgical intervention. The LLM was prompted to generate 10 synthetic datasets matching the original sample size, followed by an additional dataset representing a ten-fold amplification ( n = 1390). Two datasets were also generated with CTGAN, with sample sizes of 139 and 1390, for benchmarking. Synthetic datasets were compared to the RWD to assess fidelity (means, proportions, distributions, and bivariate correlations), utility (ML classifier performance on RWD), and privacy (duplication of records from RWD). Results : The GPT-4o-generated datasets matched or exceeded CTGAN performance, despite no fine-tuning or access to RWD for pre-training. Datasets demonstrated high univariate and bivariate fidelity to RWD without directly exposing any real patient records. Training an ML classifier on the GPT-4o-generated dataset ( n = 1390) and testing on RWD for a binary prediction task showed an F1 score (0.706) with comparable performance to training on the CTGAN data (0.705) for predicting postoperative functional status deterioration. Conclusions : GPT-4o demonstrated a promising ability to generate high-fidelity synthetic neurosurgical data. These findings also indicate that data synthesized with GPT-4o can effectively augment clinical data with small sample sizes, and train ML models for prediction of neurosurgical outcomes. Further investigation is necessary to improve the preservation of distributional characteristics and boost classifier performance. Keywords Neurosurgery \cdot Synthetic Data \cdot Large Language Models \cdot Generative Adversarial Networks \cdot Machine Learning 1 Introduction Neurosurgical research relies on data collected from research initiatives or routinely during surgical and perioperative care to identify disease patterns, evaluate interventions, and assess patient outcomes. High-quality neurosurgical data is also increasingly applied toward the training and validation of machine learning (ML) models for outcome prediction, image analysis, and operative planning [1-3]. However, there are significant barriers to acquiring, sharing, and using real-world data (RWD). Use of clinical data is constrained by data availability, incomplete data, small sample sizes, privacy regulations, and resource-intensive preprocessing and de-identification procedures [4-6]. The relative rarity of many intracranial diseases and limited patient volumes at individual institutions [2,7] pose barriers to assembling comprehensive and high-quality datasets for meaningful analyses. Routinely captured data is often missing information [5], either during care or when patients are lost to follow-up, which may limit the evaluation of long-term outcomes in procedural care. Requirements for accessing (e.g., institutional review boards, data privacy regulations), sharing (e.g., data-sharing agreements), and using (e.g., pre-processing, de-identification) clinical data are necessary to maintain patient privacy, yet costly, resource-intensive, and often inaccessible for researchers [8,9]. Processes to obtain data-sharing agreements restrict cross-institutional research collaboration [10], which is often necessary to assemble large neurosurgical datasets. Beyond the time and resources necessary for data preprocessing, de-identification procedures often remove or obscure information which may be relevant for analyses (e.g., demographics, geographics, temporal factors). To address these challenges, several initiatives have been undertaken to facilitate data sharing and collaboration among neurosurgical researchers. The American Spine Registry, Stereotactic Radiosurgery Registry, and the Quality Outcomes Database (QOD) including the NeuroVascular Quality Initiative-Quality Outcomes Database (NVQI-QOD) and the QOD Tumor Registry [11-13], host clinical data from participating institutions across the United States. While these registries have made significant contributions [14,15], they still face limitations in international collaboration and scope. Clinical registries focus on specific subdisciplines, which may not include all areas of neurosurgery and perioperative care. Collectively, these factors emphasize the importance of exploring alternative approaches to provide researchers with high-quality neurosurgical data while maintaining patient privacy. Synthetic data has emerged as a promising solution to challenges associated with data scarcity and privacy. This approach involves the creation of artificial datasets which retain statistical properties and relationships within RWD, but are designed to preserve patient privacy and enable greater opportunities for data sharing and use [16,17]. Current methods of creating synthetic data predominantly involve generative adversarial networks (GANs) and variational autoencoders (VAEs) [18,19]. While GANs and VAEs have demonstrated significant fidelity and utility of synthetic imaging and tabular data [20-22], several limitations persist. These approaches require technical expertise, access to RWD for training, and are associated with several privacy concerns—particularly when trained on smaller datasets [23]. Recent results have emerged in generating synthetic tabular data using another form of generative artificial intelligence: large language models (LLMs). Using zero-shot plain-language prompting, the LLM GPT-4o was capable of generating synthetic data which retained univariate statistical properties and simple between-parameter relationships found in real-world perioperative data [24]. While promising, questions remained regarding the utility of data for training ML models, capacity for data enhancement (i.e., amplification, augmentation), and applicability toward neurosurgical data. The present study aims to expand upon existing research which has evaluated a zero-shot approach to synthetic tabular data generation using GPT-4o. Specifically, we evaluate the fidelity, utility, and privacy of synthetic neurosurgical data generated with GPT-4o, and benchmark performance against a GAN designed for tabular data synthesis: conditional tabular generative adversarial network (CTGAN) [25]. Comparisons in means, proportions, distributions, bivariate correlations, ML model performance, and uniqueness of records are made between synthetic and RWD. These metrics are also evaluated with amplified sample sizes and with the addition of newly synthesized, interrelated features. By focusing on a small neurosurgical dataset that encompasses a variety of pathologies and perioperative parameters [26], this work seeks to demonstrate the applicability of LLMs toward addressing challenges in data access and availability for neurosurgical research. 2 Methods 2.1 Real-World Dataset and Parameter Selection The real-world neurosurgical dataset used as a comparator in this study was prospectively collected from a single high-volume tertiary care center in Milan, Italy [26,27]. It included records of 143 patients aged 65 years or older, undergoing various neurosurgical interventions between 1st January, 2018 and 31st December, 2019. This dataset was chosen for its breadth of clinically relevant parameters. The selection of parameters for the present analysis was guided by their clinical significance, completeness, and to reflect a diversity of data formats (i.e., numerical, text), variable types (i.e., continuous, categorical, ordinal, binary), distributions (i.e., normal, skewed), and measures relevant to perioperative assessment. A total of 12 parameters were selected, and are outlined in Table 1 . Table 1: Summary of selected parameters from the real-world neurosurgical dataset. Category ( n ) Parameters (units) Demographic data (2) Age (years), biological sex (M/F) Preoperative measures (5) Body mass index (kg/m 2 ), American Society of Anesthesiologists (ASA) physical status classification (1-6), heart disease (yes/no), diabetes (yes/no), previous brain radiotherapy (yes/no) Surgical parameters (2) Histology, Milan complexity scale (MCS) (0-8) Postoperative outcomes (3) Postoperative length of stay (days), Karnofsky performance status (KPS) deterioration

Read the complete text