Structured Insight from Unstructured Data: Large Language Models for SDOH-Driven Diabetes Risk Prediction
Sasha Ronaghi, Prerit Choudhary, David H. Rehkopf, Bryant Lin · 2026
Read Structured Insight from Unstructured Data: Large Language Models for SDOH-Driven Diabetes Risk Prediction by Sasha Ronaghi, Prerit Choudhary, David H. Rehkopf, Bryant Lin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-13388.txt.
Introduction
Sasha Ronaghi, Prerit Choudhary, David H. Rehkopf, and Bryant Lin pull diabetes risk clues from free text in health files. Social facts of health, such as home, food, and work strain, sway type two diabetes care. Yet they are oft absent from the neat fields of a health file. Most such facts are caught by fixed forms, which miss the nuance of each clinic's crowd.
Free text in notes holds a rich store of such facts. The team uses large language models to draw them out in a firm form. That form then feeds a risk guess for each patient. For clinics, the route may catch risks that fixed forms leave blank.
Catalog record
| CREATOR | Sasha Ronaghi, Prerit Choudhary, David H. Rehkopf, Bryant Lin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-13388
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Structured Insight from Unstructured Data: Large Language Models for SDOH-Driven Diabetes Risk Prediction Sasha Ronaghi, Prerit Choudhary, David H. Rehkopf, Bryant Lin Abstract Social determinants of health (SDOH) play a critical role in Type 2 Diabetes (T2D) management but are often absent from electronic health records and risk prediction models. Most individual-level SDOH data is collected through structured screening tools, which lack the flexibility to capture the complexity of patient experiences and unique needs of a clinic’s population. This study explores the use of large language models (LLMs) to extract structured SDOH information from unstructured patient life stories and evaluate the predictive value of both the extracted features and the narratives themselves for assessing diabetes control. We collected unstructured interviews from 65 T2D patients aged 65 and older, focused on their lived experiences, social context, and diabetes management. These narratives were analyzed using LLMs with retrieval-augmented generation to produce concise, actionable qualitative summaries for clinical interpretation and structured quantitative SDOH ratings for risk prediction modeling. The structured SDOH ratings were used independently and in combination with traditional laboratory biomarkers as inputs to linear and tree-based machine learning models (Ridge, Lasso, Random Forest, and XGBoost) to demonstrate how unstructured narrative data can be applied in conventional risk prediction workflows. Finally, we evaluated several LLMs on their ability to predict a patient’s level of diabetes control (low, medium, high) directly from interview text with A1C values redacted. LLMs achieved 60% accuracy in predicting diabetes control levels from interview text. This work demonstrates how LLMs can translate unstructured SDOH-related data into structured insights, offering a scalable approach to augment clinical risk models and decision-making. I INTRODUCTION Social determinants of health (SDOH) such as financial stress, access to transportation, and social support—play a significant role in shaping diabetes experience and outcomes [1]. However, social determinants are often missing from electronic health record (EHR) data, which serves as the foundation for most risk prediction models and provides essential context for clinicians delivering care [2, 3]. Truong et al show that only 1.9% of admissions include ICD-10 Z-codes, which are the only diagnosis codes that identify non-medical factors that may affect a patient’s health [4]. Traditionally, questionnaire-style screenings have been the primary method for collecting individual-level SDOH data [5] as they allow for large-scale data collection and potential integration with EHR models. Commonly used tools include PRAPARE [6] and the Center for Medicare and Medicaid Services’ Accountable Health Communities screening tool [7]. The limited structure and pre-defined categories within these tools often fail to capture the richness of patients’ lived experiences and the unique needs and circumstances of different patient demographics. To overcome this challenge, clinics are increasingly utilizing home-grown, customized questionnaires tailored to a clinic’s patient demographics, which can present data interoperability challenges for larger-scale population health analysis [5]. This trend highlights the need for a more flexible mode of SDOH data collection. Additionally, although diabetes is influenced by behaviors, psychological characteristics, and socioeconomic factors [8], existing risk prediction models do not integrate these domains [9]. This underscores the need for larger-scale data collection methodologies that capture behaviors, psychological characteristics, and socioeconomic factors in order to develop more holistic diabetes risk prediction tools. Collecting social determinant data in an unstructured manner, such as through clinical encounters or free-text boxes, presents an opportunity for capturing richer, more nuanced details about patients’ lives, including aspects of their disease management and social context that are rarely addressed in structured data collection methods [10]. However, until recently, the analysis of unstructured data using traditional machine learning models was constrained by the need for extensive manual preprocessing and feature extraction [11]. Large language models (LLMs) now present a transformative opportunity to process and analyze unstructured language efficiently and at scale [12]. LLMs excel at understanding context, identifying patterns, and extracting key insights from free-text narratives, enabling the discovery of novel risk factors and relationships that were previously inaccessible. This study leverages the capabilities of LLMs to analyze unstructured life stories from 65 T2D patients aged 65 years and older, focusing on their day-to-day disease management, diabetes literacy, and SDOH-related challenges. Although we apply these tools to a relatively small sample size, this work is aimed to serve as a proof of concept to illustrate how LLMs can analyze large quantities of unstructured qualitative data quickly for use within the clinical setting and for quantitative risk prediction modeling. Importantly, the study was conducted in a cohort with high Asian American and Pacific Islanders (AAPI) representation (46.2% of participants), providing valuable insight into this population. AAPI have a higher type 2 diabetes (T2D) prevalence and are less likely to take anti-diabetic medications than non-Hispanic White individuals [13]. Despite these disparities, AAPI are underrepresented in both epidemiological and genomic T2D research and the inherent diversity of these populations is often overlooked [14]. To our knowledge, this is one of the first times, across any disease outcome, that new methodologies for prediction are being developed with a focus on AAPI populations first, rather than as an afterthought. II METHODS Our study population consisted of 65 participants. These participants were recruited directly from a Stanford Healthcare clinic. Participants were primarily aged 65–69 (28.6%), 70–74 (25.4%), and 75–79 (27.0%), with smaller proportions aged 80–84 (9.5%) and 85–89 (9.5%), and 95–99 (1.6%). 46.2% of participants identify as Asian American and Pacific Islander, 32.3% as White, 12.3% as Hispanic, 7.7% as Black, and 6.2% as Middle Eastern/North African. The gender distribution of the population includes 70.5% identifying as male, 29.5% as female. The data for the project was obtained under IRB approval from Stanford University. We conducted unstructured interviews asking general conversational questions about patients’ life stories and experiences with diabetes, enabling participants to discuss topics they find significant. This approach ensures visibility into their lived experiences and highlights self-identified challenges and priorities. Interview lengths varied, with a minimum word count of 236, a maximum of 7,366, and most under 1,500 words. Alongside these narratives, we collected the full electronic health record clinical data for each participant. The mean most recent A1C level among participants was 6.83, with a median of 6.6, a standard deviation of 1.05, and a range from 4.5 to 10.3. We employ Stanford SecureGPT [15], which enables secure utilization of ChatGPT models for high-risk, protected health information. Using this platform, we applied LLMs to our dataset through three methods: 1. Extracting structured, free-form SDOH data : Converting unstructured narrative content into structured, actionable insights. 2. Using extracted SDOH data for risk prediction : Generating quantitative variables from qualitative narratives to serve as inputs for machine learning-based risk prediction models. 3. Predicting level of diabetes control : Asking LLMs to predict a patient’s level of diabetes control from the unstructured narratives. II-A Extracting structured, free-form SDOH data Because the interviews were unstructured, they covered a wide range of topics. To identify and organize themes within the dataset, we prompted ChatGPT-4o to read through each interview transcript, identify main themes and sub-themes, and provide three supportive quotes for each theme (Prompt 1 in Appendix). To further consolidate the dataset, we used sentence transformers to generate embeddings for the codes and applied agglomerative clustering to group similar codes. These clustered codes were manually reviewed and consolidated into