Using reasoning LLMs to extract SDOH events from clinical notes
Ertan Doganl, Kunyu Yu, Yifan Peng · 2026
Read Using reasoning LLMs to extract SDOH events from clinical notes by Ertan Doganl, Kunyu Yu, Yifan Peng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13502.txt.
Introduction
Dogan, Yu, and Peng use reasoning models to extract Social Determinants of Health events from clinical notes. Those conditions shaping how people live, work, and age are mostly trapped in unstructured text. Earlier BERT-based approaches perform well but need sophisticated implementation and heavy computing. Their method uses four modules: concise guideline-based prompts, few-shot learning with curated examples, self-consistency for robust outputs, and post-processing for quality control. It achieves a micro-F1 score of 0.866, competitive with leading models, with simpler implementation and strong performance.
Catalog record
| CREATOR | Ertan Doganl, Kunyu Yu, Yifan Peng |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43362 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.13502
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Using reasoning LLMs to extract SDOH events from clinical notes Ertan Doganl, Kunyu Yu, Yifan Peng Abstract Social Determinants of Health (SDOH) refer to environmental, behavioral, and social conditions that influence how individuals live, work, and age. SDOH have a significant impact on personal health outcomes, and their systematic identification and management can yield substantial improvements in patient care. However, SDOH information is predominantly captured in unstructured clinical notes within electronic health records, which limits its direct use as machine-readable entities. To address this issue, researchers have employed Natural Language Processing (NLP) techniques using pre-trained BERT-based models, demonstrating promising performance but requiring sophisticated implementation and extensive computational resources. In this study, we investigated prompt engineering strategies for extracting structured SDOH events utilizing LLMs with advanced reasoning capabilities. Our method consisted of four modules: 1) developing concise and descriptive prompts integrated with established guidelines, 2) applying few-shot learning with carefully curated examples, 3) using a self-consistency mechanism to ensure robust outputs, and 4) post-processing for quality control. Our approach achieved a micro-F1 score of 0.866, demonstrating competitive performance compared to the leading models. The results demonstrated that LLMs with reasoning capabilities are effective solutions for SDOH event extraction, offering both implementation simplicity and strong performance. keywords social determinants of health LLM machine learning prompting clinical notes SDOH SHAC n2c2 1 Introduction Social Determinants of Health (SDOH) are the conditions in which people live, work, and organize their lives, ultimately influencing their health outcomes [ 1 ] . These determinants encompass both protective factors that reduce health risks, such as access to healthcare and safe environments, and risk factors that can lead to adverse health outcomes, including poverty and substance abuse. SDOHs are traditionally grouped into five domains: economic stability, access to and quality of education, access to and quality of healthcare, neighborhood and built environment, and social and community context [ 1 ] . Growing evidence indicates that SDOH have a significant impact on population health and quality of life, with significant effects on both physical and mental health [ 1 , 2 ] . Due to their importance for individual well-being and public health, researchers seek to ensure that these factors are systematically incorporated into patient diagnosis and clinical decision-making processes. Since most SDOH data exists within unstructured clinical notes generated through natural patient-provider interactions rather than in structured, machine-readable formats [ 3 , 4 ] , specialized processing methods are required to extract and standardize this valuable information into usable formats. Consequently, researchers have developed strategies using Natural Language Processing (NLP) methods, including rule-based approaches and pre-trained language models (LMs) [ 5 , 6 , 7 , 8 ] . A recent effort is the 2022 n2c2/UW shared task on extracting SDOH. This competition scored participating research teams on the extraction of five categories of SDOH events: tobacco use, alcohol use, drug use, living status, and employment status [ 3 ] . Teams were tasked with extracting events as combinations of labeled text spans and set values that provided comprehensive and accurate descriptions of patient SDOH events. An example of annotations is shown in Figure 1 . (a) (b) Figure 1 : A sample of SDOH annotations in the 2022 n2c2/UW shared task. The top performing teams in the 2022 n2c2/UW shared task leveraged LMs pre-trained for Named Entity Recognition (NER), with most approaches involving fine-tuning BERT-based models for SDOH extraction [ 9 , 7 ] . While these methods achieved strong performance, their reliance on extensive pretraining and large annotated datasets created barriers to broader adoption. In addition, such models require further fine-tuning to recognize SDOH beyond the shared task scope, a resource-intensive process often hindered by limited data on rare SDOH events. Building on this challenge, subsequent studies used the annotated dataset with advanced large language models (LLMs), including OpenAI’s ChatGPT-4 and Meta’s LLaMa [ 5 , 6 ] . While these efforts achieved notable results, they did not match the performance of fine-tuned BERT-based models on event extraction. The LLMs used were general-domain models without domain-specific fine-tuning for medical text analysis, resulting in inferior performance compared to task-specific BERT models that benefit from substantial training data [ 6 ] . For less common SDOH, LLMs tended to perform closer to BERT-based models. The recent emergence of LLMs with enhanced reasoning capabilities, such as OpenAI’s o4-mini and Deepseek’s R1, offers new opportunities to advance SDOH extraction. In this study, we propose optimizing LLM prompting strategies to provide precise instructions and guidelines while leveraging the advanced reasoning capabilities of next-generation LLMs. Specifically, our objective is two-fold: (1) to investigate various prompting strategies for SDOH extraction using reasoning LLMs, and (2) to identify an intuitive querying approach that enables reasoning-capable LLMs to effectively identify and extract SDOH information from patient records in a structured format readily applicable to clinical decision-making workflows. We evaluated our approach on the 2022 n2c2/UW SDOH shared task using three reasoning LLMs, including OpenAI’s o4-mini, Gemini 2.5 Flash, and Llama-3.1-8B. Experiments demonstrated that our strategy using OpenAI’s o4-mini model yielded results competitive with those of the top-performing teams in the n2c2/UW shared task. Additionally, our strategy achieved higher precision than all leading teams. 2 Related Work Recent studies have investigated the application of LLMs to health-related topics [ 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ] . For example, Guevara et al. [ 11 ] and Consoli et al. [ 10 ] applied LLMs to clinical texts to identify SDOH information and evaluated ChatGPT-family models (GPT-3.5, GPT-4) using few-shot prompt strategies. These studies demonstrated results comparable to those of traditional language-model-based strategies, but primarily focused on classifying the presence of SDOH events rather than performing span-level extraction of SDOH events. In contrast, our work investigates the use of LLMs for span-level SDOH extraction, capturing detailed attributes such as event status, history, and the amount (for substance use). In parallel, LLMs with reasoning capabilities have been enhanced in models such as OpenAI and DeepSeek, incorporating techniques such as chain-of-thought prompting to improve performance on tasks requiring logic inference and analytical reasoning [ 22 , 23 , 24 , 25 ] . For example, Tordjman et al. [ 24 ] applied models such as OpenAI’s O1 and Deepseek’s R1 to text-based diagnosis and summarization and achieved promising results. However, these studies did not explore SDOH analysis or extraction. Building on this line of work, we apply reasoning-capable LLMs, including OpenAI’s o4-mini, Gemini 2.5 Flash, and Llama-3.1-8B, to SDOH extraction at the span level from clinical texts. We also apply self-consistency techniques [ 25 ] alongside chain-of-thought reasoning to improve extraction accuracy and robustness for SDOH events. 3 Materials and Methods 3.1 Dataset The Social History Annotated Corpus (SHAC) corpus [ 26 ] , used in the 2022 n2c2/UW SDOH challenge, is a publicly available, deidentified health records corpus containing 4,480 annotated social history sections (Table 1 ). SHAC comprises two data sources: MIMIC-III and the University of Washington (UW), each divided into training, testing, and development subsets. The annotations were in the Brat standoff format [ 27 ] , a human- and machine-readable annotation format that involves labeled text spans which