Beyond Negation Detection: Comprehensive Assertion Detection Models for Clinical NLP
Veysel Kocaman, Yigit Gul, M. Aytug Kaya, Hasham Ul Haq, Mehmet Butgul, Cabir Celik, David Talby · 2025
Read Beyond Negation Detection: Comprehensive Assertion Detection Models for Clinical NLP by Veysel Kocaman, Yigit Gul, M. Aytug Kaya, Hasham Ul Haq, Mehmet Butgul, Cabir Celik, David Talby (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17425.txt.
Introduction
Veysel Kocaman and co-authors build models that judge how a medical fact is asserted in clinical notes, in this 2025 paper. A note may say a finding is present, absent, hypothetical, conditional, or about someone else. Tools that only catch negation misread much of this.
Several model families were trained and set against commercial cloud services, a classic rule system, and a leading chat model. Their fine-tuned large model scored highest overall, with the biggest lead on hypothetical statements. Deep models won on conditional and other-person cases, and a few-shot version stayed close behind. The models ship inside an open clinical text toolkit.
Catalog record
| CREATOR | Veysel Kocaman, Yigit Gul, M. Aytug Kaya, Hasham Ul Haq, Mehmet Butgul, Cabir Celik, David Talby |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-17425
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Negation Detection: Comprehensive Assertion Detection Models for Clinical NLP Veysel Kocaman, Yigit Gul, M. Aytug Kaya, Hasham Ul Haq, Mehmet Butgul, Cabir Celik, David Talby Abstract Assertion status detection is a critical yet often overlooked component of clinical NLP, essential for accurately attributing extracted medical facts. Past studies narrowly focused on negation detection, resulting in underperforming commercial solutions such as AWS Medical Comprehend, Azure AI Text Analytics, and GPT-4o due to their limited domain adaptation. To address this gap, we developed state-of-the-art assertion detection models, including fine-tuned LLMs, transformer-based classifiers, few-shot classifiers, and deep learning (DL) approaches and evaluated our models against cloud-based commercial API solutions and legacy rule-based NegEx approach as well as GPT-4o. Our fine-tuned LLM achieves the highest overall accuracy (0.962), outperforming GPT-4o (0.901) and commercial APIs by a notable margin, particularly excelling in Present (+4.2%), Absent (+8.4%), and Hypothetical (+23.4%) assertions. Our DL-based models surpass commercial solutions in Conditional (+5.3%) and Associated with Someone Else (+10.1%), while few-shot classifier offers a lightweight yet highly competitive alternative (0.929), making it ideal for resource-constrained environments. Integrated within Spark NLP, our models consistently outperform black-box commercial solutions while enabling scalable inference and seamless integration with medical NER, Relation Extraction, and Terminology Resolution. These results reinforce the importance of domain-adapted, transparent, and customizable clinical NLP solutions over general-purpose LLMs and proprietary APIs. keywords assertion detection ,negativity scope ,Spark NLP ,Natural Language Processing ,LLM ,healthcare NLP † † copyrightyear: 2025 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: In: R. Campos, A. Jorge, A. Jatowt, S. Bhatia, M. Litvak (eds.): Proceedings of the Text2Story’25 Workshop, Lucca (Italy), 10-March-2025 † † email: [email protected] † † email: [email protected] † † email: [email protected] † † email: [email protected] † † email: [email protected] † † email: [email protected] † † email: [email protected] † † address: John Snow Labs inc. 16192 Coastal Highway, Lewes, DE 19958, USA 1 Introduction The widespread adoption of Electronic Health Records (EHRs) has transformed healthcare, with 96% of non-federal acute care hospitals and 78% of office-based physicians in the United States using certified EHR systems by 2021. This digitization has created vast patient data repositories, opening new avenues for clinical applications and research. Office of the National Coordinator for Health Information Technology (2022) . To harness this valuable information and discover patterns in EHRs, various Natural Language Processing (NLP) tasks have been performed. Among these, the classification of assertions stands out as a critical but understudied task. Accurate assertion classification allows for the determination of whether a medical concept is present , absent , possible , hypothetical , conditional , or associated with someone other than the patient, crucial for extracting actionable insights from EHRs, driving clinical decision-making, and facilitating healthcare analytics Wen et al. (2024) . In other words, the status of an assertion explains how a named entity (e.g. clinical finding, procedure, lab result) pertains to the patient by assigning a label such as present (”patient is diabetic”), absent (”patient denies nausea”), conditional (”dyspnea while climbing stairs”), or associated with someone else (”family history of depression”). Table 1 illustrates different assertion classes with their label distribution and sizes. Although early studies often equated assertion detection with negation detection in time, sophisticated machine learning and deep learning methodologies evolved rudimentary rule-based approaches. Early techniques such as NegEx Chapman et al. (2001) , ConText Chapman et al. (2007) , NegFinder Mutalik et al. (2001) and NegExpander Aronow et al. (1999) relied on hand-crafted rules and regular expressions, achieving high precision but suffering from low recall due to rigid patterns Perez et al. (2023) . In order to learn more about rule-based approaches towards assertion detection, a reader is advised to check out a comprehensive study Uzuner et al. (2009) evaluating these approaches in detail. Deep learning methods, particularly transformer-based models and attention mechanisms, emerged as powerful alternatives, offering more nuanced understanding of clinical text. However, these approaches consistently faced challenges such as requiring large annotated datasets and struggling with minority classes, especially in detecting possible medical assertions. Recent developments have focused on addressing these limitations through innovative approaches like multi-task learning, pre-training techniques, and Large Language Models (LLMs). Bhatia et al. demonstrated the effectiveness of a multitask learning approach for jointly modeling named entity recognition and negation assertion in clinical texts. By utilizing shared parameters, their model achieved improved contextual representation and overcame challenges associated with neural networks in negation detection, outperforming rule-based systems in conjunction with the proposed conditional softmax decoder Bhatia (2019) . Chen et al. explored applying attention-based bi-LSTM architectures for negation and assertion detection in clinical notes, leveraging the ability to selectively focus on relevant information and automatically capture semantic details without relying on external knowledge inputs Chen (2019) . Aken et al. proposed a comprehensive study of clinical assertion detection models by manually annotating 5,000 assertions in the MIMIC-III dataset, evaluating medical language models’ performance and transferability across different medical domains, and releasing their annotated dataset to address label sparsity and diversity challenges in existing research van Aken et al. (2021) . Similarly, Wang et al. proposed a novel prompt-based learning approach for assertion classification that addresses existing limitations by leveraging few-shot learning and advanced reasoning techniques Wang et al. (2022) . Yuan et al. proposed a deep learning approach for automatic Electronic Medical Record (EMR) sectioning using MIMIC-III data, developing hand-crafted rules to create gold-standard labels and generating multiple note versions with varied section heading formats to train models that achieve robust adaptability and high accuracy in EMR segmentation Yuan and Yu (2024) . Ji et al. proposed a novel method leveraging Large Language Models (LLMs) with advanced reasoning techniques like Tree of Thought (ToT), Chain of Thought (CoT), and Self-Consistency (SC), combined with Low-Rank Adaptation (LoRA) fine-tuning, to transform assertion detection into a generative task that enables more nuanced, contextually aware, and data-efficient medical text understanding across multiple assertion categories Ji et al. (2024) . While existing clinical NLP approaches have made significant strides in assertion detection, predominantly focusing on negation, they have consistently fallen short of providing a comprehensive, multi-category framework capable of robustly addressing the full spectrum of medical concept assertions. In this paper, we present a comprehensive implementation of assertion detection within Healthcare NLP library Kocaman and Talby (2022) (based on Spark NLP Kocaman and Talby (2021) Haq et al. (2021) Haq et al. (2022) ecosystem), utilizing state-of-the-art models and annotators to achieve high accuracy and efficiency in clinical NLP tasks. Our approach transcends traditional negation detection methods by offering a comprehensive, fully integrable end-to-end solution that addresses the entire spectrum of assertion types, including present, absent, possible, hypothetical, conditional, and assertions associated with someone other than the patient . This holistic method leverages advanced deep learning architectures, few-shot learning techniques, and flexible rule-based systems to overcome common challenges in clinical texts, such as class imbalance and ambiguous concept expressions. Specifically, we explore the following architectures/ modules that we developed during this study to detect assertion status from clinical notes: Table 1: Examples from the i2b2 dataset illustrating different assertion classes with their label distribution. Text Label Description Size Overnight, the patient became hypoxic ,