OpenMed NER: Open - Source, Domain - Adapted State - of - the - Art Transformers for Biomedical NER Across 12 Public Datasets
Maziyar Panahi · 2025
Read OpenMed NER: Open - Source, Domain - Adapted State - of - the - Art Transformers for Biomedical NER Across 12 Public Datasets by Maziyar Panahi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01630.txt.
Introduction
Maziyar Panahi releases open models that tag medical entities in text in a solo 2025 paper. Most health data sits in free notes, and pulling names of drugs, diseases, and genes out of it is a base chore. Top scores usually ride on closed or heavy models. His suite adapts open transformers cheaply and shares them for all.
The recipe warms the models on a third of a million passages from papers and scrubbed notes, then fine tunes a sliver of weights per task. Twelve known benches span chemicals, diseases, genes, and species. The suite set fresh best scores on ten of the twelve, with the largest leaps on gene and cell line sets. The author says smart adaptation, not scale, won the day.
Catalog record
| CREATOR | Maziyar Panahi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-01630
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OpenMed NER: Open - Source, Domain - Adapted State - of - the - Art Transformers for Biomedical NER Across 12 Public Datasets Maziyar Panahi Abstract Named-entity recognition (NER) is fundamental to extracting structured information from the >80% of healthcare data that resides in unstructured clinical notes and biomedical literature. Despite recent advances with large language models, achieving state-of-the-art performance across diverse entity types while maintaining computational efficiency remains a significant challenge. We introduce OpenMed NER , a suite of open-source, domain-adapted transformer models that combine lightweight domain-adaptive pre-training (DAPT) with parameter-efficient Low-Rank Adaptation (LoRA). Our approach performs cost-effective DAPT on a 350k-passage corpus compiled from ethically sourced, publicly available research repositories and de-identified clinical notes (PubMed, arXiv, and MIMIC-III) using DeBERTa-v3, PubMedBERT, and BioELECTRA backbones. This is followed by task-specific fine-tuning with LoRA, which updates less than 1.5% of model parameters. We evaluate our models on 12 established biomedical NER benchmarks spanning chemicals, diseases, genes, and species. OpenMed NER achieves new state-of-the-art micro-F1 scores on 10 of these 12 datasets , with substantial gains across diverse entity types. Our models advance the state-of-the-art on foundational disease and chemical benchmarks (e.g., BC5CDR-Disease, +2.70 pp), while delivering even larger improvements of over 5.3 and 9.7 percentage points on more specialized gene and clinical cell line corpora (BC2GM and CLL, respectively). This work demonstrates that strategically adapted open-source models can surpass closed-source solutions. This performance is achieved with remarkable efficiency: training completes in under 12 hours on a single GPU with a low carbon footprint ( < 1.2 <\!1.2 kg CO 2 e), producing permissively licensed, open-source checkpoints designed to help practitioners facilitate compliance with emerging data protection and AI regulations, such as the EU AI Act . 1 Introduction Recent advances in domain-specific language models, such as BioBERT ( Lee et al. 2019 ) , ClinicalBERT ( Alsentzer et al. 2019 ) , and PubMedBERT ( Gu et al. 2021 ) , have driven substantial progress in biomedical named-entity recognition (BioNER) ( Raza et al. 2022 ; Keraghel et al. 2024 ) . Yet, deploying these powerful models in real-world research and clinical settings requires overcoming persistent challenges like distribution shift across diverse corpora, tight compute budgets, and the need for locally-deployable models to comply with stringent data protection regulations (e.g., GDPR, HIPAA) . This paper demonstrates that a strategic combination of (i) domain-adaptive pre-training (DAPT) ( Gururangan et al. 2020 ) , (ii) parameter-efficient fine-tuning via Low-Rank Adaptation (LoRA) ( Hu et al. 2022 ) , and (iii) strong transformer backbones like DeBERTa-v3 ( He et al. 2021 ) can yield state-of-the-art results while remaining computationally efficient and fully accessible. The vast majority of information in biomedical research and healthcare is locked in unstructured text. Electronic Health Records (EHRs) alone are estimated to be \sim\!80\% unstructured: a category comprising clinical notes, research publications, drug labels, and patient narratives ( Li et al. 2021 ; Yang et al. 2022 ) . Biomedical text also exhibits a high density of unique entities compared to general-domain corpora, making BioNER a particularly knowledge-intensive task ( Lee et al. 2019 ; Yadav and Bethard 2018 ) . Named-entity recognition therefore serves as a foundational technology for downstream applications in precision medicine, drug discovery, and clinical decision support ( Keraghel et al. 2024 ; Raza et al. 2022 ) . This work, OpenMed NER, makes three primary contributions: 1. We develop and release a suite of BioNER models that establish new state-of-the-art performance on 10 of 12 public benchmarks, outperforming proprietary systems with a lightweight and fully open-source framework. 2. We provide a crucial ablation study quantifying the synergy between DAPT and LoRA, which together yield a 2\text{\textendash}4\% absolute F 1 gain over using either technique in isolation. 3. We release all final model checkpoints under the permissive Apache 2.0 license. This enables local, on-premise deployment, which is critical for applications handling sensitive health data and for facilitating compliance with data protection regulations like the EU AI Act. The biomedical domain poses linguistic challenges that extend far beyond general-domain NER. Biomedical prose features rapid neologism, systematic yet mutable nomenclatures, dense abbreviations, and morphological variants that proliferate with each new discovery ( Lee et al. 2019 ; Lin and Wilbur 2004 ) . Clinical entities further complicate matters through unclear boundaries, frequent nesting, and cross-category aliases. For instance, the phrase “upper respiratory tract infection (URTI)” can denote either a Disease or a Symptom depending on context ( Alex et al. 2007 ) . While early neural architectures such as BiLSTM-CNNs ( Chiu and Nichols 2016 ) reduced feature engineering, they still struggled with out-of-vocabulary terms and complex semantics. The transformer era, initiated by BERT ( Devlin et al. 2018 ) , spurred a new generation of domain-specific models. Variants like BioBERT ( Lee et al. 2019 ) , ClinicalBERT ( Alsentzer et al. 2019 ) , PubMedBERT ( Gu et al. 2021 ) , and BioELECTRA ( Kanakarajan et al. 2021 ) proved that domain-adaptive pre-training (DAPT) ( Gururangan et al. 2020 ) is critical for bridging the gap between general language understanding and specialised biomedical semantics. More recent advances, including powerful backbones like DeBERTa-v3 ( He et al. 2021 ) and parameter-efficient fine-tuning methods like LoRA ( Hu et al. 2022 ) , offer further opportunities to push the state of the art. Despite this rapid progress, critical gaps remain in BioNER research. First , few head-to-head studies compare fully open-source models with proprietary cloud NLP APIs, leaving the real-world performance and cost trade-offs opaque and not always transparent in peer-reviewed literature. Second , the synergy between modern backbones (e.g., DeBERTa-v3) and parameter-efficient training has not been systematically evaluated across the full spectrum of biomedical entity types. Third , most prior work benchmarks on a narrow subset of public datasets, often omitting clinically relevant corpora that test for generalization, such as the BC4CHEMD ( Krallinger et al. 2015 ) and Linnaeus ( Gerner et al. 2010 ) datasets. We address these gaps directly. By unifying efficient domain adaptation with exhaustive, cross-dataset benchmarking, OpenMed NER shows that open-source solutions can equal or even exceed proprietary biomedical NER systems. 2 Related Work Progress in biomedical named-entity recognition (BioNER) mirrors three macro-trends in natural language processing: (i) the shift from feature-based methods to end-to-end neural encoders ( Chiu and Nichols 2016 ; Yadav and Bethard 2018 ) , (ii) the replacement of generic word embeddings with domain-specific pre-trained language models (PLMs) ( Lee et al. 2019 ; Gu et al. 2021 ) , and (iii) the rise of parameter-efficient adaptation, notably LoRA, which updates less than 1.5% of model weights while preserving performance ( Hu et al. 2022 ) . OpenMed NER is built upon these three pillars. 2.1 Early Neural and Production-Scale Pipelines The BiLSTM-CNN-Char architecture of Chiu and Nichols 2016 represented a significant step forward for neural BioNER. Its re-implementation in the commercial Spark NLP library was significant for bringing these neural architectures to the scalable Apache Spark ecosystem, enabling large-scale, distributed inference across clusters ( Kocaman and Talby 2022 ) . Using more modern BERT-based models, Spark NLP has also reported strong, state-of-the-art results on several key benchmarks, demonstrating high performance on chemical and anatomical entity