ReProCon: Scalable and Resource-Efficient Few-Shot Biomedical Named Entity Recognition CCS: Computing methodologies Supervised learning by classification CCS: Applied computing Health care information systems
Jeongkyun Yoo, Nela Riddle, Andrew Hoblitzell · 2025
Read ReProCon: Scalable and Resource-Efficient Few-Shot Biomedical Named Entity Recognition CCS: Computing methodologies Supervised learning by classification CCS: Applied computing Health care information systems by Jeongkyun Yoo, Nela Riddle, Andrew Hoblitzell (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-16833.txt.
Introduction
Jeongkyun Yoo, Nela Riddle, and Andrew Hoblitzell present ReProCon in a 2025 arXiv paper. ReProCon finds named entities in biomedical text when labeled examples are scarce. Fine-grained types and uneven label counts make this task hard.
The method gives each category several prototype examples instead of one, which captures synonyms and varied contexts. A contrastive step keeps categories well apart, and a meta-learning update helps the model adapt fast. Its light encoder uses far less memory than big transformer models. Scores came close to BERT-based systems, and quality held up as categories grew from 19 to 50.
Catalog record
| CREATOR | Jeongkyun Yoo, Nela Riddle, Andrew Hoblitzell |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-16833
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ReProCon: Scalable and Resource-Efficient Few-Shot Biomedical Named Entity Recognition CCS: Computing methodologies Supervised learning by classification CCS: Applied computing Health care information systems Jeongkyun Yoo, Nela Riddle, Andrew Hoblitzell Abstract . Named Entity Recognition (NER) in biomedical domains faces challenges due to data scarcity and imbalanced label distributions, especially with fine-grained entity types. We propose ReProCon, a novel few-shot NER framework that combines multi-prototype modeling, cosine-contrastive learning, and Reptile meta-learning to tackle these issues. By representing each category with multiple prototypes, ReProCon captures semantic variability, such as synonyms and contextual differences, while a cosine-contrastive objective ensures strong interclass separation. Reptile meta-updates enable quick adaptation with little data. Using a lightweight fastText + BiLSTM encoder, with much lower memory use, ReProCon achieves a macro-F 1 score close to BERT-based baselines ( ∼ \sim 99% of BERT performance). The model remains stable with a label budget of 30% and only drops 7.8% in F 1 when expanding from 19 to 50 categories, outperforming baselines such as SpanProto and CONTaiNER, which see 10–32% degradation in Few-NERD. Ablation studies highlight the importance of multi-prototype modeling and contrastive learning in managing class imbalance. Despite difficulties with label ambiguity, ReProCon demonstrates state-of-the-art performance in resource-limited settings, making it suitable for biomedical applications. Disclaimer: This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Correspondence: Nela Riddle ([email protected]), Andrew Hoblitzell ([email protected]) 1. Introduction Named Entity Recognition (NER) identifies and categorizes entity mentions in unstructured text, serving as a foundation for downstream biomedical NLP tasks, including knowledge graph construction, literature curation, and clinical decision support. In biomedical domains, NER is particularly challenging due to the high cost of annotation and the long-tailed label distributions in public corpora, such as MedMentions ( Mohan and Li 2019 ) , BC2GM ( Krallinger et al. 2008 ) , and NCBI-Disease ( Doğan et al. 2014 ) , which often contain contextually variable entity categories. Few-shot learning mitigates the data bottleneck by training on only a handful of labeled examples. Although recent work shows promise on general domain benchmarks (e.g., Few-NERD ( Ding et al. 2021 ) ), NER has achieved limited success in the biomedical domain with a large number of entity types. In such cases, existing approaches often suffer from poor performance due to the overwhelming diversity of categories and a lack of ontological awareness. To address these challenges, we propose ReProCon , a prototype-based cosine-contrastive NER framework that addresses these gaps. The key components are the following: (1) Multi-prototype modeling : Each category has M distinct prototypes that capture various contexts of biomedical entities, designed to be angularly separated for better differentiation. (2) Cosine supervised-contrastive objective : The optimization aims for the entity spans to align closely with their corresponding category prototypes while being angularly distanced from others. (3) First-order meta-updates (Reptile) : Episode-based updates enable rapid parameter adaptation without the computational burden of second-order gradients ( Nichol et al. 2018 ) . Empirical evaluation in MedMentions demonstrates that ReProCon effectively tackles imbalanced samples and hierarchical complexity of the dataset, such as concept structures based on the Unified Medical Language System (UMLS) ( Bodenreider 2004 ) , while achieving performance comparable to or exceeding state-of-the-art methods, such as BioBERT ( Lee et al. 2020 ) and SpanProto ( Wang et al. 2022a ) , with improved computational efficiency. 2. Related Work 2.1. Domain-specific Language Models and Biomedical Resources Biomedical PLMs. Large-scale pre-training on domain-specific corpora significantly enhances biomedical NLP performance. Models like BioBERT, PubMedBERT ( Gu et al. 2021 ) , and SciBERT ( Beltagy et al. 2019 ) utilize PMC articles or PubMed abstracts instead of generic BERT training data, achieving substantial improvements in biomedical NER and question answering. These results underscore the data mismatch, differences in vocabulary, and context between the general and biomedical domains as the main bottleneck for domain transfer. UMLS. The Unified Medical Language System (UMLS) integrates more than 100 biomedical vocabularies into a unified ontology. Despite its extensive coverage, UMLS’s heterogeneous sources, biannual updates, and complex licensing pose challenges for few-shot evaluations, particularly in data preparation and model adaptation. To address this, we derive a dynamic semantic type hierarchy from MedMentions annotations ( Mohan and Li 2019 ) , enabling ReProCon to capture the latest ontological structures while avoiding licensing constraints. 2.2. Prototypical Network Several studies have implemented the prototypical network for the NER few-shot learning task. Classic prototypical networks assume one centroid per category ( Snell et al. 2017 ) . ReProCon extends hyperspherical prototype networks by incorporating multiple prototypes per category, enabling a finer-grained representation of biomedical entities. Mettes et al. applied hyperspherical prototype networks to NER tasks, including those within and between domains. They adapted classic prototypical networks by training prototypes as part of the end-to-end process, using a loss term designed to maximize the separation between prototypes ( Mettes et al. 2019 ) . In this study, we adopted the concept of calculating the repulsion loss between prototypes. 2.3. Meta-Learning Model-Agnostic Meta-Learning learns a single parameter initialization that can fine - tune new tasks with only a few gradient updates ( Finn et al. 2017 ) . It employs an inner loop for task-specific fine-tuning within an outer loop for generalized parameter optimization that computes second-order gradients. This approach aims to produce model parameters that are adaptable to new tasks with minimal data. However, backpropagating through gradient steps introduces significant computational and memory overhead, often causing instability in deep or noisy tasks. Reptile is a first-order meta-learning algorithm that learns a task - agnostic parameter initialization by repeatedly fine-tuning the sampled tasks and adjusting the shared weights to the adapted ones. Unlike MAML, Reptile eliminates second-order derivatives, so its meta-update is both simpler and substantially faster, while retaining strong few-shot performance. This outer loop translation of inner loop progress approximates higher-order information. 2.4. Supervised Contrastive Learning Supervised contrastive learning improves representation robustness in a few-shot setting and has also been applied in recent NER tasks to improve representation robustness under limited supervision ( Khosla et al. 2021 ) . For example, Das et al. implemented contrastive learning for the NER task by calculating losses over entity spans ( Das et al. 2022 ) . However, existing formulations assume one-to-one prototype alignment, which struggles with fine-grained biomedical categories. We extended this approach by introducing a multi-prototype contrastive loss that uses category-wise minimum pooling to align projected spans with the closest prototype in each category. This loss function enables both semantic flexibility and interclass separation, which is crucial for fine-grained classification of biomedical entities. 3. Data Processing 3.1. Sentence and Entity Preprocessing We first normalize whitespace and remove text enclosed in parentheses, brackets, braces, or angle brackets, as they often duplicate named entities, introducing redundant contextual variability. We applied the same preprocessing to both the sentences and their entity strings. 3.2. Semantic Type Normalization and Hierarchical Disambiguation The MedMentions corpus annotates entities with semantic type IDs of the Unified Medical Language System (UMLS), many of which are fine-grained (e.g., Organization , Substance ), exhibiting highly imbalanced sample distributions. Moreover, a single span can carry multiple types of labels, introducing ambiguity. To enhance learning efficiency by focusing on well-represented categories, we design a two-stage procedure through sparsity