OEMA: Ontology-Enhanced Multi-Agent Collaboration Framework for Zero-Shot Clinical Named Entity Recognition
Xinli Tao, B.S, Xin Dong, M.S, Xuezhong Zhou, Ph.D · 2025
Read OEMA: Ontology-Enhanced Multi-Agent Collaboration Framework for Zero-Shot Clinical Named Entity Recognition by Xinli Tao, B.S, Xin Dong, M.S, Xuezhong Zhou, Ph.D (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-15211.txt.
Introduction
Xinli Tao, Xin Dong, and Xuezhong Zhou present OEMA, a team of cooperating AI agents that finds medical terms in clinical notes without any labeled training examples. Pulling diseases, drugs, and procedures out of electronic health records is vital for research and care, but hand-labeling records is expensive. Language models can do the task zero-shot, yet their example choice and self-checking are often sloppy.
OEMA splits the work among agents: one drafts candidate examples, another filters them at the token level using the SNOMED CT medical ontology, and the design adds self-improvement on top. Matching example granularity to the task is a core concern throughout. The framework aims at dependable clinical extraction with no annotation budget.
Catalog record
| CREATOR | Xinli Tao, B.S, Xin Dong, M.S, Xuezhong Zhou, Ph.D |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-15211
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OEMA: Ontology-Enhanced Multi-Agent Collaboration Framework for Zero-Shot Clinical Named Entity Recognition Xinli Tao, B.S, Xin Dong, M.S, Xuezhong Zhou, Ph.D Abstract Objective: With the rapid expansion of unstructured clinical texts in electronic health records (EHRs), clinical named entity recognition (NER) has become a crucial technique for extracting medical information. However, traditional supervised models such as CRF and BioClinicalBERT suffer from high annotation costs. Although zero-shot NER based on large language models (LLMs) reduces the dependency on labeled data, challenges remain in aligning example selection with task granularity and effectively integrating prompt design with self-improvement frameworks. Materials and Methods: To address these limitations, we propose OEMA, a novel zero-shot clinical NER framework based on multi-agent collaboration. OEMA consists of three core components: (1) a self-annotator that autonomously generates candidate examples; (2) a discriminator that leverages SNOMED CT to filter token-level examples by clinical relevance; and (3) a predictor that incorporates entity-type descriptions to enhance inference accuracy. Results: Experimental results on two benchmark datasets, MTSamples and VAERS, demonstrate that OEMA achieves state-of-the-art performance under exact-match evaluation. Moreover, under related-match criteria, OEMA performs comparably to the supervised BioClinicalBERT model while significantly outperforming the traditional CRF method. Discussion: OEMA combines ontology-guided reasoning with multi-agent collaboration to address two key challenges in zero-shot clinical NER: granularity mismatch and prompt–self-improvement integration. Its ontology-based filtering reduces noise and enhances semantic alignment, showing strong performance across different LLMs. Conclusion: OEMA improves zero-shot clinical NER, achieving near-supervised performance under related-match criteria. Future work will focus on continual learning and open-domain adaptation to expand its applicability in clinical NLP. Tao et al. \ast ]Corresponding authors. [email protected] ; [email protected] . keywords Medical ontology; Clinical natural language processing; Zero-shot learning; Named entity recognition; Multi-agent systems † † journal: Journal Title Here † † doi: DOI HERE † † copyright-year: 2022 † † year: 2019 † † access: Advance Access Publication Date: Day Month Year † † appnotes: Paper † † firstpage: 1 † † address: Department of Artificial Intelligence, Beijing Key Lab of Traffic Data Analysis and Mining, School of Computer Science & Technology, Beijing Jiaotong University, Beijing, 100044, China † † corresponding: [ 1 Introduction Electronic health records (EHRs) contain vast amounts of unstructured clinical information, including clinical notes, and this type of information is of great value to clinical experts [ 11 ] , for which many studies have been devoted to the problem of clinical information extraction [ 12 ] . A key aspect in the extraction of clinical information is named entity recognition (NER), which is focused on identifying specific concepts such as medical problems, treatments, and examinations [ 14 ] . Early clinical natural language processing systems typically relied on predefined lexical resources and syntactic/semantic rules derived from manual analysis of large amounts of text [ 22 ] . In the last decade, machine learning-based approaches (e.g., CRF [ 10 ] ) have become increasingly popular [ 9 ] . In recent years, migration learning-based models (e.g., BioClinicalBERT [ 1 ] ) have been applied to the task of clinical named entity recognition, showing improved performance with fewer annotated samples [ 2 ] , but also requiring time-consuming development of annotated corpora by clinical experts [ 6 ] . Recently, with the advancement of large language models (LLMs), the task of NER has also ushered in new research directions and application possibilities [ 15 , 17 , 3 ] . Owing to their vast search space and large-scale pre-training data, LLMs show great potential for zero-shot NER, as evidenced by recent empirical studies [ 24 ] . Existing work has proceeded along two main avenues: first, prompt-engineering methods that construct more effective zero-shot prediction prompts to enhance LLMs’ generalization to out-of-distribution (OOD) knowledge [ 27 , 7 , 23 ] ; and second, self-improvement frameworks that use the LLM’s own generated self-consistency scores [ 21 ] to automatically label unlabeled data, thereby building a self-annotated corpus and improving performance at inference time through self-annotated few-shot in-context learning (ICL) [ 25 , 19 ] . As shown in Fig. 1 , despite these advances, current zero-shot NER methods still face two challenging problems: Figure 1: Challenge Analysis Diagram. In zero-shot learning, OEMA tackles two key challenges: (1) the mismatch between example selection and task granularity, and (2) the lack of effective integration between prompt design and the self-improvement framework. • Challenge 1: The mismatch between example selection and task granularity . In self-improvement frameworks, self-annotated examples must be added to prompts to guide the LLM. However, example selection is based on shallow, sentence-level strategies (e.g., random sampling or k-nearest neighbors [ 25 ] ), whereas NER is a token-level task—relying on sentence-level semantic similarity is therefore inappropriate [ 20 ] . • Challenge 2: The lack of practical integration between prompt design and self-improvement frameworks . Although existing studies have suggested that self-improvement frameworks are “prompt-agnostic” and, in theory, can incorporate any advanced prompt design methods to boost performance [ 25 ] , this has not been empirically validated, causing advanced prompts to fall short of realizing their potential within self-annotation frameworks [ 26 ] . To address the above challenges, we propose OEMA, a zero-shot clinical NER framework illustrated in Fig. 2 , which comprises three collaborative agents: a self-annotator for unlabeled data, a discriminator for selecting examples, and a predictor for final NER. First, the self-annotator uses LLMs to label an unlabeled corpus, thereby constructing a self-annotated corpus [ 25 ] . Next, the discriminator retrieves K candidate examples via cosine-similarity–based search, then applies a dedicated ICL method to extract top-level SNOMED CT concepts and their source text spans from both the test sentence and each example. It assigns each example a usefulness score with respect to the target input and selects the top k examples ( K>k ) as the final example set. Finally, the predictor performs inference on the target test sentence using a prompt that combines entity-type description with the self-annotated few-shot examples (built from that example set). Experimental results on two benchmark datasets demonstrate that OEMA achieves state-of-the-art performance, while ablation experiment, hyperparameter tuning, and case study jointly validate its effectiveness and interpretability. In summary, the main contributions of this study can be distilled into three aspects: 1. To address Challenge 1, we propose a medical‐ontology‐driven, token‐level example selection strategy. This approach refines self‐annotated example filtering from the traditional sentence level to a finer-grained token level, and employs a dynamic scoring mechanism to select examples that are highly relevant to the target test sentence, thereby enhancing the guidance provided to the LLM. This strategy effectively narrows the gap between example selection and task granularity, enabling more fine-grained and task-aligned example selection, particularly suitable for clinical NER tasks. 2. To address Challenge 2, we design the OEMA framework under a multi-agent collaborative mechanism, integrating advanced prompt design with the self-improvement framework. The former introduces type-level semantic priors through entity-type descriptions, effectively compensating for the generalization limitations of the latter’s self-annotated few-shot learning. This results in a dual prompting strategy—“type priors + structured examples”—which effectively alleviates the performance bottleneck between prompt design and the self-improvement framework. To the best of our knowledge, this is the first work to explicitly emphasize the synergy between type