CellTypeAgent: Trustworthy cell type annotation with Large Language Models
Jiawen Chen, Jianghao Zhang, Huaxiu Yao, Yun Li · 2025
Read CellTypeAgent: Trustworthy cell type annotation with Large Language Models by Jiawen Chen, Jianghao Zhang, Huaxiu Yao, Yun Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08844.txt.
Introduction
Naming the type of each cell in a sequencing experiment is essential and slow when done by hand. Jiawen Chen and co-authors build CellTypeAgent, a language-model agent that labels cell types and checks its answers against trusted databases. That verification step aims to curb the confident errors language models are known for.
Across nine real data sets covering 303 cell types from thirty-six tissues, the agent beat existing methods on accuracy. The database checks also cut hallucinated labels. The authors present their 2025 system as a more trustworthy route to a chore every single-cell project faces, speeding analysis without surrendering reliability to the model's guesses.
Catalog record
| CREATOR | Jiawen Chen, Jianghao Zhang, Huaxiu Yao, Yun Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-08844
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CellTypeAgent: Trustworthy cell type annotation with Large Language Models Jiawen Chen, Jianghao Zhang, Huaxiu Yao, Yun Li Abstract Cell type annotation is a critical yet laborious step in single-cell RNA sequencing analysis. We present a trustworthy large language model (LLM)-agent, CellTypeAgent, which integrates LLMs with verification from relevant databases. CellTypeAgent achieves higher accuracy than existing methods while mitigating hallucinations. We evaluated CellTypeAgent across nine real datasets involving 303 cell types from 36 tissues. This combined approach holds promise for more efficient and reliable cell type annotation. keywords cell type annotation, LLM agent, trustworthy † † equal-contributors: These authors contributed equally to this work. † † equal-contributors: These authors contributed equally to this work. 1 Main Cell type annotation, serving as an essential first step in single-cell RNA sequencing (scRNA-seq) analysis, demands considerable manual effort from researchers ( hu2023cellmarker ; hou2024assessing ) . The standard practice involves comparing marker genes expressed in each cell cluster with established marker genes documented in the literature, which is not only time-consuming but also labor-intensive. Advancements in artificial intelligence (AI), particularly in natural language processing (NLP), have introduced Large Language Models (LLMs) like Generative Pre-trained Transformers (GPT) ( achiam2023gpt ) . Trained on extensive datasets encompassing a wide range of textual information, LLMs possess the ability to understand and generate human-like language. Their proficiency in pattern recognition and contextual understanding make them powerful tools for various tasks, including those beyond natural language applications ( gong2023evaluating ) . Specifically, Hou et al. hou2024assessing have assessed the capability of GPT models in automated cell type annotation. Their study demonstrated that GPT not only effectively reduced the manual workload but also outperformed many existing semi-automated and fully automated methods. This highlights the potential of GPT models to revolutionize the cell type annotation process by enhancing both efficiency and accuracy. Despite the promising capabilities of LLMs, there are prevalent concerns regarding their reliability, particularly the phenomenon known as “hallucination” ( farquhar2024detecting ; huang2025survey ) . LLMs are known to occasionally generate responses that are nonsensical or factually incorrect, including fabricating references or providing misleading information. This issue is well-documented and poses significant challenges, especially in critical fields like medicine and biology, where accuracy is paramount. In this brief communication, we propose a trustworthy LLM agent, CellTypeAgent, specifically designed for cell type annotation using marker genes. We address the challenges associated with LLM hallucinations by integrating GPT models with existing databases. By combining the advanced language understanding capabilities of GPT with real-data verification, we aim to enhance the reliability of cell type annotations while maintaining efficiency. Our CellTypeAgent conducts an initial inference of potential cell types using an LLM. From the candidates suggested by the model, we then determine the final cell type by cross-referencing with the CellxGene database ( czi2023cz ) ( Figure 1 a). CellxGene is a comprehensive repository that includes single-cell gene expression data of 1634 datasets from 257 studies, allowing queries based on species (7 species), tissue type (50 tissue types), cell type (over 41 million cells covering 714 cell types), and marker gene name (106,944 genes). By selecting the cell type with the highest average gene expression from this database, we ensure that the final annotation is grounded in empirical data, thereby increasing the trustworthiness of the results. Figure 1: CellTypeAgent workflow. (a) CellTypeAgent first suggests several cell type candidates using an LLM. Then the final cell type is decided by cross-referencing with the CellxGene database. (b) A human adipose example of identified final cell type using CellTypeAgent, CellxGene, and GPTCellType. We systematically evaluated the performance of CellTypeAgent across nine datasets ( Figure 2 ) ( han2020construction ; han2018mapping ; the2022tabula ; liu2023single ; lee2020lineage ; kim2020single ; eraslan2022single ) . For comparison, we assessed its accuracy against competing methods, including GPTCelltype ( hou2024assessing ) , CellxGene alone ( czi2023cz ) , and another database, PanglaoDB ( franzen2019panglaodb ) . Cell type annotation accuracy was evaluated using manual annotations from the original studies as the benchmark ( Section 2 ). Our findings reveal that CellTypeAgent consistently outperforms other methods in all datasets ( Figure 2 a). We further examined CellTypeAgent’s performance using various base LLM models, which provide the initial cell type candidate recommendations ( Figure 2 b). Among these models, the o1-preview model achieved the highest accuracy, suggesting that stronger base models lead to improved annotations. We also evaluated CellTypeAgent against GPTcelltype across different base models and consistently observed superior performance from our method. Notably, when using a weaker base model (treating GPTcelltype as an LLM-only approach), CellTypeAgent demonstrated an even more pronounced improvement, underscoring the benefits of integrating a real database for supporting evidence when the model itself is less robust. In addition to model capability, data privacy is a common concern when using closed-source LLMs like ChatGPT ( neel2023privacy ) . To address this, we tested CellTypeAgent with Deepseek-R1 ( guo2025deepseek ) , an open-source LLM. Our results (Figure 2b) showed that CellTypeAgent improved the performance by 5.1%, bringing it very close to the top-performing base models. This gain even surpassed the performance achieved by GPT-4o under GPTcelltype, where GPT-4o outperformed Deepseek-R1 , illustrating that open-source models can deliver competitive results while alleviating privacy concerns with the help of database verification. Figure 2: CellTypeAgent result. (a) Average agreement score in 9 datasets. We note here that the numerical differences between our GPTCelltype scores and those reported by GPTCelltype are due to updates in the GPT-4 model. All experiments in our study were conducted using GPT models from the May to October 2024 versions. (b) Average agreement score across 9 datasets with various base LLM models. (c) Comparison of performance with different numbers of cell type candidates. (d) Comparison of performance with different numbers of marker genes. (e) Performance with mixed cell type marker genes. Each test is conducted by combining marker genes of two cell types A and B. Pure type A/B is the performance of cell type annotation using marker genes of A/B only. Mixed A&B is the performance of cell type annotation using marker genes of A and B. Avg pure A&B is the average performance of pure type A and pure type B in each test. (f) Comparison of performance with and without literature review and gene summary. (g) An example illustrating LLM’s heavy reliance on inputting text. We found that the initial inference by the LLM is an essential step in accurate cell type annotation. When comparing CellTypeAgent to using CellxGene alone for inference (i.e., without the initial LLM step), we observed that the CellxGene-only approach exhibited suboptimal performance across most datasets ( Figure 2 a). As an illustrative example, we consider annotating pericyte cells in a human adipose tissue ( Figure 1 b). Querying CellxGene yielded multiple cell types (mural cells, pericytes, and muscle cells) exhibiting similarly high average gene expression, leading to ambiguity and frequent misclassification (for example, classifying pericytes as mural cells in this example). In contrast, the LLM component of CellTypeAgent proposes initial candidate cell types based on its learned knowledge from literature. The top 3 candidates in this example are fibroblast, pericyte cell, and adipocyte. This guided inference significantly enhances the accuracy of cell type annotation by effectively narrowing down the most probable