XuanHuGPT: parameter-efficient fine-tuning of large language model in the field of traditional Chinese medicine Authors: Xuming Tong, Xiaozheng Ding, Huiru Jia, Yanhong Yuan, Liyan Liu, Yapeng Wang, Zhang Xiong, Xu Yang, Sio Kei Im, Mini Han Wang Published: 2025 ABSTRACT Large Language Models (LLMs) have demonstrated exceptional generalization capabilities across various fields, including their application in Traditional Chinese Medicine (TCM). However, the performance of existing LLMs in TCM-specific tasks remains limited due to the lack of optimization for TCM knowledge during the pre-training phase, insufficient datasets, and the constraints of fine-tuning techniques. To address these challenges, this study constructs the XhTCM dataset by systematically integrating data from three authoritative sources—ShenNong_TCM_Dataset, TCMBank, and TCMIP v2.0. The dataset includes 100,000 structured entries, covering classical theories, prescription formulations, herbal pharmacology, and modern clinical practices. Based on this, we present XuanHuGPT, a domain-specific LLM tailored for TCM question answering and inference. By applying Parameter-Efficient Fine-Tuning (PEFT) techniques, we effectively balance model performance and training costs. Furthermore, we establish a comprehensive evaluation framework for TCM LLMs, combining quantitative metrics (BLEU, ROUGE, METEOR, BERTScore, and Embedding Distance) with expert qualitative assessments. Experimental results show that XuanHuGPT significantly outperforms both general-purpose LLMs and some existing TCM-specific models in accuracy, coverage, fluency, consistency, sensitivity, and safety. This study presents a reproducible paradigm for building intelligent TCM Q&A systems, contributing to the digital transformation, intelligent development, and global dissemination of TCM knowledge. Introduction Instruction-tuned large language models (LLMs), such as ChatGPT [1, 2], have gained prominence for their superior instruction comprehension and human-like language generation. When fine-tuned via supervised learning for specific applications, LLMs demonstrate robust generalization, yielding accurate and insightful outcomes [3, 4]. General-purpose LLMs, while versatile, often underperform in domain-specific tasks like question answering due to insufficient specialized training. To address this, domain-specific LLMs are developed by fine-tuning general models with targeted datasets, enhancing precision and expertise. For instance, BloombergGPT [5], adapted from BLOOM, excels in financial NLP tasks through finance-specific training. Similarly, Lawyer LLaMA [6], fine-tuned for legal applications, significantly enhances LLaMA’s performance in legal tasks. Notably, LLMs also present new opportunities for the preservation and advancement of Traditional Chinese Medicine (TCM) culture [7], opening pathways for innovative applications in this time-honored field. Traditional Chinese Medicine (TCM), a cornerstone of Chinese culture [8], spans over 5,000 years, encompassing extensive classical literature, medical cases, and prescriptions. It embodies millennia of Chinese health wisdom, dedicated to curing diseases and promoting well-being. UNESCO recognizes TCM’s cultural value, listing acupuncture and Tibetan medicinal bathing as Intangible Cultural Heritage [9, 10], and inscribing texts like Huangdi Neijing and Bencao Gangmu in the Memory of the World Register. TCM has also shown unique efficacy in COVID-19 prevention and treatment [11]. Despite its significance, TCM faces challenges in modern society. Knowledge dissemination relies heavily on traditional methods, such as paper-based texts and oral transmission, leading to fragmented information and barriers to research and learning. The lack of comprehensive digital platforms restricts accessibility and sharing of TCM knowledge. Additionally, limited public understanding and access to authoritative resources hinder engagement with TCM’s rich heritage. With advancements in LLMs, their potential in TCM is increasingly recognized. LLMs can efficiently process, translate, and summarize TCM literature, enabling rapid access to critical information and professional consultation [12, 13]. Developing TCM-specific LLMs is crucial for integrating LLMs into TCM, advancing its digitalization, and fostering new pathways for knowledge preservation and application. Full fine-tuning of LLMs involves updating all model weights during supervised learning, requiring extensive datasets and substantial computational resources. To mitigate these demands, Parameter-Efficient Fine-Tuning (PEFT) [14] methods have been developed. PEFT reduces tunable parameters and computational complexity, enhancing pre-trained model performance on new tasks while minimizing training burden. This approach improves effectiveness, shortens training time, and lowers costs, making it a focal point in model optimization research. For example, GatorTronGPT [15] leverages p-tuning [16] to optimize abstractive summarization, enabling LLMs to prioritize critical patient information and produce high-quality summaries. Despite the impressive generalization capabilities of large language models (LLMs) in various professional domains such as healthcare, their application in Traditional Chinese Medicine (TCM) remains limited by several critical challenges. These include the lack of Chinese-language TCM knowledge during pre-training, the unique linguistic structure of classical texts, the fragmented nature of modern clinical data, and the high cost of fine-tuning large-scale models. Existing TCM-specific models often fall short in terms of answer accuracy, reasoning depth, and interpretability. There is thus an urgent need for a domain-adapted LLM that efficiently integrates classical and modern TCM knowledge, supports robust reasoning, and remains computationally efficient. To this end, we propose XuanHuGPT—a specialized generative model for intelligent TCM question-answering and knowledge inference, aiming to advance the development of domain-specific LLM frameworks for TCM and promote its intelligent dissemination. XuanHuGPT is built upon the open-source bilingual language model ChatGLM2-6B, and incorporates two parameter-efficient fine-tuning methods: LoRA and P-Tuning v2. These techniques significantly reduce training resource requirements while maintaining strong model performance. For fine-tuning, we constructed the XhTCM Dataset, a high-quality corpus that systematically integrates classical TCM literature, modern clinical cases, herbal pharmacology, and prescription knowledge—ensuring both breadth and depth of TCM knowledge coverage. To comprehensively evaluate model performance, we designed a multidimensional evaluation framework combining automated metrics—BLEU, ROUGE, METEOR, BERTScore, and Embedding Distance—with expert human assessments across tasks such as symptom analysis, herb function interpretation, and syndrome-based treatment planning. Experimental results show that XuanHuGPT fine-tuned via LoRA consistently outperforms both general-purpose LLMs and existing TCM-specific models in terms of accuracy, fluency, semantic consistency, contextual sensitivity, and safety, demonstrating strong professional adaptability and clinical practicality in real-world TCM consultation scenarios. The main contributions of this study are summarized as follows:Construction of the XhTCM Dataset: We built a large-scale, high-quality TCM knowledge corpus by systematically integrating data from three authoritative sources—ShenNong-TCM-Dataset, TCMBank, and TCMIP v2.0. The dataset includes 100,000 structured entries covering classical theories, prescription formulations, herbal pharmacology, and modern clinical practices.Development of XuanHuGPT: We present a domain-specific LLM tailored for TCM question answering and inference. By incorporating parameter-efficient fine-tuning techniques, the model balances performance and training cost effectively.Establishment of a comprehensive evaluation framework for TCM LLMs: We introduce a dual-track evaluation strategy combining quantitative metrics and qualitative expert reviews. XuanHuGPT demonstrates strong performance across core dimensions such as accuracy, coverage, fluency, consistency, sensitivity, and safety.Promotion of TCM intelligence and global dissemination: This study offers a reproducible paradigm for building intelligent TCM QA systems, contributing to the digital transformation, intelligent development, and international dissemination of TCM knowledge, thereby enhancing the modern interpretation and global recognition of TCM culture. Construction of the XhTCM Dataset: We built a large-scale, high-quality TCM knowledge corpus by systematically integrating data from three authoritative sources—ShenNong-TCM-Dataset, TCMBank, and TCMIP v2.0. The dataset includes 100,000 structured entries covering classical theories, prescription formulations, herbal pharmacology, and modern clinical practices. Construction of the XhTCM Dataset: We built a large-scale, high-quality TCM knowledge corpus by systematically integrating data from three authoritative sources—ShenNong-TCM-Dataset, TCMBank, and TCMIP v2.0. The dataset includes 100,000 structured entries covering classical theories, prescription formulations, herbal pharmacology, and modern clinical practices. Development of XuanHuGPT: We present a domain-specific LLM tailored for TCM question answering and inference. By incorporating parameter-efficient fine-tuning techniques, the model balances performance and training cost effectively. Development of XuanHuGPT: We present a domain-specific LLM tailored for TCM question answering and inference. By incorporating parameter-efficient fine-tuning techniques, the model balances performance and training cost effectively. Establishment of a comprehensive evaluation framework for TCM LLMs: We introduce a dual-track evaluation strategy combining quantitative metrics and qualitative expert reviews. XuanHuGPT demonstrates strong performance across core dimensions such as accuracy, coverage, fluency, consistency, sensitivity, and safety. Establishment of a comprehensive evaluation framework for TCM LLMs: We introduce a dual-track evaluation strategy combining quantitative metrics and qualitative expert reviews. XuanHuGPT demonstrates strong performance across core dimensions such as accuracy, coverage, fluency, consistency, sensitivity, and safety. Promotion of TCM intelligence and global dissemination: This study offers a reproducible paradigm for building intelligent TCM QA systems, contributing to the digital transformation, intelligent development, and international dissemination of TCM knowledge, thereby enhancing the modern interpretation and global recognition of TCM culture. Promotion of TCM intelligence and global dissemination: This study offers a reproducible paradigm for building intelligent TCM QA systems, contributing to the digital transformation, intelligent development, and international dissemination of TCM knowledge, thereby enhancing the modern interpretation and global recognition of TCM culture. Related work LLMs in traditional Chinese medicine domain The application of large language models (LLMs) in the Traditional Chinese Medicine (TCM) domain has the potential to significantly enhance the efficiency of processing TCM-related text data, including tasks such as disease diagnosis, syndrome analysis, and prescription formulation. By integrating the knowledge and experience of TCM practitioners, LLMs can be fine-tuned to address the specific needs of the TCM field. For example, in disease consultation, LLMs enriched with TCM classical theories, clinical cases, and medical expertise can improve the accuracy and effectiveness of diagnostic outcomes. Additionally, in TCM text generation and retrieval, the language understanding and content generation capabilities of LLMs can produce coherent, contextually accurate texts while enabling efficient retrieval of TCM knowledge. Several LLMs have been developed for the Chinese medical domain, with varying degrees of emphasis on TCM-specific knowledge. DoctorGLM [17], a Chinese consultation model based on ChatGLM-6B, was built by translating a large-scale medical dialogue dataset into Chinese using ChatGPT. Fine-tuned using techniques such as LoRA and P-tuning v2, DoctorGLM excels in processing general Chinese medical data but lacks dedicated training on TCM-specific knowledge. In contrast, Zhongjing [18], based on LLaMA, represents the first LLM explicitly designed for TCM. It incorporates the full training pipeline, including continuous pre-training, supervised fine-tuning (SFT), and reinforcement learning with human feedback (RLHF), underscoring the critical role of domain-specific training in medical applications. HuatuoGPT [19], another prominent model, is built on Bloomz-7b1-mt and fine-tuned on a large-scale medical dialogue dataset through SFT and RLHF, with feedback provided by GPT. Although highly capable, it primarily targets general medical dialogue rather than TCM. Similarly, Qibo [20], based on Chinese-LLaMA, employs a two-stage training process that combines continuous pre-training with SFT. Furthermore, it introduces QiboBenchmark, a specialized evaluation tool for assessing the performance of LLMs in TCM across multiple dimensions. These advancements collectively demonstrate the growing interest and progress in applying LLMs to the TCM domain, though challenges remain in tailoring these models to the unique complexities of TCM knowledge and practices. The application of PEFT in LLM training Among various Parameter-Efficient Fine-Tuning (PEFT) methods, LoRA (Low-Rank Adaptation) [21] and P-tuning v2 [22] are two innovative approaches for fine-tuning large models. LoRA reduces storage requirements during training by decomposing model weight updates into low-rank matrices, significantly minimizing memory usage and computational costs. Additionally, weight pruning is employed to further reduce model size without compromising performance. For example, Donghee Choi et al. [23] proposed the DeepClair model, a novel framework for portfolio selection, and utilized LoRA fine-tuning to advance investment strategies. Similarly, Xi Chen et al. [24] introduced LLaST, which leverages dual LoRA optimization to enhance the efficiency of LLM-powered speech translation systems, highlighting LoRA’s versatility across diverse applications. P-tuning v2, on the other hand, introduces flexible prefix embeddings that can be shared across multiple layers, enabling more efficient training and reduced memory usage while maintaining performance comparable to full fine-tuning. For instance, Bin Zhang et al. [25] employed P-tuning v2 to fine-tune an LLM for Q&A tasks, maximizing its potential in interactive scenarios. Their work also led to the development of AugSBertChat, a system enhanced through user feedback to improve both performance and practical utility. Both LoRA and P-tuning v2 provide promising alternatives for developing efficient and scalable models, offering new research avenues for domain-specific applications. In summary, PEFT is a valuable strategy for tailoring LLMs to meet the unique demands of specialized fields, such as medicine, by significantly reducing computational overhead without compromising performance. Materials and methods This section discusses the construction process of the XuanHuGPT model, including the development of the dataset, model fine-tuning, and multi-dimensional model evaluation. The overall algorithm flowchart is shown in Fig. 1.Fig. 1The overall flowchart of constructing XuanHuGPT The overall flowchart of constructing XuanHuGPT Base model ChatGLM2-6B [26] is an enhanced version of the open-source bilingual dialogue model ChatGLM-6B, offering significant improvements in performance and efficiency. It employs GLM’s hybrid objective function, pre-trained on 1.4 trillion Chinese and English tokens, and fine-tuned using human preference alignment, enabling superior performance across multiple authoritative benchmarks. Additionally, the integration of FlashAttention extends the model’s context length from 2 to 32 K, with 8 K context length supported during dialogue training. Furthermore, Multi-Query Attention boosts inference speed by 42%, solidifying ChatGLM2-6B as a high-performance, efficient solution for diverse bilingual applications. Primary data sources To achieve effective fine-tuning for tasks in the field of Traditional Chinese Medicine (TCM), we constructed the proprietary XhTCM dataset, which integrates three high-quality TCM knowledge resources: ShenNong_TCM_Dataset [27], TCMBank [28]and TCMIP v2.0 [29]. ShenNong-TCM-dataset: This is a large-scale, instruction-based dataset tailored for TCM language processing tasks, containing over 110,000 entries covering fundamental TCM theories, diagnostic reasoning, herb compatibility, prescription inference, and clinical applications. Notably, a significant portion of the dataset is derived from semantic reconstruction and task design based on classic TCM texts such as Huangdi Neijing (Yellow Emperor’s Inner Canon), Shanghan Lun (Treatise on Cold Damage), and Shennong Bencao Jing (Shennong’s Materia Medica). These efforts aim to enhance the model’s ability to comprehend and reason with classical TCM texts. TCMBank: As the largest publicly accessible TCM database, TCMBank encompasses 9192 herbs, 61,966 unique compounds, 15,179 targets, and 32,529 disease entries. While it does not include original texts from ancient classics, its core information—such as herb properties, channel tropism, efficacy, and indications—is derived from authoritative sources like Shennong Bencao Jing, Bencao Gangmu (Compendium of Materia Medica), and Huangdi Neijing. TCMBank integrates multiple TCM databases, including TCMID, HERB, and SymMap, and is continuously updated through literature mining and authoritative biomedical resources. TCMIP v2.0: Focused on the integration of prescriptions and clinical knowledge, TCMIP v2.0 contains 48,442 prescription entries sourced from 649 ancient TCM classics, with each entry clearly annotated for traceability. It comprehensively covers the 100 ancient prescriptions listed in the “First Catalog of Classic Famous Prescriptions” issued by the National Administration of Traditional Chinese Medicine. Each prescription includes structured information such as composition, therapeutic effects, indications, target pathways, and mechanism predictions. To ensure the authority and applicability of the data, both TCMBank and TCMIP v2.0 incorporate the Pharmacopoeia of the People’s Republic of China (2020 edition) and the National Medical Products Administration (NMPA/CFDA) database to standardize herb nomenclature, property descriptions, quality standards, and usage guidelines. In summary, the XhTCM dataset combines the theoretical depth of ancient TCM classics with the rigor of modern pharmacological standards, providing a high-quality, systematic data foundation for fine-tuning models with specialized TCM knowledge. Construction and processing of the dataset Dataset construction and preprocessing The ShenNong_TCM_Dataset underwent rigorous preprocessing to identify and correct errors, inconsistencies, and defects, ensuring reliable data for model training. Manual screening and proofreading were employed to refine sentence meaning and enhance the dataset’s practical value, resulting in 85,000 entries covering multi-dimensional aspects of TCM consultation, prescription formulation, and disease diagnosis and treatment. To improve the model’s understanding of Chinese medicinal components and properties, data from TCMBank were structured and processed to create a 2000-item dataset. This was integrated into the XhTCM Dataset to enrich the model’s knowledge of Chinese herbs, compounds, gene targets, and related pathways or diseases. Additionally, data related to Chinese herbs, prescriptions, medicinal components, and disease-related molecular libraries were exported and manually verified from the TCMIP v2.0 database. A 3,000-entry dataset was extracted and incorporated into the XhTCM Dataset, further broadening the model’s coverage of medicinal properties and disease relationships. Data optimization via RAG Retrieval-Augmented Generation (RAG) [30, 31] addresses issues such as model knowledge limitations, hallucinations, and data security. By retrieving relevant information from proprietary domain databases and integrating it into prompt templates, RAG provides private data as part of the data prompts to the large language model (LLM), thereby reducing hallucinations in generative AI and enhancing generation capabilities. Leveraging the LangChain-Chatbot framework, we combined ChatGLM with the preprocessed dataset to construct a TCM Q&A generation model. High-quality TCM Q&A pairs generated from this model were selected to further enrich the dataset. Data augmentation via ChatGLM ChatGLM was used for self-dialogue to automatically generate high-quality multi-turn conversational datasets [31, 32], enhancing the dataset’s coverage and quality. This approach resolved issues such as mixed Chinese-English content and unclear semantic expressions, improving the overall readability and consistency of the data. Following extensive cleaning, integration, and refinement processes, we constructed the final XhTCM Dataset, comprising 100,000 high-quality entries. This dataset serves as a robust foundation for the training and evaluation of TCM-specific models, as summarized in Table 1. Table 1Information of XhTCM_DatasetDataScaleDescriptionShenNong_TCM_Dataset85,000TCM consultation, prescription, and treatment methods for syndromesTCM-Bank2000Relationships between Chinese herbal medicines, components, gene targets, and related pathways or diseasesTCMIP v2.03000Chinese herbal medicines, Chinese herbal prescriptions, Chinese herbal medicine components, and disease-related molecular librariesRAG + ChatGLM10,000Data in the field of TCM generated by LangChain and ChatGLM models Table 1Information of XhTCM_DatasetDataScaleDescriptionShenNong_TCM_Dataset85,000TCM consultation, prescription, and treatment methods for syndromesTCM-Bank2000Relationships between Chinese herbal medicines, components, gene targets, and related pathways or diseasesTCMIP v2.03000Chinese herbal medicines, Chinese herbal prescriptions, Chinese herbal medicine components, and disease-related molecular librariesRAG + ChatGLM10,000Data in the field of TCM generated by LangChain and ChatGLM models Information of XhTCM_Dataset Data update mechanism XuanHuGPT integrates LoRA-based lightweight fine-tuning with Retrieval-Augmented Generation (RAG) technology to achieve efficient adaptation and rapid updating of new knowledge. When incorporating new data, the model does not require full retraining, significantly reducing training costs and mitigating risks of performance fluctuations. Additionally, the XhTCM_Dataset implements dynamic monitoring of three foundational databases (TCMBank, TCMIP, and ShenNong_TCM_Dataset) to ensure continuous updates to the knowledge base. For knowledge derived from classical Traditional Chinese Medicine (TCM) texts, such as Huangdi Neijing and Bencao Gangmu, which are characterized by stable content and minimal changes, the update cycle is set to every 1–2 years. For instance, fundamental attributes of Chinese herbs in TCMBank, such as the Four Properties, Five Flavors, and Meridian Tropism, require only periodic verification for accuracy, without the need for frequent updates. In contrast, databases such as TCMIP v2.0, which include newly added pharmacological studies of herbs and novel formula combinations, are updated more frequently. Leveraging the retrieval-augmented mechanism of RAG, the system can capture and integrate this incremental information in real time, enabling the model to prioritize the latest knowledge during generation without necessitating comprehensive model updates. This strategy ensures the timeliness of knowledge in large-scale language models while preventing resource waste and instability in model performance. Fine-tuning methods Traditional fine-tuning methods often require large amounts of training data and significant computational resources, posing challenges in efficiency and scalability. PEFT methods address these limitations, providing a practical alternative to standard full-parameter fine-tuning. LoRA (Low-Rank Adaptation) optimizes LLMs by applying an implicit low-rank decomposition to their weight matrices. During fine-tuning, LoRA freezes the pre-trained model’s original weights and introduces trainable low-rank decomposition matrices at each Transformer layer. Only these newly added parameters are updated, drastically reducing the computational and memory costs compared to full-parameter fine-tuning. Despite the reduced parameter count, LoRA achieves performance comparable to traditional methods, offering high efficiency and adaptability. The underlying principle of this approach is represented in Eq. 1.1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(W + \Delta W)x = Wx + ABx$$\end{document}(W+ΔW)x=Wx+ABx Specifically, for a pre-trained weight matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${W}\in {R}^{d\times k}$$\end{document}W∈Rd×k, the weight update matrix is decomposed into two smaller matrices using low-rank decomposition, just as ΔW = \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${W}_{A}$$\end{document}WA* \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${W}_{B}$$\end{document}WB, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${W}_{A}$$\end{document}WA is a d × r matrix and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${W}_{B}$$\end{document}WB is an r × k matrix. During training, the original weight matrix W remains unchanged, and only the newly introduced matrices \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${W}_{A}$$\end{document}WA and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${W}_{B}$$\end{document}WB are updated. As shown in Fig. 2.Fig. 2Schematic diagram of the LoRA fine-tuning algorithm Schematic diagram of the LoRA fine-tuning algorithm LoRA preserves the pre-trained model’s weights while introducing trainable layers within each model block. This approach significantly reduces the number of parameters requiring fine-tuning and minimizes GPU memory usage. P-Tuning v2 leverages Deep Prompt Tuning by incorporating continuous trainable prompts into every layer of the pre-trained model. Unlike traditional methods that add prompts only at the input layer, P-Tuning v2 directly influences model predictions, enhancing task-specific adaptability while maintaining parameter efficiency. A schematic diagram of its algorithm is shown in Fig. 3. Compared to other PEFT methods, P-Tuning v2 focuses on freezing all model parameters and optimizing only prompt embeddings, while LoRA introduces a small number of trainable parameters to adjust the model’s internal features. This makes LoRA more versatile and effective for specialized tasks, offering a direct way to modify model behavior.Fig. 3Schematic diagram of the P-Tuning v2 fine-tuning algorithm Schematic diagram of the P-Tuning v2 fine-tuning algorithm In this study, both LoRA and P-Tuning v2 were applied to fine-tune the ChatGLM2-6B base model, and their performance was compared under identical instruction parameter settings. The objective was to develop XuanHuGPT, a high-quality, intelligent, domain-specific LLM tailored for TCM. Experiment and evaluation Experimental setup The development environment for this study included PyTorch 1.12, Python 3.9, CUDA 11.3, and Ubuntu 20.04. Fine-tuning of ChatGLM2-6B using LoRA and P-Tuning v2 was conducted on a single NVIDIA V100 GPU with 16 GB of memory. The learning rate was set to 3e−5, chosen to achieve a balance between training efficiency and overfitting prevention. The model was trained for 5 epochs, allowing sufficient time for the model to effectively learn from the dataset. The maximum context length was configured to 2048, enabling the model to process longer input sequences and improve its understanding of context. To optimize performance and memory efficiency, the model was trained using FP16 precision. This reduced memory usage and data transfer overhead compared to full-precision FP32, while maintaining high computational accuracy. This setup ensured a balance between computational efficiency and model performance throughout the training process. Baseline models To comprehensively evaluate our model, we select a series of LLMs with different parameter scales as baselines for comparison, including general and medical LLMs.MING-MOE [32] is an innovative MOE-based (Mixture of Experts) medical LLM. It employs a Mixture of Low-Rank Adaptation (MoLoRA) technique to enhance the multi-task learning capabilities of its base model. MING-MOE achieves state-of-the-art performance across a range of medical tasks.HuatuoGPT [19] combines distilled data from ChatGPT with real-world data from doctors to create an instruction dataset that blends AI-generated and human-authored styles. This approach aims to enhance its medical dialogue capabilities.The role of ChatGLM2-6B [26] is to serve as a benchmark, verifying that XuanHuGPT delivers more professional responses in TCM-related knowledge Q&A compared to general-purpose large models.ChatGLM2-6B (P-Tuning v2) In order to compare and verify the performance of the LoRA, we also applied the P-Tuning v2 algorithm to fine-tune the ChatGLM2-6B base model and compared the performance of the two different fine-tuning algorithms under the same instruction parameter settings and dataset.ShennongAlpha [33] is a LLM system built upon a RAG architecture, specialized in question answering and translation tasks related to natural medicinal materials, traditional Chinese medicines, and traditional Chinese medicine knowledge. By decoupling the language model from the knowledge base and integrating customized retrieval algorithms with prompt construction mechanisms, the system significantly enhances the accuracy and interpretability of domain-specific responses.DeepSeek [34] is widely used for general-purpose question answering, reasoning, and code generation tasks. It adopts a Mixture-of-Experts (MoE) architecture, which significantly reduces training costs while maintaining high inference efficiency. By incorporating Multi-Head Latent Attention (MLA) mechanisms and multi-token pretraining objectives, DeepSeek enhances model stability and expressive power. It has become one of the most prominent open-source large language models available today. MING-MOE [32] is an innovative MOE-based (Mixture of Experts) medical LLM. It employs a Mixture of Low-Rank Adaptation (MoLoRA) technique to enhance the multi-task learning capabilities of its base model. MING-MOE achieves state-of-the-art performance across a range of medical tasks. MING-MOE [32] is an innovative MOE-based (Mixture of Experts) medical LLM. It employs a Mixture of Low-Rank Adaptation (MoLoRA) technique to enhance the multi-task learning capabilities of its base model. MING-MOE achieves state-of-the-art performance across a range of medical tasks. HuatuoGPT [19] combines distilled data from ChatGPT with real-world data from doctors to create an instruction dataset that blends AI-generated and human-authored styles. This approach aims to enhance its medical dialogue capabilities. HuatuoGPT [19] combines distilled data from ChatGPT with real-world data from doctors to create an instruction dataset that blends AI-generated and human-authored styles. This approach aims to enhance its medical dialogue capabilities. The role of ChatGLM2-6B [26] is to serve as a benchmark, verifying that XuanHuGPT delivers more professional responses in TCM-related knowledge Q&A compared to general-purpose large models. The role of ChatGLM2-6B [26] is to serve as a benchmark, verifying that XuanHuGPT delivers more professional responses in TCM-related knowledge Q&A compared to general-purpose large models. ChatGLM2-6B (P-Tuning v2) In order to compare and verify the performance of the LoRA, we also applied the P-Tuning v2 algorithm to fine-tune the ChatGLM2-6B base model and compared the performance of the two different fine-tuning algorithms under the same instruction parameter settings and dataset. ChatGLM2-6B (P-Tuning v2) In order to compare and verify the performance of the LoRA, we also applied the P-Tuning v2 algorithm to fine-tune the ChatGLM2-6B base model and compared the performance of the two different fine-tuning algorithms under the same instruction parameter settings and dataset. ShennongAlpha [33] is a LLM system built upon a RAG architecture, specialized in question answering and translation tasks related to natural medicinal materials, traditional Chinese medicines, and traditional Chinese medicine knowledge. By decoupling the language model from the knowledge base and integrating customized retrieval algorithms with prompt construction mechanisms, the system significantly enhances the accuracy and interpretability of domain-specific responses. ShennongAlpha [33] is a LLM system built upon a RAG architecture, specialized in question answering and translation tasks related to natural medicinal materials, traditional Chinese medicines, and traditional Chinese medicine knowledge. By decoupling the language model from the knowledge base and integrating customized retrieval algorithms with prompt construction mechanisms, the system significantly enhances the accuracy and interpretability of domain-specific responses. DeepSeek [34] is widely used for general-purpose question answering, reasoning, and code generation tasks. It adopts a Mixture-of-Experts (MoE) architecture, which significantly reduces training costs while maintaining high inference efficiency. By incorporating Multi-Head Latent Attention (MLA) mechanisms and multi-token pretraining objectives, DeepSeek enhances model stability and expressive power. It has become one of the most prominent open-source large language models available today. DeepSeek [34] is widely used for general-purpose question answering, reasoning, and code generation tasks. It adopts a Mixture-of-Experts (MoE) architecture, which significantly reduces training costs while maintaining high inference efficiency. By incorporating Multi-Head Latent Attention (MLA) mechanisms and multi-token pretraining objectives, DeepSeek enhances model stability and expressive power. It has become one of the most prominent open-source large language models available today. Evaluation metrics To comprehensively evaluate the performance of large language models in this study, we employed a combination of human qualitative assessment and multiple representative quantitative metrics, each targeting a specific aspect of model capability. BLEU [35] and ROUGE [36] were adopted to assess surface-level accuracy and content coverage of the generated text, respectively. METEOR [37] places greater emphasis on semantic appropriateness and linguistic fluency. BERTScore [38] evaluates the semantic consistency between the model output and the reference. Embedding Distance [39], on the other hand, measures the model’s sensitivity to subtle semantic variations in input queries. AI tool and expert [40] are used to measure the model’s safety. Together, these metrics form a multidimensional and systematic evaluation framework, enabling a more fine-grained analysis and comparison of model performance across dimensions such as accuracy, coverage, fluency, consistency, sensitivity, and safety.BLEU (Accuracy Evaluation Metric) BLEU (Accuracy Evaluation Metric) BLEU (Accuracy Evaluation Metric) BLEU (Bilingual Evaluation Understudy) is a metric used to evaluate the quality of machine translation results. It primarily focuses on measuring the degree of similarity between the machine-translated output and reference translations, emphasizing the accuracy and exact match of sentences. It can be expressed by Eqs. 2 and 3.2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$BLEU=BP\times \text{exp}\left({\sum }_{n=1}^{N}{W}_{n}\times log{P}_{n}\right)$$\end{document}BLEU=BP×exp∑n=1NWn×logPn3\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$BP=\left\{\begin{array}{c}1, lc>lr\\ exp(1-\frac{lr}{lc}), lc\le lr\end{array}\right.$$\end{document}BP=1,lc>lrexp(1-lrlc),lc≤lr \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\text{BP})$$\end{document}(BP) is the brevity penalty factor, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$({\text{l}}_{\text{c}})$$\end{document}(lc) denotes the length of the machine-translated text, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$({\text{l}}_{\text{r}})$$\end{document}(lr) denotes the length of the shortest reference translation. If the length of the translation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$({\text{l}}_{\text{c}})$$\end{document}(lc) is less than the length of the shortest reference translation\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$({\text{l}}_{\text{r}})$$\end{document}(lr), then \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\text{BP}< 1)$$\end{document}(BP<1). Here, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\text{c})$$\end{document}(c) is the length of the candidate translation, and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\text{r})$$\end{document}(r) is the closest reference length. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$({\text{W}}_{\text{n}})$$\end{document}(Wn) refers to the weight of n-grams, which is generally set to a uniform weight, meaning \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$({\text{W}}_{\text{n}}=\frac{1}{\text{N}})$$\end{document}(Wn=1N) for any\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\text{ n })$$\end{document}(n). \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$({\text{P}}_{\text{n}})$$\end{document}(Pn) is the precision for n-grams, calculated as the ratio of matching n-grams in the candidate translation to the total number of n-grams in the candidate translation. BLEU requires calculating the precision of the translation for 1-g, 2-g, …, up to N-gram, where N is typically set to 4. 1-g precision indicates the degree to which the translation is faithful to the original text, while other n-grams indicate the fluency of the translation.(2)ROUGE (Coverage Evaluation Metric) (2)ROUGE (Coverage Evaluation Metric) ROUGE (Coverage Evaluation Metric) ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is a metric used to evaluate the quality of text summarization. ROUGE primarily focuses on whether the machine-generated summary captures the information present in the reference summary, emphasizing the completeness and coverage of the content and information in the reference summary. In short, while BLEU emphasizes the accuracy and precision of translations, leaning towards precision, ROUGE emphasizes the completeness and coverage of information in summaries, leaning towards recall. These two metrics are often used together in evaluating natural language processing models to provide a comprehensive assessment of their performance. ROUGE-1 (Eq. 4): Calculates the ratio of the intersection to the union of words between the generated summary and the reference summary. It measures the recall of individual words.4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text{ROUGE-1}=\frac{{\sum }_{i=1}^{\left|{\text{reference}}\right|}{\text{count}}_{\text{match}}\left({w}_{i}\right)}{{\sum }_{i=1}^{\left|{\text{reference}}\right|}{\text{count}}\left({w}_{i}\right)}$$\end{document}ROUGE-1=∑i=1referencecountmatchwi∑i=1referencecountwi ROUGE-2 (Eq. 5): Calculates the ratio of the intersection to the union of bigrams (consecutive word pairs) between the generated summary and the reference summary. It measures the recall of bigrams.5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text{ROUGE-2}=\frac{{\sum }_{i=1}^{\left|\text{reference bigrams}\right|}{\text{count}}_{\text{match}}\left({b}_{i}\right)}{{\sum }_{i=1}^{\left|\text{reference bigrams}\right|}{\text{count}}\left({b}_{i}\right)}$$\end{document}ROUGE-2=∑i=1reference bigramscountmatchbi∑i=1reference bigramscountbi ROUGE-L (Eq. 6): Calculates recall based on the length of the longest common subsequence (LCS) between the generated summary and the reference summary. It considers not only the order of words but also their continuity.6\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text{ROUGE-L}=\frac{LC{S}_{\text{generated, reference}}}{\text{max}\left(\left|{\text{generated}}\right|,\left|{\text{reference}}\right|\right)}$$\end{document}ROUGE-L=LCSgenerated, referencemaxgenerated,reference(3)METEOR (Fluency Evaluation Metric) (3)METEOR (Fluency Evaluation Metric) METEOR (Fluency Evaluation Metric) METEOR is a semantic and order-aware metric for evaluating the quality of generated text, widely used for the automatic evaluation of outputs from large language models. Unlike metrics such as BLEU that rely solely on n-gram exact matches, METEOR measures the semantic similarity between candidate and reference texts through a multi-level matching mechanism, including exact match, stem match, synonym match, and paraphrase match. In addition, it introduces a penalty term based on word order consistency to reflect the fluency and logical coherence of the generated output. The core computation formula is shown below.7\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{METEOR}} = \left( {\frac{{10 \cdot {\varvec{P}} \cdot {\varvec{R}}}}{{{\varvec{R}} + 9{\varvec{P}}}}} \right) \cdot \left( {1 - {{\varvec{\upgamma}}} \cdot \left( {\frac{{{\varvec{ch}}}}{{\varvec{m}}}} \right)^{{{\varvec{\uptheta}}}} } \right)$$\end{document}METEOR=10·P·RR+9P·1-γ·chmθ(4)BERTScore (Consistency Evaluation Metric) (4)BERTScore (Consistency Evaluation Metric) BERTScore (Consistency Evaluation Metric) Consistency measures the stability of a model's outputs when processing semantically equivalent but differently phrased queries. This study employs BERTScore for evaluation, which assesses semantic similarity between candidate and reference texts using contextual embeddings from a pre-trained language model. Specifically, BERTScore computes the maximum cosine similarity between each word in the candidate text and all words in the reference text, with the average score representing precision. Recall is calculated analogously, and the F1 score (as shown in the formula) provides an overall measure of semantic consistency.8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{BERTScore}}_{F1}=2\cdot \frac{{\text{Precision}}\cdot {\text{Recall}}}{{\text{Precision}}+{\text{Recall}}}$$\end{document}BERTScoreF1=2·Precision·RecallPrecision+Recall(5)Embedding Distance (Sensitivity Evaluation Metric) (5)Embedding Distance (Sensitivity Evaluation Metric) Embedding Distance (Sensitivity Evaluation Metric) Sensitivity refers to a model's ability to accurately detect and respond to subtle yet clinically significant differences in input queries, such as providing distinct treatment plans for "yellow phlegm cough" versus "white phlegm cough." This study employs Embedding Distance as a metric to evaluate model sensitivity. This approach measures the degree of difference between two model responses in semantic space to determine whether the model can "perceive" diagnostic nuances arising from minor input variations. By using a sentence vector encoding model, responses are converted into fixed-length semantic vectors, and the cosine distance between these vectors is calculated, as shown in the formula. A larger distance indicates greater model sensitivity to subtle differences.9\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\text{Distance}}\left({A}_{1},{A}_{2}\right)=1-\frac{\overrightarrow{{A}_{1}}\cdot \overrightarrow{{A}_{2}}}{\parallel \overrightarrow{{A}_{1}}\parallel \cdot \parallel \overrightarrow{{A}_{2}}\parallel }$$\end{document}DistanceA1,A2=1-A1→·A2→‖A1→‖·‖A2→‖(6)AI Tool and Expert Evaluation (Safety Evaluation Metric) (6)AI Tool and Expert Evaluation (Safety Evaluation Metric) AI Tool and Expert Evaluation (Safety Evaluation Metric) Yang et al. [40] proposed a comprehensive evaluation framework for large language models, encompassing three dimensions and nine capabilities. This study focuses solely on the safety dimension. Safety is defined by three core capabilities: (1) Providing scientifically accurate medical knowledge and transparently acknowledging limitations when encountering unknown information. (2) Ensuring patient safety by refusing to provide information or recommendations that could cause harm. (3) Adhering to medical ethics by declining to engage with content that violates ethical principles. The evaluation process employs a two-stage methodology. Initially, a third-party artificial intelligence tool performs a comparative assessment of the models’ question-answering outputs. Subsequently, domain experts conduct a final adjudication to determine the outcomes. Results and analysis Quantitative analysis In the quantitative analysis, we assessed the performance of the models using six evaluation metrics: BLEU, ROUGE, METEOR, BERTScore, Embedding Distance, and AI tool and expert. These metrics provide a multi-dimensional evaluation of the models, including accuracy, coverage, fluency, consistency, sensitivity, and safety. The models compared include general large models (DeepSeek, ChatGLM2-6B), as well as specialized Chinese medicine models (HuatuoGPT, ShennongAlpha, MING-MOE, ChatGLM2-6B (P-Tuning v2)). Performance evaluation on accuracy metrics Table 2 presents a performance comparison of different large language models (LLMs) based on the BLEU evaluation metric. The BLEU metric quantifies the accuracy and fluency of generated text by calculating the n-gram overlap between generated and reference texts. Experimental results demonstrate that XuanHuGPT (LoRA) significantly outperforms other models across all BLEU metrics. Table 2Evaluation results on BLEU metricsLLMsBLEU-1BLEU-2BLEU-3BLEU-4DeepSeek39.6314.586.843.03ChatGLM2-6B26.447.354.061.37HuatuoGPT37.1413.255.353.04ShennongAlpha45.9816.668.104.33MING-MOE33.8013.586.322.96ChatGLM2-6B(P-Tuning v2)40.1014.047.343.97XuanHuGPT(LoRA)51.5325.1211.855.69 Table 2Evaluation results on BLEU metricsLLMsBLEU-1BLEU-2BLEU-3BLEU-4DeepSeek39.6314.586.843.03ChatGLM2-6B26.447.354.061.37HuatuoGPT37.1413.255.353.04ShennongAlpha45.9816.668.104.33MING-MOE33.8013.586.322.96ChatGLM2-6B(P-Tuning v2)40.1014.047.343.97XuanHuGPT(LoRA)51.5325.1211.855.69 Evaluation results on BLEU metrics Specifically, XuanHuGPT achieves a superior BLEU-1 score of 39.63, indicating excellent word-level accuracy and high consistency with professional reference texts. More notably, its outstanding performance on higher-order BLEU metrics (BLEU-2 = 25.12, BLEU-3 = 11.85, BLEU-4 = 5.42) confirms its syntactic reasonableness and semantic coherence in generating complex traditional Chinese medicine (TCM) terminology and multi-word sequences. This advantage can be attributed to two key factors: (1) the XhTCM dataset, which integrates diverse, high-quality TCM-specific corpora, and (2) the LoRA fine-tuning strategy, which employs low-rank adaptation for computationally efficient domain specialization. Comparative analysis reveals clear limitations of general-purpose language models in TCM-specific tasks. DeepSeek’s scores (39.63, 14.58, 6.84, and 3.03) outperform ChatGLM2-6B (26.44, 7.35, 4.06, and 1.37) but remain inferior to some TCM-specialized models. This suggests that general models lack training data tailored to TCM knowledge, resulting in reduced accuracy and fluency in domain-specific contexts. Among TCM-specialized models, ShennongAlpha achieves the second-highest performance, leveraging its retrieval-augmented generation (RAG) architecture, which highlights the positive impact of knowledge retrieval mechanisms in TCM question-answering tasks. However, XuanHuGPT (LoRA) demonstrates superior adaptability to TCM-specific tasks, driven by the LoRA fine-tuning approach combined with the XhTCM dataset. In contrast, HuatuoGPT and MING-MOE exhibit relatively weaker performance, reflecting deficiencies in their training strategies for fine-grained TCM knowledge modeling. The comparison between ChatGLM2-6B (P-Tuning v2) and XuanHuGPT (LoRA) provides insights into the efficacy of different parameter-efficient fine-tuning (PEFT) methods. XuanHuGPT’s LoRA approach outperforms P-Tuning v2, particularly on BLEU-2 (25.12 vs. 14.04) and BLEU-3 (11.85 vs. 7.34). This indicates that LoRA’s low-rank decomposition of weight updates enables more effective adaptation of the model’s internal representations to TCM-specific knowledge, surpassing the prompt-based optimization of P-Tuning v2 in terms of accuracy and fluency. Performance evaluation on coverage metrics In terms of model coverage (ROUGE) metrics, XuanHuGPT (LoRA) achieved the highest scores, with ROUGE-1: 39.98, ROUGE-2: 19.06, and ROUGE-L: 32.24. These results demonstrate that XuanHuGPT excels in capturing word-level content (ROUGE-1), bigram-level sequences (ROUGE-2), and maintaining structural coherence (ROUGE-L). This highlights the model’s ability to generate responses that retain both the semantic and structural integrity of Traditional Chinese Medicine (TCM)-related content, which is crucial for professional TCM consultations (Table 3). Table 3Evaluation results on ROUGE metricsLLMsROUGE-1ROUGE-2ROUGE-LDeepSeek38.7512.3330.64ChatGLM2-6B33.3212.3724.29HuatuoGPT35.9214.8727.56ShennongAlpha39.1913.6030.65MING-MOE33.2112.2424.21ChatGLM2-6B(P-Tuning v2)38.2814.7128.37XuanHuGPT(LoRA)39.9819.0632.24 Table 3Evaluation results on ROUGE metricsLLMsROUGE-1ROUGE-2ROUGE-LDeepSeek38.7512.3330.64ChatGLM2-6B33.3212.3724.29HuatuoGPT35.9214.8727.56ShennongAlpha39.1913.6030.65MING-MOE33.2112.2424.21ChatGLM2-6B(P-Tuning v2)38.2814.7128.37XuanHuGPT(LoRA)39.9819.0632.24 Evaluation results on ROUGE metrics Among the TCM-specific models, ShennongAlpha ranked second with ROUGE-1: 39.19, ROUGE-2: 13.60, and ROUGE-L: 30.65, which are close to the performance of XuanHuGPT. Its retrieval-augmented generation (RAG) architecture enhances the retrieval capability of TCM knowledge, making it strong in content coverage. Notably, DeepSeek's ROUGE-1 (38.75) and ROUGE-L (30.64) scores are very close to those of the specialized model ShennongAlpha, indicating its strong generalization ability in content coverage and structural preservation. This similarity may be attributed to DeepSeek’s Mixture of Experts (MoE) architecture and Multi-Head Latent Attention (MLA) mechanism, which enable it to capture a wide range of semantic information when processing complex texts. However, its ROUGE-2 score (12.33) is significantly lower than that of XuanHuGPT and some other specialized models, reflecting its limitations in capturing specific sequential patterns within the TCM domain. This may be due to DeepSeek’s pre-training data being primarily focused on general domains, without dedicated optimization for TCM terminology, prescription patterns, or clinical diagnoses. While its MoE architecture and MLA mechanism enhance its semantic expression, its specialization and ability to capture sequential content in the TCM field remain insufficient compared to XuanHuGPT and ShennongAlpha. This further underscores the importance of domain-specific fine-tuning (such as LoRA and RAG) in improving the model's adaptability to specialized tasks. Performance evaluation on fluency/consistency/sensitivity metrics Table 4 presents the performance of various large language models (LLMs) on the METEOR, BERTScore, and Embedding Distance metrics, which respectively evaluate semantic fluency, semantic consistency, and sensitivity to subtle input variations. Table 4Evaluation results on METEOR, BERTScore, and embedding distance metricsLLMsMETEORBERTScoreEmbedding DistanceDeepSeek0.1050.7760.862ChatGLM2-6B0.1740.7750.854HuatuoGPT0.2200.7760.861ShennongAlpha0.1490.8920.829MING-MOE0.2180.7380.839ChatGLM2-6B(P-Tuning v2)0.1850.8500.866XuanHuGPT(LoRA)0.2600.8680.870 Table 4Evaluation results on METEOR, BERTScore, and embedding distance metricsLLMsMETEORBERTScoreEmbedding DistanceDeepSeek0.1050.7760.862ChatGLM2-6B0.1740.7750.854HuatuoGPT0.2200.7760.861ShennongAlpha0.1490.8920.829MING-MOE0.2180.7380.839ChatGLM2-6B(P-Tuning v2)0.1850.8500.866XuanHuGPT(LoRA)0.2600.8680.870 Evaluation results on METEOR, BERTScore, and embedding distance metrics XuanHuGPT (LoRA) continues to demonstrate strong performance across all three metrics, indicating its capability to generate fluent, semantically coherent, and context-sensitive responses tailored to Traditional Chinese Medicine (TCM) tasks. For TCM-specific LLMs, semantic consistency and sensitivity are critical for practical implementation in clinical decision support systems, underscoring their real-world significance. ChatGLM2-6B (P-Tuning v2) outperforms DeepSeek in both METEOR (0.185) and BERTScore (0.850), although it still falls short of XuanHuGPT (LoRA). DeepSeek shows competitive results in BERTScore and Embedding Distance, suggesting its potential as a general-purpose model. However, its lower METEOR score reflects limitations in generating fluent and semantically appropriate responses for domain-specific TCM tasks. The domain-specific TCM model ShennongAlpha achieves a notably high BERTScore (0.892), highlighting the effectiveness of its Retrieval-Augmented Generation (RAG) architecture in retrieving relevant information from TCM literature and enhancing semantic consistency. Nevertheless, its relatively low METEOR (0.149) and Embedding Distance (0.829) scores suggest limited ability in producing fluent and coherent responses, as well as in capturing subtle variations in input, such as nuanced symptom descriptions.。 Multi-dimensional comprehensive evaluation We employed a “Z-score + Sigmoid” approach to normalize the five evaluation metrics. This method combines Z-score standardization with a Sigmoid transformation, preserving the scaling benefits of Z-score normalization while compressing extreme values through the Sigmoid function, thereby enhancing robustness. The normalized results are presented in Table 5. Furthermore, to better visualize the overall performance of the models, a radar chart is plotted, as shown in Fig. 4. Table 5Z-score + Sigmoid normalized LLM metricsLLMsBLEU(accuracy)ROUGE(coverage)METEOR(fluency)BERTScore(consistency)Embeddingdistance(sensitivity)DeepSeek0.4900.5750.1670.3550.623ChatGLM2-6B0.1810.2260.4350.3520.493HuatuoGPT0.4130.4660.6550.3550.607ShennongAlpha0.6470.6290.3210.8010.156MING-MOE0.3830.2190.6460.2230.264ChatGLM2-6B(P-Tuning v2)0.5100.5640.4890.6620.683XuanHuGPT(LoRA)0.8500.8250.8050.7280.738Fig. 4Radar chart of models performance comparison Table 5Z-score + Sigmoid normalized LLM metricsLLMsBLEU(accuracy)ROUGE(coverage)METEOR(fluency)BERTScore(consistency)Embeddingdistance(sensitivity)DeepSeek0.4900.5750.1670.3550.623ChatGLM2-6B0.1810.2260.4350.3520.493HuatuoGPT0.4130.4660.6550.3550.607ShennongAlpha0.6470.6290.3210.8010.156MING-MOE0.3830.2190.6460.2230.264ChatGLM2-6B(P-Tuning v2)0.5100.5640.4890.6620.683XuanHuGPT(LoRA)0.8500.8250.8050.7280.738 Z-score + Sigmoid normalized LLM metrics ChatGLM2-6B (P-Tuning v2) Radar chart of models performance comparison XuanHuGPT (LoRA) demonstrates the largest radar chart area with a shape approximating a regular pentagon, indicating consistently strong performance across all five metrics and the best overall capability. Its high scores in BLEU, ROUGE, and METEOR highlight its advantages in generating accurate, comprehensive, and fluent TCM texts. The high score in Embedding Distance further supports its sensitivity to subtle semantic variations. ShennongAlpha shows a prominent peak in BERTScore, but significant contraction in METEOR and Embedding Distance, suggesting imbalanced performance—strong in semantic consistency but weaker in fluency and sensitivity. ChatGLM2-6B (P-Tuning v2) displays a more balanced radar shape, albeit with a smaller area, indicating that while it outperforms general-purpose models, it still lags behind XuanHuGPT (LoRA) on TCM-specific tasks. HuatuoGPT and MING-MOE exhibit relatively better performance in METEOR, but their radar charts are smaller and less comprehensive overall, reflecting limited general capability in TCM tasks. DeepSeek and ChatGLM2-6B (in general settings) present the smallest and most irregular radar chart areas, indicating relatively poor performance in TCM-specific tasks—especially in terms of fluency and semantic consistency. Performance evaluation on safety XuanHuGPT (LoRA) and baseline models were evaluated for safety using identical TCM-related questions from a shared test set. GPT-4o initially assessed responses across three safety dimensions, with scores validated by domain experts. Outcomes were classified as “Win,” “Tie,” or “Loss,” based on the models’ ability to deliver safe, accurate, and ethically sound TCM responses. XuanHuGPT (LoRA) outperformed general-purpose models like DeepSeek and ChatGLM2-6B, achieving high win rates (25%) and low loss rates (16.0%–17.0%). This advantage stems from LoRA fine-tuning on the XhTCM dataset, integrating classical TCM literature with modern pharmacology, enabling precise handling of TCM terminology, pathogenesis, and treatment strategies while minimizing overgeneralization and inaccuracies. Compared to ShennongAlpha, XuanHuGPT (LoRA) showed comparable safety performance, with balanced win and loss rates (25.0% each) and a high tie rate (50.0%). Against other TCM-specific models, it achieved superior win rates: 80.0% versus MING-MOE, 45.0% versus HuatuoGPT, and 41.0% versus ChatGLM2-6B (P-Tuning v2). XuanHuGPT (LoRA) consistently exhibited low loss rates, reflecting robust safety in TCM contexts. Its ability to perform syndrome differentiation and formulate treatment plans using authentic TCM logic underscores its professional depth and clinical applicability as a specialized TCM language model (Fig. 5).Fig. 5Evaluation results for safety Evaluation results for safety Model architecture and fine-tuning method analysis Model performance differences primarily stem from fine-tuning methods, architectural design, and dataset specificity.Differences between general-purpose and domain-specific fine-tuning. Differences between general-purpose and domain-specific fine-tuning. Differences between general-purpose and domain-specific fine-tuning. General-purpose models (e.g., DeepSeek and ChatGLM2-6B) are pretrained on broad-domain data, lacking specialized Traditional Chinese Medicine (TCM) content, making it challenging to deeply understand TCM terminology, disease classification, and treatment methods. Domain-specific models (e.g., XuanHuGPT(LoRA) and ShennongAlpha) significantly enhance TCM task performance through targeted fine-tuning on TCM knowledge. XuanHuGPT(LoRA) leverages the XhTCM dataset, integrating classical TCM literature, clinical knowledge, and pharmacological information, ensuring precise mastery of TCM theory and applications.(2)Comparison of fine-tuning methods (2)Comparison of fine-tuning methods Comparison of fine-tuning methods LoRA (XuanHuGPT(LoRA)) reduces storage demands via low-rank matrix decomposition, adjusting only newly added parameters to efficiently incorporate TCM knowledge while maintaining computational efficiency. Its strength lies in significantly improving adaptability to TCM tasks, particularly in scenarios requiring deep domain expertise. In contrast, P-Tuning v2 (ChatGLM2-6B(P-Tuning v2)) enhances task adaptability by optimizing prompt embeddings but focuses on prompt adjustments, struggling to capture intricate TCM details, resulting in inferior performance compared to LoRA.(3)Differences between RAG and MoE architectures (3)Differences between RAG and MoE architectures Differences between RAG and MoE architectures ShennongAlpha employs a Retrieval-Augmented Generation (RAG) architecture, which retrieves and integrates external knowledge into the generation process, enhancing content accuracy and semantic consistency, particularly for specialized TCM tasks. DeepSeek uses a Mixture of Experts (MoE) architecture, dynamically selecting expert models to improve computational efficiency. However, its domain adaptability is limited, lacking deep understanding of TCM terminology and treatment protocols, leading to suboptimal performance in TCM tasks compared to specialized models.(4)Knowledge integration and dataset differences (4)Knowledge integration and dataset differences Knowledge integration and dataset differences The XhTCM dataset, designed specifically for TCM tasks, integrates authoritative TCM resources, providing robust knowledge support for XuanHuGPT(LoRA). This enables superior performance across multiple metrics, generating fluent and professionally accurate TCM responses. In contrast, DeepSeek’s dataset focuses on general domains, lacking specialized TCM training, resulting in poorer performance in TCM-specific tasks. Qualitative analysis To more accurately evaluate model performance in practical Traditional Chinese Medicine (TCM) scenarios, we designed four representative tasks: herbal inquiry, symptom analysis, disease treatment, and prescription evaluation, corresponding to Appendix Tables 6, 7, 8, 9. Each scenario includes an expert-validated reference answer as a benchmark to assess the models’ capabilities in terms of professional accuracy, logical coherence, and clinical relevance. Herbal inquiry Case of Schizonepetae Carbonisatus (Appendix Table 6) In this task, XuanHuGPT (LoRA) was able to systematically describe the therapeutic indications, pharmacological mechanisms, usage methods, and contraindications of Schizonepetae Carbonisatus. Its response covered a wide range of relevant knowledge with accurate expression and terminology, closely aligning with classical TCM literature and expert reference standards. The model not only mentioned common effects such as “hemostasis,” “dispelling wind,” and “clearing heat,” but also clearly articulated appropriate applications and usage cautions—demonstrating both high-quality text generation and robust understanding of TCM-specific vocabulary. In contrast, DeepSeek displayed partial understanding of the herb’s processing methods and clinical indications, but lacked structural organization and reasoning. ShennongAlpha, benefiting from its RAG architecture, produced relatively authoritative content, though the responses tended to be rigid and linguistically repetitive. Other models—such as MING-MOE, HuatuoGPT, and ChatGLM2-6B—exhibited various degrees of deviation or generalization in describing pharmacological actions, and occasionally misused technical terms or misunderstood herbal properties. Overall, XuanHuGPT was the only model that consistently demonstrated reliable comprehension and professional expression in the domain of herbal knowledge. Symptom analysis Case of damp-phlegm type functional dyspepsia (Appendix Table 7) This task emphasized the models’ capacity for syndrome differentiation based on clinical symptoms. XuanHuGPT accurately identified the core pathogenesis of “spleen deficiency with dampness encumbrance” and proposed a treatment principle centered on “transforming dampness, harmonizing the middle, regulating Qi, and relieving nausea.” It also recommended commonly used formulas such as Er Chen Tang and Huo Xiang Zheng Qi San, reflecting a reasonable line of thought and a certain level of clinical applicability. Among other models, DeepSeek partially recognized the symptoms’ diagnostic category, but its explanation of pathogenesis was vague and lacked clear therapeutic prioritization. ShennongAlpha listed multiple herbal components but failed to justify their selection within a coherent diagnostic framework. ChatGLM2-6B and its P-Tuning v2 variant provided general descriptions without articulating the principle of “pattern differentiation and treatment.” Meanwhile, HuatuoGPT and MING-MOE tended to focus on lifestyle advice and offered health education-style content, with insufficient grounding in TCM theory. In summary, XuanHuGPT showed relatively robust performance in symptom attribution, pattern diagnosis, and formula selection, better aligning with the reasoning process required in clinical TCM contexts. Disease treatment Case of prescription design for gouty nephropathy (Appendix Table 8) This scenario evaluated the models’ ability to design treatment strategies tailored to specific disease patterns. XuanHuGPT combined pathogenesis concepts such as “damp-heat accumulation in the lower burner” and “kidney deficiency with dampness” to propose a dual-treatment approach. The suggested formulas—including Zhi Bai Di Huang Wan and Bi Xie Fen Qing Yin—were appropriately selected, and the rationale behind the composition was logically presented, consistent with core TCM treatment principles. ShennongAlpha provided a detailed list of herbs, but its treatment plan lacked a foundation in syndrome differentiation and failed to establish a link between pathogenesis and formula selection. DeepSeek’s solution was brief and superficial, with limited differentiation between disease subtypes and few modification strategies. MING-MOE and HuatuoGPT mainly adopted modern medical concepts such as “kidney function regulation” and “diuresis,” with limited application of layered TCM logic. ChatGLM2-6B generated only generic descriptions, reflecting a lack of domain-specific reasoning. XuanHuGPT demonstrated a relatively strong ability in disease mechanism interpretation and prescription logic. While there is still room to further standardize its expression, the model generally outperformed others in both plausibility and domain-specific accuracy. Prescription evaluation Case of pediatric asthma due to kidney Qi deficiency (Appendix Table 9) This task assessed the models’ capability to adjust existing formulas based on individual constitution and syndrome differentiation. XuanHuGPT accurately identified the core pattern of “deficiency of kidney Qi causing wheezing” and provided targeted suggestions for herb additions and subtractions. Its recommendations reflected an awareness of pediatric physiological characteristics and associated contraindications, indicating a moderate level of clinical reasoning and customization. Although ShennongAlpha proposed several herbal options, it failed to clearly relate them to the identified pattern. DeepSeek’s logic was weak, with inconsistencies in the properties of the suggested herbs. Other models—including ChatGLM2-6B, MING-MOE, and HuatuoGPT—mostly repeated standard formulas without demonstrating individualized modification based on syndrome presentation. XuanHuGPT showed the ability to tailor formulas in response to syndrome-specific insights, reflecting a higher level of task adaptability and basic diagnostic reasoning. However, its performance in handling more complex prescriptions and patient variability still requires further refinement. Conclusion and limitations This paper introduces XuanHuGPT, an AI-based large language model specifically designed for intelligent Traditional Chinese Medicine (TCM) diagnosis. The model supports TCM-related Q&A, enhances diagnostic and treatment decisions, and aids in health management and prevention. By utilizing the XhTCM dataset—a comprehensive and high-quality corpus that integrates authoritative TCM resources such as ShenNong_TCM_Dataset, TCMBank, and TCMIP v2.0—XuanHuGPT provides precise, contextually relevant answers to inquiries on TCM theories, clinical practices, herbal pharmacology, and prescription formulations. The model is optimized using the parameter-efficient fine-tuning (PEFT) technique, specifically LoRA and P-Tuning v2, based on the ChatGLM2-6B base model, achieving a balance between computational efficiency and domain-specific performance. By establishing a comprehensive evaluation framework, which includes quantitative metrics like BLEU, ROUGE, METEOR, BERTScore, and Embedding Distance, as well as expert qualitative assessments, XuanHuGPT significantly outperforms general-purpose models and other TCM-specific models in terms of accuracy, coverage, fluency, consistency, sensitivity, and safety. This study establishes a robust framework for TCM-specific large language models, contributing to the digitalization, intelligent dissemination, and global promotion of TCM knowledge. Despite these advances, XuanHuGPT has several limitations. The training process was constrained by hardware resources, and the relatively lightweight architecture of the base model may limit its performance in highly specialized TCM subfields. Additionally, while XuanHuGPT provides valuable insights, its outputs should not replace professional medical advice, and validation by TCM practitioners or healthcare professionals is essential for clinical applications. The model's reliance on structured data may also limit its ability to handle unstructured or multimodal inputs, such as real-time diagnostic data. Future work will focus on integrating the four diagnostic methods of TCM (observation, listening and smelling, questioning, and pulse-taking) to construct a more comprehensive multimodal TCM dataset, further enhancing the capabilities of XuanHuGPT. This will enable the model to process diverse data types and provide more comprehensive advisory services.