Med-V1: Small Language Models for Zero-shot and Scalable Biomedical Evidence Attribution
Qiao Jin, Yin Fang, Lauren He, Yifan Yang, Guangzhi Xiong, Zhizheng Wang, Nicholas Wan, Joey Chan, Donald C. Comeau, Robert Leaman, Charalampos S. Floudas, Aidong Zhang, Michael F. Chiang, Yifan Peng, Zhiyong Lu · 2026
Read Med-V1: Small Language Models for Zero-shot and Scalable Biomedical Evidence Attribution by Qiao Jin, Yin Fang, Lauren He, Yifan Yang, Guangzhi Xiong, Zhizheng Wang, Nicholas Wan, Joey Chan, Donald C. Comeau, Robert Leaman, Charalampos S. Floudas, Aidong Zhang, Michael F. Chiang, Yifan Peng, Zhiyong Lu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05308.txt.
Introduction
Qiao Jin and team present Med-V1, small three-billion-setting evidence-attribution models. Frontier models cost too much. Trained on synthetic data, Med-V1 beats its base models on five benchmarks, matches GPT-5 with good accounts. A case study quantifies hallucinations across citation instructions, finding format affects validity, with GPT-5 making more claims at GPT-4o-like hallucination rates. A second case study finds Med-V1 spotting high-stakes clinical-guideline misattributions, revealing negative health impacts. Med-V1 offers a lean, lightweight frontier-model alternative for biomedical evidence attribution.
Catalog record
| CREATOR | Qiao Jin, Yin Fang, Lauren He, Yifan Yang, Guangzhi Xiong, Zhizheng Wang, Nicholas Wan, Joey Chan, Donald C. Comeau, Robert Leaman, Charalampos S. Floudas, Aidong Zhang, Michael F. Chiang, Yifan Peng, Zhiyong Lu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61454 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.05308
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Med-V1: Small Language Models for Zero-shot and Scalable Biomedical Evidence Attribution Qiao Jin, Yin Fang, Lauren He, Yifan Yang, Guangzhi Xiong, Zhizheng Wang, Nicholas Wan, Joey Chan, Donald C. Comeau, Robert Leaman, Charalampos S. Floudas, Aidong Zhang, Michael F. Chiang, Yifan Peng, Zhiyong Lu Abstract Assessing whether an article supports an assertion is essential for hallucination detection and claim verification. While large language models (LLMs) have the potential to automate this task, achieving strong performance requires frontier models such as GPT-5 that are prohibitively expensive to deploy at scale. To efficiently perform biomedical evidence attribution, we present Med-V1, a family of small language models with only three billion parameters. Trained on high-quality synthetic data newly developed in this study, Med-V1 substantially outperforms (+27.0% to +71.3%) its base models on five biomedical benchmarks unified into a verification format. Despite its smaller size, Med-V1 performs comparably to frontier LLMs such as GPT-5, along with high-quality explanations for its predictions. We use Med-V1 to conduct a first-of-its-kind use case study that quantifies hallucinations in LLM-generated answers under different citation instructions. Results show that the format instruction strongly affects citation validity and hallucination, with GPT-5 generating more claims but exhibiting hallucination rates similar to GPT-4o. Additionally, we present a second use case showing that Med-V1 can automatically identify high-stakes evidence misattributions in clinical practice guidelines, revealing potentially negative public health impacts that are otherwise challenging to identify at scale. Overall, Med-V1 provides an efficient and accurate lightweight alternative to frontier LLMs for practical and real-world applications in biomedical evidence attribution and verification tasks. Med-V1 is available at https://github.com/ncbi-nlp/Med-V1 . 1 Introduction Evidence attribution, also known as verification, is a foundational biomedical task that assesses whether an assertion is supported by a source document [ 1 ] . A common example is automatic fact-checking. Such systems usually retrieve evidence relevant to a given claim and then apply a verification model to determine the claim’s veracity based on that evidence [ 2 , 3 , 4 ] . The need for robust verification has grown with the increasing use of generative artificial intelligence (AI) tools such as ChatGPT, which are known to hallucinate when generating statements with citations [ 5 , 6 , 7 ] . As a result, their outputs must be thoroughly verified before downstream usage [ 8 , 9 , 10 ] , especially in high-stakes domains such as healthcare [ 11 ] . While frontier large language models (LLMs) have shown strong performance across various biomedical tasks [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ] such as question answering [ 20 , 21 , 22 ] and clinical trial matching [ 23 , 24 , 25 ] , they often rely on commercial systems and are prohibitively expensive for routine, large-scale deployment. With the rapid growth of AI-generated statements, it is conceivable that verification becomes a heavily used utility task. In practice, such large-scale verification can only be realistically performed by small language models [ 26 ] , which are more cost-effective and scalable. As such, the objective of this work is to develop a lightweight foundation model that can perform biomedical verification comparably with frontier LLMs, generalize well to unseen data distribution, and provide faithful explanations for its predictions. To this end, we first introduce a novel LLM-driven pipeline that generates MedFact-Synth, a large-scale synthetic dataset comprising biomedical claims and their high-quality verification reports. Using MedFact-Synth, we subsequently fine-tune small LLMs with only three billion parameters [ 27 , 28 ] , resulting in Med-V1. For evaluation, we compile MedFact-Bench, a benchmark consisting of five standard and re-purposed biomedical verification datasets [ 29 , 30 , 31 , 32 , 33 ] . Experimental results show that Med-V1 substantially outperforms its base models, achieving improvements of 27.0% to 71.3% across the datasets. Despite its small size, Med-V1 reaches performance levels comparable to frontier LLMs such as GPT-5 [ 34 ] . Further analysis shows that Med-V1 consistently presents high-quality explanations for its predictions. To demonstrate the real-world utility of Med-V1, we conduct two large-scale use case studies that analyze evidence attribution in both LLM-generated answers and high-stakes clinical practice guidelines. First, we apply Med-V1 to LLM-generated answers under seven citation-instruction formats by extracting claim-citation pairs and verifying each claim against its cited article, which enables large-scale quantification of citation validity and hallucination rates across formats. This analysis shows that citation format strongly affects citation validity and hallucination, and that GPT-5 generates more claims but exhibits hallucination rates similar to its predecessor GPT-4o. Second, we apply Med-V1 to citation statements in recent clinical practice guidelines and identify a nontrivial subset whose cited sources do not support the corresponding claims; after manually validating 100 flagged cases, we find that a significant proportion of them are genuine misattributions, and many involve treatment effects that might lead to potential risks in public health. In summary, these results show that Med-V1 achieves frontier-level verification performance while remaining compact and computationally efficient. This makes it a practical and accessible alternative to proprietary LLMs for large-scale biomedical verification, hallucination detection, and citation auditing. 2 Results 2.1 Overview of Med-V1 Training and Inference This work contains two phases: (i) synthetic data generation and model training (Figure 1 a) and (ii) Med-V1 inference for downstream tasks (Figure 1 b). Phase I: Synthetic data generation and model training. In the first phase, we construct MedFact-Synth, a large-scale synthetic training set including 1.5 million instances. Each instance contains: a synthetic claim to be verified, a source article serving as evidence, a rationale explaining the verification, and a 5-point Likert-scale verdict, ranging from strong contradiction (-2) and partial contradiction (-1) to neutral (0), partial agreement (+1), and strong agreement (+2). To build this dataset, we begin by sampling one million articles from the PubMed 2025 baseline [ 35 ] . For each article, GPT-4o-mini [ 36 ] is prompted to generate two claims: one that the article may support and one that it may refute. To collect diverse claim-article pairs for verification, we use MedCPT [ 37 ] to retrieve the top 10 most relevant PubMed articles for each claim. A panel of frontier LLMs then verifies each pair, generating both rationales and verdicts via a voting-based mechanism. Then, we utilize MedFact-Synth to fine-tune Med-V1, which is a three-billion-parameter (3B) small LLM available in two variants: Med-V1-L3B, initialized from Llama-3.2-3B-Instruct [ 27 ] and Med-V1-Q3B, initialized from Qwen2.5-3B-Instruct [ 28 ] . The quality of MedFact-Synth is examined in the next section, and technical details of the dataset construction and training process are described in the Methods section. Figure 1: Overview of Med-V1 training and inference . a : MedFact-Synth construction and Med-V1 training. Synthetic claims are generated from source papers and then verified by a panel of LLMs using relevant papers retrieved from PubMed. The resulting verified claim-evidence pairs form the MedFact-Synth dataset, which is then used to train Med-V1 through a combination of supervised fine-tuning and reinforcement learning. b : Inference with Med-V1. Given an assertion and a source biomedical article, Med-V1 assesses whether the article supports the assertion. The assertions can be derived from Boolean questions, factual claims, or citation statements,