Reinforcement Learning Improves LLM Accuracy and Reasoning in Disease Classification from Radiology Reports
Yishu Wei, Yi Lin, Adam Flanders, George Shih, Yifan Peng · 2026
Read Reinforcement Learning Improves LLM Accuracy and Reasoning in Disease Classification from Radiology Reports by Yishu Wei, Yi Lin, Adam Flanders, George Shih, Yifan Peng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19060.txt.
Introduction
Wei and four colleagues propose a two stage approach to reading diseases from radiology reports. Fine tuning of lightweight language models lifts accuracy but can harm reasoning. Their method first fine tunes on disease labels, then applies Group Relative Policy Optimization to refine predictions by accuracy and format alone, with no reasoning labels. Across three expert labeled data sets, tuning beats baselines, and the second stage further lifts classification while boosting recall and breadth.
Catalog record
| CREATOR | Yishu Wei, Yi Lin, Adam Flanders, George Shih, Yifan Peng |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54503 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.19060
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reinforcement Learning Improves LLM Accuracy and Reasoning in Disease Classification from Radiology Reports Yishu Wei, Yi Lin, Adam Flanders, George Shih, Yifan Peng Abstract Accurate disease classification from radiology reports is essential for many applications. While supervised fine-tuning (SFT) of lightweight LLMs improves accuracy, it can degrade reasoning. We propose a two-stage approach: SFT on disease labels followed by Group Relative Policy Optimization (GRPO) to refine predictions by optimizing accuracy and format without reasoning supervision. Across three radiologist-annotated datasets, SFT outperformed baselines and GRPO further improved classification and enhanced reasoning recall and comprehensiveness. Main Text Recent advances in deep learning-driven computer-aided diagnosis have underscored the crucial role of high-quality and diverse medical imaging datasets in model development. However, such datasets remain scarce, particularly with respect to comprehensive annotation sets that provide expert-curated labels of abnormal findings in radiographs. The generation of these annotations has traditionally been both labor-intensive and time-consuming. To address this bottleneck, increasing attention has been directed toward natural language processing (NLP) and, more recently, large language models (LLMs), as a means of automatically extracting and categorizing diagnostic information from free-text radiology reports [ 1 , 2 ] . However, the adoption of LLMs in disease classification remains constrained by practical considerations. Using external Application Programming Interfaces (APIs) reduces the need for specialized on-site hardware, but raises significant concerns about data privacy and security. In contrast, local hosting mitigates privacy risks but often demands substantial computational resources. In this context, lightweight LLMs offer a promising compromise: their smaller parameter counts facilitate on-premise deployment and enable fine-tuning on limited, task-specific datasets to achieve optimal performance in clinical settings [ 3 , 4 ] . Supervised fine-tuning (SFT) has emerged as a prominent approach for optimizing the performance of lightweight LLMs [ 5 , 6 ] . In disease classification tasks, SFT has been shown to allow lightweight LLMs to achieve accuracies comparable to those of larger and more resource-intensive models [ 7 ] . However, SFT also introduces a key limitation: it may degrade the model’s ability to generate explicit and coherent reasoning paths, in part because fine-tuning typically omits detailed reasoning or intermediate annotation labels. This limitation is particularly consequential in medical domains, such as radiology, where transparency and explainability are crucial for clinical decision-making and for fostering trust in algorithmic recommendations. Insufficient reasoning capabilities thus represent a significant barrier to the broader adoption of LLMs in the radiology domain. To overcome this challenge, we propose a multi-stage training and inference framework to improve the reasoning capabilities of lightweight LLMs for disease classification from radiology reports (Figure 1 ). In the first stage, SFT is performed using only disease labels, allowing the model to adapt to the radiology domain under outcome supervision. In the second stage, we apply a reinforcement learning algorithm (i.e., Group Relative Policy Optimization [GRPO]) on the same dataset without requiring explicit reasoning supervision [ 8 , 9 , 10 ] . Instead, we used a rule-based reward function to verify both the classification accuracy and formatting correctness . In particular, the formatting reward assigns a value of zero when no valid reasoning can be extracted in the specified output format, thereby encouraging the model to generate explicit reasoning alongside disease labels. Figure 1 : Two-stage training and inference workflow. During inference, the framework generates multiple outputs for each radiology report, each comprising both diseases and corresponding reasoning. Final disease labels are determined via majority voting, thereby improving the classification robustness and reliability. To consolidate the reasoning component, the original (unfine-tuned) lightweight LLM is prompted to synthesize all generated explanations into a single, coherent rationale. In this study, we utilized three datasets of de-identified, structured radiology reports: MIMIC-CXR, NIH-CXR [ 11 ] , and MIDRC [ 12 ] (Table 1 ). For all evaluation datasets, radiologists were instructed to re-examine the entire report and list all disease labels that they could identify. In addition, we evaluated three lightweight LLMs capable of local deployment: LLaMA 3.1-8B-Instruct (LLaMa) [ 13 ] , Qwen 2.5-3B-Instruct (Qwen) [ 14 ] , and Phi-3 Min-128K-Instruct (Phi3) [ 15 ] . Therefore, across the three gold-standard datasets, a total of nine evaluation cohorts were defined. Table 1 : Statistics of datasets. card. - cardiomediastinum. MIMIC-CXR Disease Training Reasoning Evaluation Eval-Silver NIH-CXR MIRDC Usage Fine-tuning & GRPO training Fine-tuning (reasoning) Evaluation Evaluation Evaluation Evaluation Annotator GPT4-o GPT4-o Radiologists GPT4-o Radiologists Radiologists Reports 2,000 200 100 1,000 50 50 Average length 47.9 45.8 60 48.1 52 58.3 Average labels 1.59 1.65 1.93 1.61 2.52 2.59 Disease Atelectasis 0.203 0.230 0.270 0.217 0.120 0.000 Cardiomegaly 0.176 0.215 0.350 0.198 0.010 0.060 Consolidation 0.035 0.015 0.020 0.033 0.040 0.020 Edema 0.084 0.090 0.180 0.088 0.060 0.000 Enlarged card. 0.011 0.010 0.010 0.010 0.000 0.000 Fracture 0.031 0.040 0.060 0.027 0.020 0.000 Lung lesion 0.009 0.005 0.000 0.011 0.020 0.000 Lung opacity 0.173 0.150 0.330 0.196 0.760 0.980 Pleural effusion 0.201 0.215 0.310 0.205 0.620 0.780 Pleural other 0.006 0.010 0.000 0.003 0.000 0.000 Pneumonia 0.033 0.020 0.050 0.028 0.020 0.020 Pneumothorax 0.025 0.040 0.030 0.025 0.100 0.020 Support devices 0.200 0.205 0.250 0.212 0.680 0.700 For the disease classification task, we evaluated performance using micro-averaged precision, recall, and F1 score (detailed in Section Evaluation metrics). While multiple inference is a significant contribution of our framework, we applied the same ensemble of multiple predictions to the baseline and SFT models to rule out its impact. Ensembling almost consistently improves the performance of baseline and SFT as well (Supplementary Table S1 ). To assess the quality of generated reasoning, we propose two metrics: reasoning recall , which measures the proportion of ground-truth diseases referenced in the reasoning, and reasoning comprehensiveness , which quantifies the proportion of predicted diseases that are supported by explicit reasoning. Figure 2 (Supplementary Table S2 ) demonstrates that SFT consistently outperforms the baseline across all cohorts, and the addition of GRPO yields further micro-F1 improvements in eight of nine cohorts, with the largest gains observed for Qwen on MIMIC-CXR (13.2%), Phi3 on NIH-CXR (12.8%), and LLaMA on MIDRC (8.0%). Notably, for LLaMA, the final classification accuracy nearly matches that of GPT-4o, which served as the teacher model for generating all training supervision during the SFT stage. Figure 2 : Disease detection performance on the MIMIC-CXR, NIH-CXR, and MIDRC evaluation dataset. SFT - Supervised Fine-Tuning. SFT+GRPO - Supervised Fine-Tuning followed by Group Relative Policy Optimization. Significance levels: * - p<0.05 ; ** - p<0.01 ; *** - p<0.001 ; **** - p<0.0001 ; ns - Not significant. The error bars are 95% CIs. To increase the sample size of the evaluation data, we constructed a MIMIC-CXR-Evaluation-Silver subset comprising 1,000 randomly selected reports from the remaining dataset, with labels curated by GPT-4o. As shown in Table 2 , the results on this silver-standard subset are consistent with those obtained on the gold-standard evaluation sets. However, the performance advantage of GRPO is less pronounced, likely because the evaluation labels were generated by the same model used for SFT, rendering them susceptible to similar labeling errors. To assess the impact of model scale, we applied our framework to Qwen-7B and compared it with the Qwen-3B across all three cohorts (Supplementary Table 1). Qwen-7B consistently outperformed its smaller