LLaMA-XR: A Novel Framework for Radiology Report Generation using LLaMA and QLoRA Fine Tuning Journal: Elsevier

Md. Zihad Bin Jahangir, Muhammad Ashad Kabir, Sumaiya Akter, Israt Jahan, Minh Chau · 2025

Read LLaMA-XR: A Novel Framework for Radiology Report Generation using LLaMA and QLoRA Fine Tuning Journal: Elsevier by Md. Zihad Bin Jahangir, Muhammad Ashad Kabir, Sumaiya Akter, Israt Jahan, Minh Chau (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-03178.txt.

Introduction

Writing chest X-ray reports eats radiologist time, and a tuned language model may draft them with clinical sense. Md Zihad Bin Jahangir, Muhammad Ashad Kabir, Sumaiya Akter, Israt Jahan, and Minh Chau present LLaMA-XR in a 2025 paper. Their system pairs a vision encoder with the LLaMA 3.1 model. A memory-light tuning method, QLoRA, adapts the model without full retraining. A public Indiana X-ray set judged the drafts.

Scores on two standard text measures set new marks for the task, the team reports. Coherence and clinical accuracy both improved over rival systems. The tuning strategy cuts memory needs, speeding generation. The authors pitch the tool for easing reporting loads in busy departments.

Catalog record

CREATORMd. Zihad Bin Jahangir, Muhammad Ashad Kabir, Sumaiya Akter, Israt Jahan, Minh Chau
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-03178

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LLaMA-XR: A Novel Framework for Radiology Report Generation using LLaMA and QLoRA Fine Tuning Journal: Elsevier Md. Zihad Bin Jahangir, Muhammad Ashad Kabir, Sumaiya Akter, Israt Jahan, Minh Chau Abstract Automated radiology report generation holds significant potential to reduce radiologists’ workload and enhance diagnostic accuracy. However, generating precise and clinically meaningful reports from chest radiographs remains challenging due to the complexity of medical language and the need for contextual understanding. Existing models often struggle with maintaining both accuracy and contextual relevance. In this paper, we present LLaMA-XR, a novel framework that integrates LLaMA 3.1 with DenseNet-121-based image embeddings and Quantized Low-Rank Adaptation (QLoRA) fine-tuning. LLaMA-XR achieves improved coherence and clinical accuracy while maintaining computational efficiency. This efficiency is driven by an optimization strategy that enhances parameter utilization and reduces memory overhead, enabling faster report generation with lower computational resource demands. Extensive experiments conducted on the IU X-ray benchmark dataset demonstrate that LLaMA-XR outperforms a range of state-of-the-art methods. Our model achieves a ROUGE-L score of 0.433 and a METEOR score of 0.336, establishing new performance benchmarks in the domain. These results underscore LLaMA-XR’s potential as an effective and efficient AI system for automated radiology reporting, offering enhanced clinical utility and reliability. 1 Introduction Medical imaging plays a crucial role in diagnosing diseases by providing detailed visual representations of internal structures [ 1 ] . Among various radiological imaging modalities, chest X-ray (CXR) is one of the most commonly used techniques for diagnosing a wide range of pulmonary and cardiovascular conditions [ 2 ] . Given its widespread use in clinical practice, CXR reports are essential for accurately interpreting specific abnormalities and guiding treatment decisions [ 3 ] . However, generating these reports is a labor-intensive process that demands meticulous analysis of radiological images, making it both time-consuming and susceptible to errors [ 4 ] . With the advancement of deep neural networks, automatic radiology report generation has emerged as a promising tool to assist radiologists [ 5 ] . This approach requires models to generate accurate, coherent, and medically precise reports based on radiological images, significantly reducing workload and improving efficiency. Early studies [ 6 , 7 ] in automated radiology report generation were largely inspired by recurrent neural network (RNN)-based image captioning architectures. While these methods achieved initial success, they struggled with handling long and complex sentences [ 8 , 9 ] , which are crucial for generating detailed and clinically relevant reports. To overcome these challenges, recent advances have focused on the use of large language models (LLMs), which have benefited from extensive pre-training on diverse datasets and demonstrate superior performance in generating radiology reports [ 10 , 11 , 12 , 13 , 14 , 15 , 16 ] . Those LLM-based approaches, such as GPT [ 17 ] , BERT [ 18 ] , and LLaMA [ 19 ] have shown significant promise in capturing intricate medical contexts and producing high-quality reports. However, earlier versions of these models, such as GPT-2 (up to 1.5B parameters) and BERT-Large (340M parameters), often lacked the capacity to effectively handle the complexity of medical data due to their relatively smaller scale. This limitation underscores the need for a more advanced framework to better align with the demands of medical report generation. In this study, we propose a novel framework for automated chest X-ray report generation that addresses the challenge of effectively handling the complexity of medical data. Our approach takes advantage of the LLaMA 3.1 8B parameter model [ 20 ] , which offers superior performance in generating coherent, contextually rich, and medically precise reports. Compared to its predecessors, LLaMA 3.1 benefits from enhanced training data and a significantly larger parameter scale, enabling it to better capture the nuances of medical language [ 21 , 22 ] . To extract visual features from CXR images, several techniques can be employed, including convolutional neural networks (CNNs) such as DenseNet, ResNet, and Vision Transformers. In this study, we utilize DenseNet-121 [ 23 ] , a CNN architecture known for its ability to reuse features across layers and provide detailed image embeddings [ 24 , 21 ] . However, this choice is flexible, and other architectures, such as ResNet, EfficientNet, or Vision Transformers, can be incorporated depending on specific requirements. By combining DenseNet-121 with LLaMA 3.1, our framework effectively bridges the gap between visual data and textual report generation. Furthermore, we employ fine-tuning techniques such as QLoRA [ 25 ] , which enables efficient adaptation of large language models by significantly reducing memory requirements through low-rank adaptation, allowing us to fine-tune LLaMA 3.1 on medical datasets without the high computational cost of full fine-tuning. We use Supervised Fine-Tuning (SFT) to adapt LLaMA 3.1 to medical datasets, ensuring that the generated reports are accurate and clinically relevant. For this study, we used the IU X-ray dataset [ 26 ] , a widely used benchmark dataset that contains chest X-ray images paired with the corresponding radiology reports. This dataset enables effective model training and evaluation by providing diverse medical cases with detailed annotations. This fine-tuning process not only improves the quality of report generation but also reduces the time and computational resources required for training. Our framework is designed to be scalable and efficient, making it suitable for real-world applications where accuracy and interpretability are paramount. The key contributions of this study are as follows: 1. We propose a novel framework for generating accurate and clinically relevant medical reports from chest X-ray images, leveraging the capabilities of LLaMA 3 as the core language model. 2. We demonstrate that fine-tuning LLaMA 3 using QLoRA and Supervised Fine-Tuning (SFT) enables effective adaptation to medical imaging datasets, resulting in improved report quality while significantly reducing computational time and energy consumption. 3. We conduct a comprehensive evaluation of our framework against state-of-the-art (SOTA) methods, assessing both quantitative performance metrics and the clinical quality of the generated reports. 2 Related work Automated radiology report generation has garnered significant attention in recent years, with a wide array of deep learning architectures being proposed to enhance diagnostic accuracy, linguistic coherence, and computational efficiency. This section reviews the evolution of this field, focusing on methods applicable to generate chest X-ray report. We categorize recent work into four main classes: graph-based models, reward-based learning approaches, transformer-based architectures, and large language models (LLMs). Table 1 presents a comparison of these models, summarizing their embedding strategies, architectural designs, and use of pre-trained language models. Table 1: Summary of Radiology Report Generation Methods Method Image embedding Model Pre-trained Fine-Tuning Transformer [ 27 ] Transformer Transformer ✓ Full H-Agent [ 28 ] DenseNet Reward Based - - CMAS-RL [ 29 ] ResNet-50 Reward Based - - SentSAT+KG [ 30 ] DenseNet-121 Graph Model - - R2Gen [ 31 ] ResNet-101 Custom Transformer × × PPKED [ 32 ] ResNet-152 Custom Transformer × × SGF [ 33 ] ResNet-101 Custom Transformer ✓ Full CMCL [ 34 ] ResNet-50 Curriculum Learning - - R2GenCMN [ 35 ] ResNet-101 Custom Transformer × × JPG [ 36 ] ResNet-101 Custom Transformer × × PGT [ 37 ] ResNet-101 Custom Transformer × × ITA [ 38 ] ResNet-101 Custom Transformer × × CvT-DistilGPT2 [ 21 ] CvT-21 DistilGPT2 × × MCSAM [ 22

Read the complete text