HemBLIP: A Vision–Language Model for Interpretable Leukemia Cell Morphology Analysis
Julie van Logtestijn, Petru Manescu · 2026
Read HemBLIP: A Vision–Language Model for Interpretable Leukemia Cell Morphology Analysis by Julie van Logtestijn, Petru Manescu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03915.txt.
Introduction
Julie van Logtestijn and Petru Manescu introduce HemBLIP, a vision-language model that describes leukemia cells in words a clinician can check. Judging white blood cell shape under the microscope is central to leukemia diagnosis, but most deep learning tools give verdicts without reasons, limiting trust. HemBLIP instead generates morphology-aware descriptions of peripheral blood cells.
The pair built a new dataset of fourteen thousand healthy and leukemic cells paired with expert-derived attribute captions. They adapted a general vision-language model by full fine-tuning and by a cheaper low-rank method, benchmarking against a biomedical foundation model. HemBLIP produced better captions and more accurate morphology, and the low-rank adaptation added gains at far lower computing cost. Interpretable cell descriptions could make automated blood analysis easier to audit in real laboratories.
Catalog record
| CREATOR | Julie van Logtestijn, Petru Manescu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 13 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-03915
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HemBLIP: A Vision–Language Model for Interpretable Leukemia Cell Morphology Analysis Julie van Logtestijn, Petru Manescu Abstract Microscopic evaluation of white blood cell morphology is central to leukemia diagnosis, yet current deep learning models often act as black boxes, limiting clinical trust and adoption. We introduce HemBLIP, a vision–language model designed to generate interpretable, morphology-aware descriptions of peripheral blood cells. Using a newly constructed dataset of ∼ \sim 14k healthy and leukemic cells paired with expert-derived attribute captions, we adapt a general-purpose VLM via both full fine-tuning and LoRA-based parameter-efficient training, and benchmark against the biomedical foundation model MedGEMMA. HemBLIP achieves higher caption quality and morphological accuracy, while LoRA adaptation provides further gains with significantly reduced computational cost. These results highlight the promise of vision–language models for transparent and scalable hematological diagnostics. † † address: 1 Department of Computer Science, University College London, United Kingdom 1 Introduction Leukemia remains a major global health concern, responsible for around 2.5% of all new cancer diagnoses and 3.1% of cancer deaths worldwide [ 21 ] . Despite advances in therapy, the disease continues to impose substantial burden even in high-income countries, with an incidence of roughly 14 per 100,000 and a mortality rate near 6 per 100,000 per year [ 14 ] . In low- and middle-income regions, survival rates can drop below 30%, largely due to delayed or missed diagnoses linked to workforce shortages and limited laboratory infrastructure [ 3 , 17 , 6 ] . Leukemia diagnosis relies on microscopical examination of peripheral blood smears (PBS) and bone marrow aspirates (BMA) by hematologists [ 18 ] . Morphological cues such as nuclear shape, chromatin texture, nucleoli, and cytoplasmic features are used to classify major subtypes—acute lymphoblastic (ALL), acute myelogenous (AML), chronic lymphocytic (CLL), and chronic myelogenous (CML) [ 5 ] . However, this process is labor-intensive, prone to human errors, and requires the availability of trained experts [ 2 ] . Recent studies have demonstrated the potential of deep learning vision models to detect leukemic cells in PBS and BMA [ 15 , 25 ] . However, these task-specific classifiers often require frequent retraining when imaging conditions, staining protocols, or cell morphology distributions vary, which is difficult to sustain in real-world and low-resource settings [ 20 , 13 ] . More critically, their black-box outputs typically provide only class predictions without articulating the morphological reasoning behind them. This lack of interpretability limits clinical trust, validation, and ultimately adoption in routine hematology workflows [ 1 , 8 ] . Here, we propose an explainable-by-design computational hematology approach underpinned by Vision Language Models (VLM) capable of generating narrative descriptions of cells from PBS and BMA. By jointly learning from images and text, VLMs associate visual patterns with descriptive language, enabling tasks such as image captioning and visual question answering [ 7 ] . Recent studies have shown that VLMs can be trained to generate full radiology reports and draft histopathology narratives directly from gigapixel slides [ 22 , 23 ] . We present HemBLIP, a vision–language modeling approach designed to generate morphology-aware descriptions of white blood cells in the context of leukemia diagnosis from blood film microscopy. We trained our model on a newly constructed morphology-rich dataset of \sim 14k cell–caption pairs and compared its performance with a medical foundation model (MedGEMMA) under full and Low-Rank Adaptation (LoRA) fine-tuning. 2 Materials and Methods 2.1 Dataset Construction We composed a morphology-aware dataset of peripheral blood smear images combining healthy and leukemic white blood cells (WBCs). For healthy cells, we used the WBCAtt dataset [ 24 ] , containing 10k expert-annotated WBCs (neutrophils, eosinophils, basophils, lymphocytes, monocytes) with 11 categorical attributes describing nuclear, cytoplasmic, and granular features. For leukemia, we used the LeukemiaAttri dataset [ 16 ] , which provides 10k labeled cells across five subtypes (ALL, AML, APML, CLL, CML) and seven expert-defined morphological attributes (e.g., chromatin texture, nucleoli visibility, basophilia). After filtering incomplete or erroneous entries, we selected 7,037 healthy and 7,622 leukemic cells. We further paired each cell image with a structured natural-language caption describing its morphology. Feature labels were transformed into templated sentences and augmented with GPT-4 paraphrases while constraining generation to retain factual morphological descriptors and diagnostic labels. The resulting corpus comprises \sim 14k image–text pairs covering normal and leukemic morphologies. Figure 2 illustrates two sample images and ground truth generated captions. 2.2 Model Architectures and Fine-tuning We propose HemBLIP, a vision–language model adapted from the general-purpose BLIP architecture [ 12 ] for the task of white blood cell morphology captioning. HemBLIP couples a ViT-based image encoder with a transformer text decoder and is fine-tuned to generate structured, clinically meaningful cell descriptions. To achieve efficient domain adaptation, we apply Low-Rank Adaptation (LoRA) [ 9 ] to the decoder attention layers while keeping the vision encoder frozen, enabling morphology-aware learning without extensive retraining. We additionally evaluate a fully fine-tuned HemBLIP variant to examine the trade-offs between parameter-efficient and full adaptation strategies. To contextualize performance, we compared HemBLIP against a biomedical foundation model, MedGEMMA-4B [ 19 ] , which combines a SigLIP vision tower with a medically aligned language decoder trained on diverse clinical image–text data. Model architectures are summarized in Figure 1 . All models were optimized using AdamW (learning rate 5\times 10^{-5} ) with early stopping based on validation loss. The implementation is available at: https://github.com/julievanlogtestijn-ucl/Vision-Language-Models-for-Leukemia-Detection/. (a) BLIP architecture overview. The ViT-based vision encoder and transformer text decoder are aligned through contrastive and captioning objectives. (b) MedGEMMA architecture overview. A SigLIP vision tower is coupled with a medically tuned decoder trained on multimodal clinical data. Figure 1: Comparison of the two vision–language architectures used in this study. Both were fine-tuned on the morphology-aware leukemia dataset using LoRA adapters for parameter-efficient adaptation. 2.3 Clinical Relevance Evaluation Standard captioning metrics such as BLEU, ROUGE-L, and BERTScore [ 26 , 11 ] were computed to assess lexical and semantic similarity with reference captions. To quantify morphological faithfulness, we developed a regex-based attribute extractor that performs controlled string matching over generated captions to identify mentions of predefined cytological attributes (e.g., cell size, chromatin pattern, nucleoli, basophilia). Extracted attributes were compared to ground-truth labels, and confusion matrices were used to evaluate correspondence and categorize biologically plausible errors (e.g., coarse vs. open chromatin, small vs. medium cell size). To probe whether models encode diagnostically meaningful visual information, we trained lightweight cosine-similarity classifier heads on frozen image embeddings to distinguish between leukemia subtypes and cell types. 2.4 External Validation An additional set of 507 cell images from the Blood Cell Atlas [ 4 ] , a public Kaggle collection [ 10 ] , and the low-cost microscope subset of the LeukemiaAttri dataset [ 16 ] was used for robustness testing under domain shift. These samples include both healthy and malignant cells imaged under varied staining and sensor conditions, allowing evaluation of real-world generalization. 3 Results We evaluated captioning performance, morphological interpretability, and representation quality across internal and external test sets. Example model outputs are shown in Fig. 2 . Figure 2: Example blood cell inputs with ground-truth captions and model-generated descriptions. Each model exhibits varying degrees of morphological specificity and diagnostic accuracy. 3.1 Caption Generation Performance Table 1 reports quantitative caption generation metrics on both test sets. LoRA-adapted