Dual-Encoder Transformer-Based Multimodal Learning for Ischemic Stroke Lesion Segmentation Using Diffusion MRI
Muhammad Usman 1 , Azka Rehman 2 , Muhammad Mutti Ur Rehman 3 , Abd Ur Rehman 4 , Muhammad Umar Farooq 5 · 2025
Read Dual-Encoder Transformer-Based Multimodal Learning for Ischemic Stroke Lesion Segmentation Using Diffusion MRI by Muhammad Usman 1 , Azka Rehman 2 , Muhammad Mutti Ur Rehman 3 , Abd Ur Rehman 4 , Muhammad Umar Farooq 5 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-20436.txt.
Introduction
Muhammad Usman and co-authors segment stroke lesions in diffusion MRI scans with a dual-encoder Transformer. Diffusion-weighted imaging and apparent diffusion coefficient maps show complementary views of acute and subacute damage. Separating them by hand is slow, and lesion appearance varies enough to defeat simple rules.
Working on the ISLES 2022 dataset, the team first benchmarks U-Net variants, Swin-UNet, and TransUNet under one training approach. Transformers beat the convolutional benchmarks. Their proposed design gives each modality its own encoder inside a TransUNet and feeds three adjacent slices for spatial context. The dual-encoder model tops the comparison, reaching a strong Dice score on the test set. Automated delineation of this quality can speed clinical assessment.
Catalog record
| CREATOR | Muhammad Usman 1 , Azka Rehman 2 , Muhammad Mutti Ur Rehman 3 , Abd Ur Rehman 4 , Muhammad Umar Farooq 5 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-20436
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dual-Encoder Transformer-Based Multimodal Learning for Ischemic Stroke Lesion Segmentation Using Diffusion MRI Muhammad Usman 1 , Azka Rehman 2 , Muhammad Mutti Ur Rehman 3 , Abd Ur Rehman 4 , Muhammad Umar Farooq 5 Abstract Accurate segmentation of ischemic stroke lesions from diffusion magnetic resonance imaging (MRI) is essential for timely clinical decision-making, treatment planning, and longitudinal outcome assessment. Diffusion-Weighted Imaging (DWI) and Apparent Diffusion Coefficient (ADC) scans provide complementary information regarding acute and sub-acute ischemic changes; however, reliable automated delineation of infarct regions remains challenging due to variability in lesion size, shape, and location. In this study, we investigate ischemic stroke lesion segmentation using multimodal diffusion MRI from the ISLES 2022 dataset. We first benchmark several state-of-the-art convolutional and transformer-based segmentation architectures, including U-Net variants, Swin-UNet, and TransUNet. Based on comparative performance, we further propose a dual-encoder TransUNet architecture designed to explicitly model modality-specific representations from DWI and ADC images. To enhance spatial contextual awareness, we incorporate adjacent slice information using a three-slice input configuration. All models are trained using a unified preprocessing and optimization strategy and evaluated using the Dice Similarity Coefficient (DSC). Experimental results demonstrate that transformer-based architectures outperform conventional convolutional models. The proposed dual-encoder TransUNet achieves consistent performance gains over single-encoder early-fusion approaches, with the three-slice dual-encoder configuration achieving the highest segmentation accuracy, attaining a Dice score of 85.4% on the test set. These findings highlight the importance of effective multimodal fusion and contextual representation for ischemic stroke lesion segmentation. The proposed framework provides a robust and scalable solution for automated infarct segmentation from diffusion MRI and establishes a strong baseline for future multimodal neuroimaging research. 1 Introduction Ischemic stroke remains one of the leading causes of mortality and long-term disability worldwide, imposing a substantial clinical and socioeconomic burden. Early and accurate identification of infarcted tissue is essential for treatment selection, prognosis estimation, and longitudinal disease management. Diffusion-weighted imaging (DWI) and apparent diffusion coefficient (ADC) MRI play a central role in acute and sub-acute stroke assessment due to their high sensitivity to early ischemic changes. However, manual delineation of ischemic lesions from diffusion MRI is labor-intensive and subject to inter- and intra-observer variability, motivating the development of reliable automated segmentation approaches. Over the past decade, deep learning has emerged as a powerful paradigm for medical image analysis, enabling substantial improvements across segmentation, detection, and classification tasks. Convolutional neural networks (CNNs) have demonstrated strong performance in a wide range of anatomical and pathological segmentation problems, including mandibular canal segmentation in CBCT scans, lung nodule analysis, and brain tumor segmentation [ 22 , 19 , 11 , 13 ] . These approaches commonly integrate attention mechanisms, deep supervision, and multi-scale feature extraction to enhance boundary localization and robustness to anatomical variability. Beyond conventional single-encoder CNN architectures, increasing attention has been directed toward multi-encoder and dual-encoder designs, which explicitly disentangle feature learning pathways for heterogeneous inputs. Such architectures have been shown to be particularly effective in scenarios involving multimodal data or complex appearance variations. For example, dual-encoder and multi-encoder networks have achieved improved performance in lung nodule segmentation and detection through adaptive attention mechanisms, self-distillation, and multi-view fusion strategies [ 25 , 23 , 21 , 5 ] . These findings suggest that naive early fusion via channel concatenation may limit representational capacity when different modalities encode complementary clinical information. Attention mechanisms and adaptive region-of-interest (ROI) modeling have further advanced the state of the art in medical image segmentation by suppressing irrelevant background information and focusing learning on clinically meaningful regions. Hard attention, soft attention, and cascaded attention strategies have been successfully applied across diverse imaging tasks, including brain tumor segmentation, lung imaging, and COVID-19 detection from chest X-rays [ 11 , 14 , 13 ] . Semi-supervised and multi-task learning paradigms have also demonstrated promise in improving generalization under limited annotation settings [ 12 , 15 ] . In parallel, the adoption of transformer-based architectures has introduced new opportunities for modeling long-range spatial dependencies that are difficult to capture using purely convolutional designs. Hybrid CNN-transformer models, such as TransUNet and Swin-based architectures, have shown strong performance in medical image segmentation by combining local feature extraction with global self-attention. Related efforts have also explored lightweight and deployment-friendly models suitable for IoT and edge environments, highlighting the importance of efficiency alongside accuracy [ 3 , 20 , 1 ] . Transformer-based representation learning has additionally been applied to anatomy-guided segmentation and multimodal fusion tasks in ultrasound and neuroimaging [ 2 , 16 ] . Despite these advances, ischemic stroke lesion segmentation presents unique challenges. Lesions often exhibit high variability in size, shape, and spatial distribution, and diffusion MRI is susceptible to motion artifacts and intensity heterogeneity. Prior work has explored motion correction and image reconstruction in multishot MRI using generative and adversarial learning frameworks [ 17 , 4 ] , yet systematic studies focusing on architectural design choices for multimodal diffusion MRI segmentation remain relatively limited. More broadly, artificial intelligence has demonstrated transformative potential across biomedical signal processing, imaging, and healthcare systems. Applications span phonocardiographic sensing [ 8 ] , affective computing [ 18 , 6 ] , COVID-19 analytics [ 7 , 14 , 12 ] , explainable computer-aided diagnosis [ 9 ] , and image source–agnostic pathology pipelines [ 26 ] . Multimodal representation learning has also been increasingly explored in neuroimaging for tasks such as brain age estimation and biological aging analysis [ 24 , 10 ] . These works collectively emphasize the importance of robust, interpretable, and clinically aligned AI systems. Motivated by these developments, this study focuses on automated ischemic stroke lesion segmentation using multimodal diffusion MRI from the ISLES 2022 dataset. We conduct a systematic benchmarking of state-of-the-art CNN- and transformer-based segmentation architectures and investigate the impact of architectural design on segmentation performance. Building upon prior successes of multi-encoder and attention-based models, we introduce a dual-encoder TransUNet framework that explicitly models modality-specific representations from DWI and ADC scans. Furthermore, we incorporate limited three-dimensional contextual information through adjacent slice aggregation to improve lesion boundary delineation. The main contributions of this work are summarized as follows: 1. A comprehensive benchmarking of state-of-the-art segmentation models for ischemic stroke lesion segmentation on the ISLES 2022 dataset. 2. A dual-encoder transformer-based architecture for effective multimodal fusion of DWI and ADC images. 3. An empirical analysis of the role of multi-slice contextual information in improving segmentation accuracy. Through extensive experimental evaluation, this work provides insights into effective multimodal representation learning strategies for diffusion MRI and establishes a strong foundation for future research in ischemic stroke imaging and automated clinical decision support. 2 Methodology 2.1 Problem Formulation The objective of this work is to perform automated segmentation of ischemic stroke lesions from diffusion magnetic resonance imaging (MRI). Given paired Diffusion-Weighted Imaging (DWI) and Apparent Diffusion Coefficient (ADC) scans, the task is formulated as a binary semantic segmentation problem, where each pixel is classified as either lesion or background. By leveraging complementary diffusion information from DWI and ADC modalities, the goal is to improve robustness and accuracy in delineating infarct regions across acute and sub-acute stroke cases. 2.2 Dataset Description This study utilizes the ISLES 2022 dataset, introduced as part of the MICCAI 2022 Ischemic Stroke Lesion Segmentation Challenge. The dataset