Beyond VI-RADS Uncertainty: Leveraging Spatiotemporal DCE-MRI to Predict Bladder Cancer Muscle Invasion Authors: Minghui Song, Haonan Ren, Lijuan Wang, Yihang Zhou, Xing Tang, Huanjun Wang, Yan Guo, Yang Liu, Hongbing Lu, Xiaopan Xu Published: 2025 ABSTRACT Background: The Vesical Imaging-Reporting and Data System (VI-RADS) has limited diagnostic accuracy in distinguishing non-muscle-invasive bladder cancer (NMIBC) within VI-RADS categories 2 and 3, despite its value for overall NMIBC assessment. Dynamic contrast-enhanced MRI (DCE-MRI), which reflects tumor vascularity, holds promise for improving these challenging cases but remains underutilized due to unexploited spatiotemporal information. Methods: We developed a deep learning model to comprehensively quantify spatiotemporal features from multiphase DCE-MRI in 184 patients with VI-RADS 2 or 3 (training: n = 115, validation: n = 20, testing: n = 49). The model integrated multiscale feature extraction and contextual attention mechanisms to enhance diagnostic performance. Results: The model outperformed established benchmarks (e.g., VGG, ResNet) and the conventional VI-RADS ≤ 2 threshold (sensitivity: 0.67 for NMIBC), achieving a sensitivity of 0.90 (95% CI: 0.81–0.96) for NMIBC and an area under the curve (AUC) of 0.82 (95% CI: 0.75–0.89) for overall classification. Visualizations confirmed its ability to identify key spatiotemporal patterns linked to muscle invasion. Conclusions: By leveraging comprehensive spatiotemporal information from DCE-MRI, our deep learning model significantly improves NMIBC diagnosis in VI-RADS 2/3 cases, offering a clinically valuable tool to address the limitations of current VI-RADS assessment. 1. Introduction Urinary bladder cancer (BCa) is the fourth most common malignancy in male patients globally [1,2]. Among initial diagnoses, approximately 75% are classified as non-muscle-invasive bladder cancers (NMIBCs) [3,4], while the remaining 25% are muscle-invasive bladder cancers (MIBCs) [5]. Treatment strategies for NMIBCs focus on preventing progression and ensuring close surveillance for muscularis invasion [3,4]. In contrast, MIBC often necessitates radical cystectomy to mitigate lymph node involvement and prevent distant metastasis. Accurate pretreatment diagnosis of NMIBC is crucial for developing personalized treatment and management of patients. The reference standard for diagnosing NMIBC is transurethral resection of bladder tumor (TURBT) followed by pathological examinations [3,5]. Due to the alteration of tissue architecture resulting from TURBT or the absence of detrusor muscle in the resected specimens, approximately 39% of BCa cases are at risk of being under-staged [4]. Repeated TURBTs may help reduce the diagnostic error, but are invasive, uncomfortable, time-consuming, and costly [4,6,7]. Recent advancements in MRI have established multiparametric MRI (mp-MRI) as a valuable complement for the preoperative diagnosis of detrusor muscle infiltration of BCa [8,9]. Mp-MRI consists of structural sequences, such as T2-weighted imaging (T2WI), and functional sequences, including diffusion-weighted imaging (DWI) and multiphase dynamic contrast-enhanced MRI (DCE-MRI) [8]. These MRI sequences jointly provide excellent contrast resolution for BCa staging, early recurrence detection, and treatment response assessment [8,9]. To standardize mp-MRI scanning and reporting for BCa, the Vesical Imaging-Reporting and Data System (VI-RADS) was introduced in 2018 [8,9,10,11], which employs a five-point scoring system to assess the likelihood of detrusor muscle infiltration [8,10]. According to the VI-RADS scoring system, a score of 1 indicates a high probability of non-muscle invasive bladder cancer (NMIBC), while scores of 4 or 5 suggest strong evidence of muscularis invasion [10,12,13]. Recent studies have shown that between 4.1% and 17.4% of patients with a score of 2 were later diagnosed with MIBC after surgery [6]. Additionally, a score of 3 presents significant challenges in precisely distinguishing between NMIBC and MIBC [7]. This issue has resulted in some patients with VI-RADS scores of 2 or 3 being incorrectly staged, especially when the common threshold of VI-RADS ≤ 2 is used to predict NMIBC [13]. To deal with these issues, Cai et al. developed an artificial intelligent (AI) model to predict NMIBC in VI-RADS score of 2 with tumor stalk [6], and Yu et al. proposed another AI model to predict the muscularis invasion among patients with VI-RADS score of 3 [7]. To the best of our knowledge, few AI models have been specifically developed for the accurate prediction of muscularis invasion in patients with VI-RADS 2 or 3. When investigating all the sequences in the mp-MRI and VI-RADS scoring system, DCE-MRI employs gadolinium-based contrast agents with multiphase MRI scanning to reflect the dynamics of vascular permeability within the tumor and its surrounding bladder wall, which potentially provides important dynamic spatiotemporal information for NMIBC prediction [14,15]. However, most existing studies have concentrated primarily on the peak enhancement phase post-contrast administration [2,16,17,18,19], often overlooking the rich spatiotemporal information that can be gleaned from the dynamic perfusion process during multi-temporal-phase DCE-MRI scans [20]. Currently, deep learning (DL) models such as Visual Geometry Group-16 (VGG-16), ResNet-101, DenseNet-121, and Vision Transformer (ViT) have gained great attention in the field of image-based cancer diagnostics [9,21,22,23,24,25]. However, these models often face challenges in effectively capturing the dynamic spatiotemporal information in the multi-temporal-phase DCE-MRI. A recent study by Bedel et al. introduced a BOLD transformer model that uses a series of transformer encoders combined with a new fused window attention mechanism to capture spatiotemporal information from time-series functional MRI [25], which provides valuable insights for analyzing multi-temporal-phase DCE-MRI in diagnosing NMIBC. Considering the significant variation in BCa masses among patients, a multiscale convolutional mechanism may enhance the capability of the AI model to capture both coarse and fine-grained image information [26,27], potentially improving the performance for NMIBC prediction [26,27]. Building upon these insights, the present study aimed to develop and validate a Transformer-fused multiscale 3D convolutional network (TM3DConvNet) that fully captures the multiscale spatiotemporal information from multi-temporal-phase DCE-MRI to improve NMIBC prediction among patients with VI-RADS scores of 2 or 3. 2. Materials and Methods 2.1. Patient Population Datasets were collected by a senior radiologist (H.W., 15 years of experience) from the picture archiving and communication system at The First Affiliated Hospital of Sun Yat-Sen University. Institutional Review Board (IRB) approval was obtained (Approval No. [2020]492). The ethical approval date was 8 December 2020. These data included consecutive BCa patients who underwent preoperative mp-MRI scans between March 2013 and May 2018 and had surgical procedures within two weeks of their initial imaging exams. Tumors were analyzed pathologically, and for patients with multiple lesions, the tumor site with the highest burden (largest volume or most advanced stage) was selected for analysis [8,9]. The inclusion criteria were as follows: (a) patients suspected of primary BCa with axial mp-MRI scans and VI-RADS scores of 2 or 3 before treatment, (b) available pathological results within two weeks, and (c) at least five phases of axial DCE-MRI scans. Patients were excluded if they met any of the following conditions: (a) absence of VI-RADS scores of 2 or 3, surgical treatment records, or pathological findings, (b) merely sagittal or coronal MRI scans, (c) fewer than five phases axial DCE-MRI scans, and (d) severe artifacts or poor image quality. After applying these criteria, eligible patients were randomly assigned to the training, validation, and testing cohorts roughly at a 6:1:3 ratio, as shown in Figure S1. Based on the criteria, 184 patients were eligible. Among them, 134 and 108 patients were previously reported in two separate studies [8,9]. The current study investigated whether multiscale spatiotemporal information from multi-temporal-phase DCE-MRI could effectively predict NMIBC, distinct from the previous studies focusing on recurrence risk assessment [8,9]. 2.2. MRI Examination Patients were positioned supine in the MRI unit after fasting for four to six hours and instructed to drink 500–1000 mL of water 30 min prior to ensure adequate bladder distension. MRI scans were conducted using a 3 Tesla system (Magnetom Trio, Siemens (Munich, Germany); Signa Pioneer, GE Healthcare (Chicago, IL, USA)) with an 8- or 32-channel body coil, covering the pelvic region from the aortic bifurcation to the lower edge of the pubic symphysis. The sequences included axial T2WI without fat suppression, high-spatial-resolution fat-suppressed T2WI, and axial DWI (b values of 0 and 1000 s/mm2 for Magnetom Trio, or 50 and 1000 s/mm2 for Signa Pioneer) [8,10]. Multi-temporal-phase DCE-MRI was conducted using a fat-suppressed volumetric spoiled gradient-echo sequence, both before and after intravenous injection of 0.2 mL/kg of body weight of gadopentetate dimeglumine or gadoteric acid at 2 mL/s [28,29]. Most patients underwent five phases of scanning within 20 to 131 s after contrast administration. The main scanning parameters are detailed in Table S1. All patients underwent a standardized 5-time-point DCE-MRI protocol (acquired at 0 s, 30 s, 60 s, 90 s, and 5 min post-contrast). However, due to patient individual differences (such as injection delay, preparation time, or equipment adjustments), the actual arrival time of the contrast agent at the target organ and the acquisition timing of corresponding phases varied significantly among individuals, resulting in differences in the selected acquisition time points. Minor variations in the actual number of acquired time-points occurred due to equipment constraints (60 s time-point omitted in 5% of cases) or patient-specific factors (e.g., reduced time-points in one case due to poor tolerance). The model’s input layer dynamically accommodated varying time-point numbers via multiscale convolution (e.g., adaptive channel adjustment using 1 × 1 convolutions), ensuring robust analysis. Definitions and acquisition timings for multi-temporal-phase DCE-MRI are detailed in Table S2. 2.3. Management of MRI Data Heterogeneity Arising from Coil Characteristics The coil type (8-channel for smaller patients, 32-channel for larger patients/higher SNR needs) was chosen based on body habitus and scan region, and not randomly. All coils were body coils optimized for pelvic MRI (aortic bifurcation to pubic symphysis), sharing similar design and key features (coverage, field uniformity). To address the smooth, low-frequency intensity nonuniformity caused by imperfections in coil sensitivity and B1 field inhomogeneity, N4 bias field correction algorithm was implemented with its default parameter settings. In addition, to ensure the inter-subject and inter-scanner consistency, global intensity normalization was applied. Low-quality images (artifacts, signal loss, abnormal contrast) from coil faults, patient motion, or parameter errors were excluded during preprocessing. The final 184-patient dataset met strict criteria (“no obvious coil artifacts, high clarity, good tumor contrast”), further minimizing coil-related impacts. 2.4. Image Preassessment The mp-MRI data for each patient were scored using a five-level scoring system at a two-week interval. Two senior radiologists (H.W and Y.G., with 16 and 36 years of experience, respectively) independently scored the images without prior knowledge of the patients’ surgical or histopathological findings. After scoring all MRI sequences, a VI-RADS score was assigned to each patient in accordance with established VI-RADS principles [10], and any discrepancies between the radiologists were resolved through consensus. Afterward, the senior radiologist (X.T., 15 years of experience) assessed the multi-temporal-phase DCE-MRI scans for each patient, identifying the tumor location in the five phases based on postoperative pathology results. For patients with more than five phases of data, only the pre-contrast and four following contrast phases were considered for further analysis. 2.5. Development of the Proposed TM3DConvNet Model The overall workflow of the proposed TM3DConvNet model is shown in Figure 1. Following image preassessment, the image slice with the largest tumor burden at that location of each phase was then selected, constituting a 3D multi-temporal-phase DCE-MRI data X (X∈RH×W×5) for further analysis. 2.5.1. Normalization, ROI Cropping Block and Data Augmentation To enhance the comparability of tumor signal intensity across multiple phases in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), this study employed a Z-score normalization method based on region-of-interest (ROI) statistics to standardize the image data. First, a radiologist with 13 years of diagnostic experience (X.T.) manually delineated the bladder region containing the tumor on all temporal phase images to precisely define the region of interest (ROI). Subsequently, the raw signal intensity values of all voxels within the tumor region across all phases were extracted, and their overall mean (μ) and standard deviation (σ) were calculated. Based on these global statistical measures, Z-score normalization was performed on the signal intensity Sraw of each voxel within the tumor region for each phase as follows:Snorm = Sraw−μσ Sraw: The unnormalized signal intensity (raw intensity value) in the original image. μ: The mean signal intensity of all voxels within the tumor region across all temporal phases (global mean of training dataset). σ: The standard deviation of signal intensity of all voxels within the tumor region across all temporal phases (global standard deviation of training dataset). Snorm: The normalized (standardized) signal intensity value after Z-score transformation. Data augmentation was then performed using a conditional generative adversarial network to expand the minority group (MIBC samples), improving class balance for model construction [30] (refer to the Supplementary Materials). In addition, to simulate real-world noise variations, MRI-specific Rician noise and Gaussian noise were incorporated into the images with a defined probability. After that, common data augmentation techniques such as rotations, mosaic augmentation, scaling, and translations were further utilized with both groups of samples in the training cohort to improve the data diversity for model building. 2.5.2. 3D Multi-Head Mixed Convolution (MHMC) Module Before performing multi-head convolution, the cropped 5-phase image slices were resized to match the dimensions of X′(X′∈R224×224×5), as shown in Figure 2. An image size of 224 × 224 is widely adopted in many DL models. It ensures compatibility with these models while preserving sufficient resolution to capture detailed image features [27]. Subsequently, a lightweight 3D multi-head mixed convolution (MHMC) module was used to capture the multiscale spatiotemporal features of the 5-phase image slices. The MHMC module employed three parallel 3D convolution pathways with kernel sizes of 1 × 1 × 1, 3 × 3 × 3, and 5 × 5 × 5, which were designed to extract features at multiscale receptive fields, namely local, medium, and global spatial contexts, respectively. The selection of multiscale convolutional kernels (1 × 1 × 1, 3 × 3 × 3, and 5 × 5 × 5) in our study was guided by the dual requirements of multiscale feature fusion and task adaptability: small kernels (1 × 1 × 1, 3 × 3 × 3) were specifically designed to capture localized, fine-grained tumor features, with their smaller receptive fields enabling precise delineation of critical microstructural details—including tumor margins, microvasculature, and textural heterogeneities—while minimizing information loss from over-smoothing; large kernels (5 × 5 × 5) were optimized to model global spatial characteristics of the tumor, as their extended receptive fields facilitate the integration of inter-regional feature correlations (e.g., contrasting intensely enhanced central zones with peripherally hypo-enhanced areas), effectively compensating for broader contextual information that smaller kernels might overlook. Given that the input consists of 5-phase 3D DCE-MRI sequences (each phase represented as a 3D volumetric dataset) requiring concurrent modeling of both spatial features (tumor morphology) and temporal features (dynamic enhancement kinetics), the multiscale kernel architecture (1 × 1 × 1, 3 × 3 × 3, 5 × 5 × 5) systematically addresses this multidimensional complexity by spanning a continuum of receptive fields—from single-voxel level detail (e.g., isolated enhancement disparities) to whole-tumor patterns (e.g., the 5-phase integrated enhancement signature)—thereby optimally accommodating the structural intricacies of 3D medical imaging data. Furthermore, although larger kernels (e.g., 7 × 7 × 7) could provide even more extensive receptive fields, we prioritized the combination of 1 × 1 × 1, 3 × 3 × 3, and 5 × 5 × 5 to achieve an optimized trade-off: this configuration preserves feature diversity across spatial scales, enhances model generalization in small-sample scenarios by mitigating overfitting risks, and maintains computational efficiency, which are key considerations for clinical translation. Each pathway consisted of a dual-layer convolutional structure: the first layer applied a standard 3D convolution with kernel wa, followed by batch normalization, rectified linear unit (ReLU) activation, and max pooling; the second layer utilized depth-wise separable convolutions (comprising a depth-wise convolution followed by a 1 × 1 × 1 pointwise convolution) to further refine features while significantly reducing the computational cost. This two-stage design enabled efficient yet expressive extraction of multiscale information from the input 5-phase DCE-MRI slices X∈RH×W×5. Formally, the feature extraction process in each branch can be expressed as:(1)Xi″= reshapewbwaX, i=1,3,5 where X is the input 5-phase image slice with dimensions H × W × 5, wa and wb are learnable convolutional kernels in the dual-layer structure, and reshape (⋅) is an operation that transforms the feature dimensions from 3D to 2D for further processing. After feature extraction across the three parallel branches, the resulting multi-scale feature maps were concatenated along the temporal (or phase) dimension to form a unified token representation. Specifically, the outputs X1″,X3″ and X5″ from the three branches were combined as follows:(2)X″=Concat(X1″,X3″,X5″) The final output token X″ has the shape 1 × 448 × 784 (e.g., for one dataset instance: 1 sample, 448 time points or spatial elements, and 784 features per time point), effectively encapsulating multiscale spatial and contextual information from all five DCE-MRI phases. This compact yet informative representation serves as the input to the subsequent fused window attention module for higher-level spatiotemporal modeling. 2.5.3. Fusion Window Attention Module (FWAM) The original self-attention mechanism effectively calculates global correlations in the input token, proving successful in natural language processing and computer vision. However, it struggles with larger input dimensions and increased computational complexity, often overlooking the fine-grained correlations within parts of the input token [25,27]. To address these issues, we introduced the FWAM (Figure 3). The FWAM adopted Transformer blocks to split each input token X″ into windows of W time points, with a stride of S time points between adjacent windows [25]. Each window includes a fringe of L time points on both sides, forming a base token of dimensions 1+(L+W+L)×784, as illustrated in Figure 3a. The corresponding formula is:(3)F=(T−W)/S+1 where T is the total 448 time points in this framework; W represents the size of the time points in each window, and the stride between adjacent windows is S. From this formula, the total number of token windows F to be fused can be determined. An auxiliary token, initialized as a shared, learnable vector, captured high-level features for NMIBC prediction by summarizing the 448 time points. As the input token X″ passed through the Transformer blocks, the auxiliary token and base token of each window were used for multi-head self-attention calculations, as shown in Figure 3b. Specifically, the base token xi∈RW×N (representing the central Wtime points) and the auxiliary token(s) ai,bi∈RL×N (representing the left and right fringe time points) were concatenated with a context vector Ci∈R1×N to form the query (Qi), key (Ki), and value (Vi) vectors as follows:(4)Qi =Wq{ci,xi} ∈R(1+W)×N(5)Ki=Vi=Wk{ci,ai,xi,bi} ∈R(1+W+2L)×N where Wq and Wk are the learnable projection parameters, The self-attention scores were then computed using the scaled dot-product attention mechanism:(6)attn(Qi,Ki,Vi)=σ(QiKiTdi)Vi where di is the dimension of input features, and used to stabilize the gradients during training, σ is softmax(⋅) function. Each window had its own version of the auxiliary token to capture fine-grained attention. The output auxiliary tokens and output tokens were then fed into the classification module for NMIBC prediction. After obtaining the final classification variable ci, we aggregate the learned time features from all windows, weighted by their respective importance,(7)Fo=1F∑i=1Fci where Fo is the representative features after feature fusion. Subsequently, layer normalization was applied to stabilize the hidden states at each time point and mitigate the gradient explosion problem. 2.5.4. Classification Module After the FWAM, all output tokens were converted into an embedding vector which contained the aggregative spatiotemporal information. Then a fully connection layer was designed to predict the probability of label embedding yp for each subject [25]. Then, the cross-entropy loss function calculating the classification loss between ground truth label y and the prediction yp for each subject [25], was used to minimize the classification error. The probability yp was computed using a fully connected layer as:(8)yp=σ(WoFo+bo) where Wo and bo are the weight matrix and bias matrix, respectively, and σ(·) is the sigmoid function. The cross-entropy loss function was defined as:(9)Lloss= −1N∑i=1N[y·logyp+(1−y)·log(1−yp)] 2.6. Performance Evaluation and Comparison The TM3DConvNet model was developed using the training cohort, with NMIBC cases labeled as negative. The model was then validated in the testing cohort and compared to common DL models like VGG-16, ResNet-101, DenseNet-121, and ViT for NMIBC diagnosis. Additionally, the performance using multi-temporal-phase DCE-MRI data was compared to the single-phase DCE-MRI for NMIBC diagnosis. All models were run on a lab server with Windows 10, an NVIDIA GeForce RTX 3090 GPU, 10,496 CUDA cores, and 24 GB of VRAM. 2.7. Statistical Analysis Statistical analyses were conducted using SPSS software (version 20; IBM Corp., Armonk, NY, USA) and R software (version 4.1.1; R Foundation for Statistical Computing, Vienna, Austria). The interobserver agreement of the VI-RADS score was evaluated with the weighted k value. The performance of the proposed model and other common DL models was evaluated using sensitivity, specificity, accuracy, and area under the curve (AUC). DeLong’s tests with Bonferroni correction were used to compare the receiver operating characteristic (ROC) curves of these models for NMIBC diagnosis [31]. A two-tailed p-value < 0.05 was considered statistically significant. 3. Results 3.1. Demographics of Eligible Patients An interobserver agreement analysis was conducted to assess the VI-RADS scoring consistency between two radiologists, yielding a weighted kappa value of 0.86 (95% CI: 0.78, 0.89). After that, 184 patients (median age 67 years, range 55–72; 15 women) were eligible. Among them: (i) 136 (73.9%) had NMIBC and 48 (26.1%) had MIBC; (ii) 141 (76.6%) had solitary tumors and 43 (23.4%) had two to four synchronous tumors; (iii) 121 (65.8%) underwent TURBT operation. Detail information is shown in Table 1. 3.2. Performance Comparison of the Proposed TM3DConvNet with Other Models The prediction performance of the proposed TM3DConvNet model was comprehensively evaluated in both the training and testing cohorts, and systematically compared with widely used deep learning models, including VGG-16, ResNet-101, DenseNet-121 and ViT. In addition to deep learning models, we performed comparative analysis with the conventional imaging assessment method—the Versatile Integrated Radiological Assessment for Bladder Cancer Staging (VI-RADS). Two board-certified radiologists (H.W. and Y.G.) with extensive experience in urogenital imaging independently evaluated the multiphase DCE-MRI examinations according to standardized VI-RADS criteria (1–5 scale), where scores 1–2 correspond to non-muscle-invasive bladder cancer (NMIBC) and scores 3–5 indicate muscle-invasive bladder cancer (MIBC). To ensure diagnostic consistency, any scoring discrepancies were resolved through consensus discussion between the evaluating radiologists. The diagnostic performance of VI-RADS in differentiating NMIBC from MIBC was determined based on the final consensus scores and corresponding pathological findings, demonstrating a sensitivity of 70% (95% CI: 63–77), specificity of 80% (95% CI: 73–86), and overall accuracy of 0.73 (95% CI: 0.69–0.83). Figure S2 shows the training loss curves of these models, in which TM3DConvNet exhibited the fastest convergence speed and the lowest loss value. Figure 4 and Table 2 present the performance of TM3DConvNet, established benchmark networks (VGG, ResNet, DenseNet, Vision Transformer), and the conventional VI-RADS (≤2-score threshold) for NMIBC prediction across the training, validation, and testing cohorts. In the testing cohort, TM3DConvNet demonstrated superior performance, achieving a sensitivity of 0.90 for NMIBC and an area under the curve (AUC) of 0.82 for overall classification. Table 3 presents DeLong’s tests with Bonferroni correction to analyze the differences in the receiver operating characteristic (ROC) curves between TM3DConvNet and the established benchmark networks in the testing cohort. Table 4 shows the ablation experiments of the important modules designed in the proposed TM3DConvNet model for predicting bladder cancer detrusor muscularis invasion. Additionally, the performance of the benchmark networks with single-phase DCE-MRI was also evaluated in the testing cohort, as shown in Table S2, which was inferior to that of the proposed TM3DConvNet using the multi-temporal-phase DCE-MRI. 3.3. Spatiotemporal Attention Visualization and Model Interpretation To visualize the spatiotemporal attention and attention transition across DCE-MRI phases and interpret the model mechanism, gradient-weighted class activation mapping was applied in feature extraction and classification. Figure 5 shows the spatiotemporal attention visualized for (a) NMIBC and (b) MIBC prediction using multi-temporal-phase DCE-MRI. Figure 6 displays the dynamic distribution characteristics of the attention mechanism in a deep learning network during the inference process of axial Dynamic Contrast-Enhanced Magnetic Resonance Imaging (axial DCE-MRI) across multiple phases (Phase I–V). Each group of subfigures contains two main components: the original DCE-MRI images (left grayscale images with white circles marking the lesion areas) and the corresponding network attention heatmaps (right color images where red and yellow highlight regions of concentrated attention). Additionally, the “Ground truth” (true label) and “Predict” (predicted label) for each phase were both NMIBC (Non-Muscle Invasive Bladder Cancer). From the visualized results, it can be seen that in different DCE-MRI phases, the network’s attention precisely focused on the areas adjacent to the lesions. At the same time, there were significant differences in the spatial distribution and activation intensity of the attention heatmaps across various phases. This indicates that the network’s attention mechanism is not fixed but dynamically adjusts according to the imaging features of different DCE-MRI phases. Technical Limitation and Future Directions: Notably, the current 3 mm slice thickness in our DCE-MRI acquisitions (as specified in Figure 5’s caption) presented a fundamental limitation for pixel-level localization of microstructures—particularly the tumor–myometrium boundary—despite our implementation of hybrid 1 × 1 × 1 and 3 × 3 × 3 convolutional kernels. This physical imaging constraint underscores the need for future investigations to integrate: (1) higher-resolution imaging protocols (e.g., 1.5 mm thin-section scanning), and (2) anatomy-guided attention mechanisms to enhance the spatial correspondence between predictive heatmaps and clinically critical features. In addition, the TM3DConvNet model was further tested on sagittal multi-temporal-phase DCE-MRI data from 5 initially excluded cases, achieving 60% accuracy. Figure S3 presents the spatiotemporal attention visualization for predicting some of these BCa cases with sagittal multi-temporal-phase DCE-MRI data. Figure S4 illustrates the spatiotemporal attention transition across sagittal DCE-MRI phases for one of these cases, specifically for non-muscle-invasive BCa characterization. 4. Discussion DCE-MRI is a valuable imaging modality that tracks the movement of a gadolinium-based contrast agent within tissues and blood vessels over time [12]. This technique enhances signal sensitivity to effectively highlight blood flow, vascular permeability, and the distinct perfusion patterns associated with tumors, thereby providing crucial insights into the pathophysiological and morphological characteristics of tumor masses [12]. For preoperative diagnosis of muscularis invasion in patients with VI-RADS scores of 2 or 3, multi-temporal-phase DCE-MRI is invaluable. This technique, particularly in the early and late phases after contrast administration, emphasizes key features of the bladder’s inner layers, such as the urothelium and lamina propria, which are essential in predicting NMIBC [9]. Although DCE-MRI has unique advantages in visualizing tumor-associated vascular dynamics and the layered structure of the bladder wall, we acknowledge that its inherent low spatial resolution may limit the conclusiveness of image interpretation, particularly when evaluating fine anatomical structures or subtle enhancement patterns. We believe that addressing this issue requires collaborative innovation across imaging, analysis, and clinical practice. This integrated strategy—including techniques such as super-resolution reconstruction, better explainable AI (XAI) methods and clinical validation loop—is not only essential for advancing imaging technology, but also a cornerstone for achieving precise diagnosis and precision therapy. In this study, we introduced a 3D DL model termed TM3DConvNet, specifically designed to extract the spatiotemporal information inherent in multi-temporal-phase DCE-MRI data for the noninvasive prediction of muscle invasion of BCa patients with VI-RADS scores of 2 or 3. To the best of our knowledge, this is the first investigation of employing multi-temporal-phase DCE-MRI in conjunction with a 3D DL strategy for the diagnosis of NMIBC. Although the number of slices per patient was relatively small, the key slices from the five phases complemented each other in the temporal dimension, providing the model with a complete information chain of tumor dynamic enhancement features. A key component of the proposed model is the MHMC. This module diverges from traditional models that depend on single-scale convolutional kernels arranged across multiple layers. Instead, the MHMC efficiently captures multiscale features from various receptive fields in three-dimensional data using just two convolutional layers. This optimization not only streamlines the model architecture but also minimizes the number of parameters, helping to alleviate computational demands. Our core future work will be to conduct multi-center, large-sample external validation to further consolidate the generalizability of our study conclusions. To further enhance the extraction and fusion of the spatiotemporal attention within these features, we integrated FWAM. Unlike conventional ViT models which focus primarily on global self-attention among input tokens, the FWAM could well fuse both global (coarse) and local (fine-grained) attention, thus enabling a comprehensive synthesis of spatiotemporal information. The ablation studies substantiate that TM3DConvNet, through its dual integration of the MHMC and FWAM modules, significantly outperforms configurations employing either module in isolation. When assessed against established benchmark networks such as VGG-16, ResNet-101, DenseNet-121 and ViT, TM3DConvNet exhibited superior predictive capabilities for BCa detrusor muscle invasion prediction. This underscores the crucial requirement to capture both multiscale spatial characteristics and temporal dynamics inherent in multi-temporal-phase DCE-MRI data during BCa phenotyping. Furthermore, our findings indicate that the 3D model utilizing comprehensive five-phase DCE-MRI data significantly outperformed their two-dimensional counterparts using single-phase data, solidifying the value of multi-temporal-phase DCE-MRI in BCa diagnosis. Specifically, when compared to the conventional VI-RADS ≤ 2 threshold. which achieved a sensitivity of 0.67 for NMIBC diagnosis, our proposed TM3DConvNet model demonstrated great performance improvement, with the sensitivity largely improved to 0.90 for NMIBC identification and an AUC of 0.82 for overall classification on the testing cohort. Using gradient-weighted class activation mapping (Grad CAM), we observed that TM3DConvNet effectively targets the tumor and surrounding bladder wall to extract and classify multiscale features. Notably, our results also reveal important shifts in spatiotemporal attention across the various phases of DCE-MRI, emphasizing the essential role of multiphase imaging in predicting NMIBC. Nonetheless, it is important to recognize the limitations of this study. The findings are constrained by the reliance on a single-center database, which might impact the generalizability of our results. To enhance clinical applicability, future work should focus on larger, multicenter datasets incorporating diverse MRI platforms. Moreover, the manual cropping of the bladder region using rectangular boxes to exclude complex background has proven to be time-consuming. Future research could further explore fully automated AI segmentation tools (e.g., deep learning models based on U-Net) to completely replace manual operations, thereby further improving efficiency and objectivity. Developing a multitask pipeline that integrates automatic bladder segmentation alongside NMIBC prediction could significantly improve clinical workflow. Finally, considering the potential of mp-MRI in NMIBC diagnostics, future research should explore a static-dynamic integration deep learning model capable of extracting static features from non-contrast sequences, such as T2-weighted MRI, diffusion-weighted MRI, and apparent diffusion coefficient maps, in conjunction with dynamic features from multi-temporal-phase DCE-MRI. This approach could not only advance bladder cancer diagnosis but also benefit other oncological predictions. 5. Conclusions In summary, the proposed TM3DConvNet model demonstrates a robust capacity to extract multiscale spatiotemporal information from multi-temporal-phase DCE-MRI, thereby improving the assessment of detrusor muscularis invasion status among diagnostically challenging BCa cases with VI-RADS 2 or 3.