Ultrasound and SWE-based transfer learning for predicting fibrotic NASH Authors: Fei Xia, Kun Wang, Yuhe Wang, Chaoxue Zhang, Junli Wang Published: 2025 ABSTRACT AbstractThe aim of this study was to develop a combined deep-learning model utilizing liver ultrasound, liver elastography images, and clinical features to predict and diagnose fibrotic non-alcoholic steatohepatitis (NASH). A rat model of liver steatosis and fibrosis was established through a high-fat diet and subcutaneous CCl₄ injections. Two-dimensional ultrasound and shear wave elastography (SWE) images were acquired. Three deep learning models, based on the ResNet-18 architecture, were designed: (1) a pure image model using only liver ultrasound, (2) a pure image model using only liver elastography, and (3) a combined model incorporating liver ultrasound, liver elastography images, and clinical features. The performance of these models was evaluated using three-fold cross-validation, receiver operating characteristic (ROC) curves, decision curve analysis (DCA), and calibration curves. The combined deep learning model demonstrated the highest area under the curve (AUC) of 0.879. DCA revealed that the multimodal model provided superior net benefits across most threshold probability ranges for predicting and diagnosing fibrotic NASH. The combined deep learning model based on the ResNet-18 architecture exhibits promising performance in predicting and diagnosing fibrotic NASH.Supplementary InformationThe online version contains supplementary material available at 10.1038/s41598-025-28753-5.Keywords: Liver fibrosis, Non-alcoholic steatohepatitis (NASH), Deep learning, Real-time shear wave elastographySubject terms: Diseases, Medical research Introduction Non-alcoholic fatty liver disease (NAFLD), driven by the global prevalence of obesity and type 2 diabetes mellitus (T2DM), has become the most common cause of chronic liver disease worldwide1. The primary characteristic of NAFLD is fat accumulation in the liver parenchyma, which can progress to a more severe inflammatory stage characterized by hepatocellular injury with or without fibrosis, known as non-alcoholic steatohepatitis (NASH)2. NASH is a risk factor for liver fibrosis, which indicates a poor prognosis. However, not all NASH patients progress to fibrosis3. Previous studies have demonstrated that patients with fibrosis-related NASH are at significantly increased risk of liver-related complications and progression to cirrhosis4,5. Additionally, liver fibrosis has been identified as the strongest predictor of disease-specific progression and mortality6,7. Consistent with previous research, we classify NASH patients with a fibrosis stage ≥ 2 as fibrotic NASH8,9. These patients face the highest risk of disease progression, benefit the most from pharmacological treatment, and are the population most in need of therapeutic intervention. The timely diagnosis of fibrotic NASH patients is of great significance for the clinical management of liver disease. Currently, non-invasive clinical methods for assessing and evaluating NAFLD, such as computed tomography (CT), magnetic resonance imaging (MRI), and ultrasound, are ineffective in accurately detecting fibrotic NASH patients10. In recent years, artificial intelligence (AI) has emerged as a powerful tool in medical diagnosis, particularly in medical image analysis. Deep learning, a subset of AI based on deep neural networks (DNNs) with multiple hidden layers, is at the forefront of this field11. Neural networks, inspired by the structure and function of biological neurons, possess non-linear, adaptive, and parallel processing capabilities, making them well-suited for multi-parameter diagnostic models. These models have demonstrated promising results in predicting, classifying, and assessing liver fibrosis12,13. Deep learning is a rapidly evolving technology with tremendous potential in various areas of medical image analysis, including disease classification, segmentation, detection, and image registration. Existing studies have shown that multimodal deep learning models can enhance model performance. For example, Sato et al. integrated ultrasound images with clinical information such as patient age and blood biochemical indicators, resulting in an improvement in diagnostic accuracy from 68.52% to 96.30% (AUC = 0.994)14. In addition, deep learning models can expand their application scope from diagnosis to treatment decision-making. For instance, Chi Zhang et al. developed an acute pancreatitis model that innovatively combines lesion segmentation with severity assessment to assist in formulating personalized treatment plans15. Moreover, deep learning models can be combined with Grad-CAM to generate heatmaps, visually illustrating diagnostic reasoning. For example, the model developed by Hong et al. used Grad-CAM to highlight key regions associated with visual impairment diseases16. These advancements further validate the effectiveness and cutting-edge nature of deep learning-based diagnostic models. Currently, deep learning models employed for ultrasound diagnosis primarily rely on two-dimensional cross-sectional images for predictive tasks. However, the information derived from two-dimensional cross-sections is inherently limited and may not fully capture the complexity of lesions. Real-Time Shear Wave Elastography (SWE) is an ultrasound-based technique that quantifies tissue stiffness by measuring tissue elasticity, thereby enabling the assessment of tissue characteristics17. Previous research has substantiated the efficacy of SWE in detecting fibrosis18,19. This study aimed to develop a combined deep-learning model that leverages liver ultrasound, liver elastography images, and clinical features to assess the risk of fibrotic NASH. The performance of this combined model was compared with deep learning models trained exclusively on 2D ultrasound images and models trained solely on elastography images. Materials and methods Animal model This study was approved by the Animal Care and Use Committee of our hospital and complied with the national standards for animal care and use in China. All procedures were conducted in accordance with the ARRIVE guidelines, and all methods were performed in accordance with the relevant guidelines and regulations. Ethical approval number: 2024-KY-112. A total of ninety male Sprague-Dawley (SD) rats (initial weight: 140–170 g) were used. The rats used in this study were obtained from the Pengyue Experimental Animal Breeding Co., Ltd. in Jinan, China.After a three-day acclimatization period, the rats were randomly divided into two groups: a normal control group (n = 30) fed a standard diet and a high-fat diet group (n = 60) fed a high-fat diet. The dietary energy composition was as follows: 20% protein, 20% carbohydrates, and 60% fat. The high-fat diet group received a CCl₄ oil mixture (1:4) from week 8 to week 14 (0.2 mL/kg, twice weekly, intraperitoneally).In the control group, 10 rats were randomly selected at weeks 4, 10, and 14, while in the high-fat diet group, 10 rats were randomly selected at weeks 4, 6, 8, 10, 12, and 14. After intraperitoneal injection of pentobarbital sodium for anesthesia, imaging examinations were performed, and blood samples were collected from the abdominal aorta. Subsequently, under anesthesia, cervical dislocation was performed to euthanize the rats, and liver tissue was collected. Multimodal ultrasound image acquisition Before the acquisition of ultrasound images, all rats were fasted for 24 h. Anesthesia was induced by intraperitoneal injection of sodium pentobarbital (3% saline solution, 40 mg/kg). Following successful anesthesia, the rats’ abdomens were shaved, and the animals were secured on the operating table for ultrasound scanning. Ultrasound examinations were performed using an ACUSON Sequoia real-time shear wave elastography ultrasound diagnostic system (Siemens, USA) equipped with a conventional linear array 10L4 probe (4–10 MHz). The examination commenced with routine B-mode ultrasound scanning. After the standard B-mode scan, 2D images of the liver’s median lobe, left lateral lobe, and left medial lobe were consistently acquired and stored. Subsequently, the modality was switched to 2D-shear wave elastography (SWE) mode to obtain elastography data for the same liver lobes. These ultrasound images, encompassing both grayscale ultrasound and SWE results, were archived in the Digital Imaging and Communications in Medicine (DICOM) format for subsequent radiomics analysis. Serum and tissue pathological analysis After completing the ultrasound scans, the rats’ body weight was measured. Subsequently, blood was collected via puncture of the abdominal aorta. The rats were then euthanized by cervical dislocation under anesthesia. The liver was rapidly dissected, washed with physiological saline, and the median lobe was fixed in 10% neutral formalin for subsequent analysis. The collected serum samples were appropriately labeled and sent to the hospital’s laboratory for analysis. The tests included measurement of alanine aminotransferase (ALT), aspartate aminotransferase (AST), AST/ALT ratio, gamma-glutamyl transferase (GGT), total cholesterol (CHOL), triglycerides (TG), high-density lipoprotein cholesterol (HDL-C), and low-density lipoprotein cholesterol (LDL-C). The liver tissue was fixed in 10% neutral formalin for 24 h, followed by conventional tissue processing and hematoxylin-eosin (HE) staining. Pathological diagnosis of NAFLD was based on the Steatosis, Activity, and Fibrosis (SAF) scoring system. NASH is characterized by hepatocellular fat accumulation, ballooning degeneration, and portal inflammation, each scored on a scale of 0–3. A total SAF score ≥3 indicates NASH. Significant fibrosis is defined as a fibrosis stage ≥F2 (F0: no fibrosis; F1: peri-sinusoidal or peri-portal fibrosis; F2: peri-sinusoidal and portal/peri-portal fibrosis; F3: bridging fibrosis; F4: cirrhosis). Fibrotic NASH is diagnosed when both NASH and significant fibrosis are present. Deep learning model development This study designed three deep learning models: a pure image model utilizing solely liver ultrasound images, a pure image model employing exclusively liver elastography images, and a combined model integrating both liver ultrasound and liver elastography images, along with pertinent clinical features. The deep learning model employed in this study is based on the ResNet-18 architecture, which leverages residual modules to effectively mitigate degradation and vanishing gradient problems during the training of deep networks. We constructed three separate models: a unimodal 2D ultrasound model, a unimodal SWE model, and a multimodal fusion model. For the fusion model, features were first extracted independently from 2D ultrasound and SWE images using separate network branches, and then fused at the image level via concatenation along the channel dimension. Additionally, clinical features were mapped into a 100-dimensional vector through a fully connected layer and subsequently concatenated with the 512-dimensional image features output from the ResNet branches. These combined features were further integrated using a self-attention mechanism to enhance the model’s ability to capture heterogeneous information, and finally passed through a fully connected classifier to generate predictions. For model training, all networks were initialized with pre-trained ResNet-18 parameters. The Adam optimizer was employed, with an initial learning rate set to 0.0001 and a batch size of 8. During training, we monitored the loss on the validation set and applied early stopping if no loss reduction was observed within 30 epochs, in order to prevent overfitting. The training process was implemented using the PyTorch framework on an NVIDIA GeForce RTX 4060 Ti GPU.To enhance the generalization capability of the model, a variety of commonly used data augmentation techniques were applied to all input ultrasound and SWE images. Specifically, random horizontal flipping was applied with a probability of 50%, meaning that half of the training samples were flipped left-to-right to increase data diversity. In addition, images were randomly cropped with a scaling factor ranging from 0.8 to 1.0 and then resized back to the standard input size required by the model, simulating variations in the field of view during image acquisition. Brightness, contrast, saturation, and hue were also randomly adjusted with predefined probabilities: brightness and contrast were varied within ± 20%, saturation within ± 10%, and hue within ± 5 degrees. All augmentations were performed online during training, allowing the model to encounter a wider range of image variations in each epoch. This reduces reliance on specific acquisition conditions and improves the model’s robustness to real-world data. These augmentation strategies are consistent with existing literature and standard practices in medical image processing, ensuring the validity of the samples while effectively mitigating the risk of overfitting due to limited dataset size. Univariate analysis was conducted to identify indicators associated with fibrosis, followed by multivariate logistic regression analysis to determine independent predictors. To predict fibrotic non-alcoholic steatohepatitis based on ultrasound images alone or in combination with clinical factors, we employed various deep-learning architectures. For the pure image model, a ResNet18 network with a fully connected layer was utilized to extract high-dimensional features from the imaging data and directly predict the probability of fibrotic non-alcoholic steatohepatitis. For the combined model ResNet-based Hybrid Network(ResHy-Net), as shown in Fig. 1, a Convolutional Neural Network (CNN) network was employed as a powerful feature extractor, with the final layer and classification layer removed to obtain necessary feature maps. Ultrasound features from different modalities were extracted using the CNN network and concatenated along the channel dimension. Clinical features were transformed into a 100-dimensional vector through a fully connected layer and concatenated with the 512-dimensional ultrasound image features. This combined feature vector was then input into a multi-head self-attention module (Fig. 2), where features were segmented into fixed-size blocks and subjected to multi-head self-attention calculations to incorporate positional information. The processed blocks were subsequently classified using a classifier, with the fused information decoded through a multi-layer perceptron to predict the fibrosis status of the rat liver. To address the issue of overfitting due to data imbalance, a weighted oversampling method and binary cross-entropy loss function were employed. Overview of the ResHy Network. Using CNNs, features from 2D ultrasound and SWE images were extracted separately, and the feature maps were concatenated. A spatial attention module was then employed to fuse the features from different sequences along with clinical characteristics. Finally, the liver’s status was determined by a classifier, specifically a multi-layer perceptron. The details of spatial attention module. Split the feature map into fxed-size patches and perform multi-head self-attention calculation on these diferent patches to obtain a feature map with position information. Considering the imbalance between fibrotic NASH and non-fibrotic NASH samples in our study, we employed a combined strategy of weighted oversampling and loss function reweighting to effectively mitigate model bias caused by class imbalance. Specifically, during the oversampling phase, we adopted the WeightedRandomSampler implemented in PyTorch, with sampling weights assigned as the inverse of the class frequencies in the training set. In other words, since fibrotic NASH samples are underrepresented, each instance in this class was given a higher probability of being selected during training. This approach ensures a more balanced distribution of modality data within each training epoch. Unlike naive oversampling methods that may repeatedly duplicate rare samples, our strategy achieves distributional rebalancing while preserving the full diversity of the dataset and avoiding overfitting to individual instances. At the loss function level, to further enhance the model’s discriminative ability for the minority class, we incorporated a weighting scheme into the conventional binary cross-entropy (BCE) loss, resulting in a weighted binary cross-entropy loss. Specifically, the loss assigns weights inversely proportional to the sample counts of each class: fibrotic NASH (the minority class) samples are given higher penalty weights (e.g., the positive class weight is set as the ratio of negative samples to total samples, and vice versa for the negative class). This ensures that during training error optimization, the model does not become overly biased toward the majority class. This combined strategy of weighted sampling and weighted BCE loss ensures that the model not only increases the training frequency of minority class samples at the data sampling level but also heightens the sensitivity to misclassification of minority classes at the loss penalty level. It is a well-established, mainstream, and effective approach for addressing class imbalance in medical datasets. The weighted binary cross-entropy loss (Weighted BCE) we employed is defined as follows: where N is the total number of samples, yi is the true label of the i-th sample (yi = 1 for fibrotic NASH, and yi = 0 for non-fibrotic NASH), pi is the predicted probability from the model, and ω1 are ω0 the loss weights for the fibrotic NASH (positive class) and non-fibrotic NASH (negative class), respectively. These weights are set as the inverse of the class sample counts: Where N1 and N0 represent the number of positive and negative samples, respectively. This weighting scheme ensures that during optimization, greater emphasis is placed on the minority class samples. At the sampling level, we adopted the WeightedRandomSampler based on the above weights ω1 are ω0 in order to increase the occurrence probability of minority class samples during training. Implementation of CNN and cross-validation To evaluate the performance of ResNet-based Hybrid Network(ResHy-Net), an independent network based on CNN ResNet18, the dataset was divided into training and validation sets. In our study, the train-validation split was performed at the animal level rather than the image level. Specifically, we ensured that all images derived from the same rat were assigned exclusively to either the training set or the validation set, never to both. This approach was adopted to strictly prevent data leakage and to ensure that the evaluation of model performance reflects its generalization to unseen subjects rather than repeated sampling of the same individual.In addition, we employed three-fold cross-validation to evaluate the performance of the model. Initially, three rounds of training were conducted using individual ultrasound images. Subsequently, three rounds of training were performed with multimodal ultrasound images. In each round, one group served as the validation set, while the remaining data was used for training. A separate model was generated from each training session. The overall performance was evaluated by averaging the validation results from the three models. This approach mitigated the impact of uneven data distribution and provided a more robust assessment of the model’s performance. Image augmentation techniques, including flipping, cropping, and adjustments to brightness, contrast, saturation, and hue, were applied to the training images. The network was implemented using PyTorch with the Adam (Adaptive Moment Estimation) optimizer. The initial learning rate was set to 10⁻⁴, and the batch size was 8. An early stopping strategy was implemented, terminating training if the validation loss did not decrease for 30 epochs. The ResNet parameters were initialized using pre-trained weights from the ImageNet dataset. Network training was conducted on an NVIDIA GeForce RTX 4060 Ti GPU. Statistical analysis To evaluate the overall model accuracy, we used the output probabilities from the softmax layer of the model and applied a threshold of 0.5 to determine the fibrosis status. An average softmax score greater than 0.5 indicated a prediction of fibrotic status, while an average softmax score less than 0.5 indicated a prediction of non-fibrotic status. Although 0.5 is the standard threshold for neural network classification, this threshold can be adjusted between 0 and 1 to modify the network’s sensitivity and specificity. Using this method, algorithm performance was evaluated by calculating the area under the curve (AUC) across different threshold ranges. To assess the algorithm’s generalizability, a three-fold cross-validation approach was employed. In each iteration, one-third of the data was used for validation, while the remaining two-thirds were used for training. The reported results are from the validation sets after three-fold cross-validation. Finally, the model’s performance was evaluated using ROC curves. Decision curve analysis (DCA) was employed to compare the clinical value of various predictive models, and calibration curves were used to assess model fit. DCA calculations were primarily performed using the R software packages “rms” and “rmda”. A p-value of less than 0.05 was considered statistically significant for all two-sided tests. Results The clinical characteristics of rats Three of the 90 rats died unexpectedly, leaving 87 rats for further analysis. Table 1 presents the basic clinical information of the remaining rats. Among these, 23 rats were diagnosed with fibrotic NASH, and 64 rats were diagnosed with non-fibrotic NASH. The mean weight of rats with non-fibrotic NASH was approximately 479.4 ± 90.2 g, while the average weight of rats with fibrotic NASH was approximately 583.3 ± 62.5 g. Significant differences (p < 0.05) were observed between the non-fibrotic NASH and fibrotic NASH rat groups in terms of weight, ALT, AST, GGT, CHOL, LDL-C, and SWE. To identify independent clinical predictors, multivariate logistic regression analysis was performed on the relevant clinical factors. As shown in Table 2, weight, CHOL, and SWE (p < 0.05) were identified as independent clinical predictive factors. Table 1Clinical characteristics.CharacteristicsNon-Fibrotic -NASHFibrotic -NASH P Weight(g), mean (SD)479.4 ± 90.2583.3 ± 62.5< 0.001ALT(U/L), median [Q1,Q3]37.0 [30.0,66.5]113.0 [62.5,302.0]< 0.001AST(U/L), median [Q1,Q3]86.0 [68.8,200.2]440.0 [186.0,580.5]< 0.001AST/ALT, median [Q1,Q3]2.6 [2.1,3.6]2.8 [2.2,3.5]0.616GGT(U/L), median [Q1,Q3]1.0 [1.0,2.0]2.0 [1.5,11.0]0.023CHOL(mmol/L), mean (SD)1.9 (0.4)1.7 (0.5)0.030TG(mmol/L), mean (SD)0.7 (0.3)0.8 (0.3)0.613HDL-C(mmol/L), mean (SD)1.0 (0.4)0.8 (0.2)0.063LDL-C(mmol/L), mean (SD)0.3 (0.1)0.4 (0.2)0.0042D-SWE (kPa), median [Q1,Q3]4.3 [3.8,4.8]6.2 [5.1,6.8]< 0.001 Clinical characteristics. Table 2Univariate and multivariate logistic regression analysis for Fibrosis-NASH in the training set.CharacteristicsUnivariate analysisMultivariate analysisOR (95% CI) p OR (95% CI) p Weight1.014<0.0011.0160.022(1.007–1.021)(1.002–1.030)ALT1.012<0.0010.9990.987(1.006–1.018)(0.982–1.018)AST1.003<0.0011.0010.867(1.001–1.005)(0.992–1.010)AST/ALT0.9410.616NANA(0.743–1.192)GGT1.7580.0233.1390.103(1.082–2.856)(0.794–12.413)CHOL0.2410.0300.8990.018(0.067–0.870)(0.832–0.971)TG1.4880.613NANA(0.320–6.918)HDL-C0.2910.063NANA(0.079–1.067)LDL-C2.9240.001NANA(1.699–5.028)2D-SWE6.891<0.0016.8470.003(2.923–16.245)(1.895–24.733)CI, confidence interval; NA, not available; OR, odds ratio; SWE, shear wave elastography. Univariate and multivariate logistic regression analysis for Fibrosis-NASH in the training set. CI, confidence interval; NA, not available; OR, odds ratio; SWE, shear wave elastography. Interpretability of deep learning radiomics models Explainable artificial intelligence can enhance model transparency and trust, providing better insights into the underlying decision-making processes. This is crucial for ensuring fairness, avoiding bias, and maintaining safety in critical fields such as healthcare and finance20. To better understand the decision-making basis of the ResNet-based Hybrid Network(ResHy-Net) model, this study employed Gradient-weighted Class Activation Mapping (Grad-CAM) to visualize the response areas of the deep learning model when assessing fibrotic non-alcoholic steatohepatitis. Significance maps were obtained by applying Grad-CAM to the last convolutional layer of the CNN. Additionally, we calculated the relative weights of each clinical risk factor in the combined model (Fig. 3). Representative example of Grad-CAM saliency map from the combined deep learning model. The red-highlighted area indicates the region most relevant to the diagnosis of fibrotic-NASH as identified by the model. The input image was cropped based on the manually delineated Region of Interest (ROI), and Grad-CAM was applied to this cropped image, which corresponds to the actual input used for model prediction. Comparison of different deep learning models The average cross-validation test accuracy (ACC) of the ResNet-based Hybrid Network(ResHy-Net) model trained on individual 2D ultrasound images was 0.776 ± 0.0139, The average cross-validation test area under the curve value was 0.736 ± 0.0526, The average cross-validation test ACC of the ResNet-based Hybrid Network(ResHy-Net) model trained on individual SWE ultrasound images was 0.776 ± 0.0139, and the average cross-validation test AUC value was 0.776 ± 0.066. The combined model, integrating 2D ultrasound, SWE images, and independent clinical predictors, achieved an ACC of 0.880 ± 0.0372, and an average cross-validation AUC of 0.879 ± 0.0316. A comparison of the accuracy and AUC scores of the three models is presented in Table 3.The performance metrics of all three models, including precision, recall, sensitivity, and specificity, are presented in Supplementary Tables S1–S3.We also included models based solely on clinical features, elasticity values (SWE), and a 2D + SWE deep learning model. All of these models demonstrated inferior performance compared to the combined model. The performance metrics of these models are presented in Table 4. Table 3The cross-validation results for different models.Fold Description2D-US ModelSWE ModelCombined ModelFold NumberACCAUCACCAUCACCAUCFold 0 0.793 0.706 0.793 0.699 0.914 0.915 Fold 1 0.776 0.692 0.776 0.768 0.828 0.838 Fold 2 0.759 0.810 0.759 0.860 0.897 0.884 mean ± SD 0.776 ± 0.0139 0.736 ± 0.0526 0.776 ± 0.0139 0.776 ± 0.066 0.880 ± 0.0372 0.879 ± 0.0316 The cross-validation results for different models. Table 4Performance metrics of other deep learning models.ModelNameCohortACCAUCSenSpePPVNPVF12D + SWETRAIN0.897±0.0520.963 ± 0.0260.917 ± 0.0280.892 ± 0.0620.758 ± 0.1620.965 ± 0.0160.824 ± 0.1062D + SWETEST0.770 ± 0.0270.787 ± 0.1030.619 ± 0.2060.762 ± 0.2260.731 ± 0.3450.765 ± 0.1180.565 ± 0.143ClinicTRAIN0.782 ± 0.0870.788 ± 0.0580.677 ± 0.1730.826 ± 0.1610.881 ± 0.0470.619 ± 0.2320.618 ± 0.128ClinicTEST0.782 ± 0.1390.727 ± 0.0590.585 ± 0.1030.827 ± 0.1850.803 ± 0.0060.708 ± 0.2600.613 ± 0.047SWETRAIN0.701 ± 0.1980.654 ± 0.1930.590 ± 0.0850.772 ± 0.2310.788 ± 0.0820.504 ± 0.3530.449 ± 0.299SWETEST0.746 ± 0.1330.515 ± 0.2170.463 ± 0.4460.856 ± 0.2230.834 ± 0.1550.389 ± 0.3470.404 ± 0.351 Performance metrics of other deep learning models. 0.897± 0.052 The ROC curves for each cross-validation of the three models are presented in Fig. 4. The combined model exhibited exceptionally high sensitivity and specificity. The clinical utility of the models was directly evaluated using DCA. The DCA curves of the standalone 2D and standalone SWE deep learning models are shown in Supplementary Figures S1–S2.The DCA curve for the combined deep learning model (Fig. 5) demonstrates that, across most threshold probability ranges, this model offers a greater net benefit in predicting and diagnosing fibrotic NASH. Additionally, the calibration curve was utilized to assess the prediction performance and accuracy of the multimodal model. In the training set, the average p-value of the Hosmer-Lemeshow (H-L) test for this model was 0.24 (0.26, 0.23, 0.23). In the validation set, the p-values of the H-L test for the combined model were 0.30 (0.06, 0.31, 0.54). These results indicate that the multimodal model is highly suitable for both the training set and the validation set (Fig. 6). ROC curves of the three models. (A): ROC curve of the 2 d deep learning model (B): ROC curve of the SWE deep learning model. (C): ROC curve of the combined deep learning model. DCA Curves of the Combined Deep Learning Model A: DCA Curves for Each Cross-Validation in the Training Set B: DCA Curves for Each Cross-Validation in the Validation Set. Calibration curves of the combined deep learning model. (A): Calibration curves for each cross-validation in the training set (B): Calibration curves for each cross-validation in the validation set. Discussion Multiple studies have confirmed that patients with fibrotic NASH (≥ Stage 2 fibrosis) in NAFLD are at the highest risk of disease progression. Timely diagnosis and treatment of fibrotic NASH can significantly delay disease progression and prevent NAFLD-related complications, which is crucial for improving the long-term quality of life for these patients5,21. Liver biopsy, while the gold standard for monitoring NAFLD progression, has certain limitations and drawbacks22–24. Therefore, developing a non-invasive and repeatable method to predict and diagnose the risk level of fibrotic NASH (≥ Stage 2 fibrosis) is crucial. Currently, deep learning has been widely applied in medical imaging, enabling the extraction of clinically relevant information from complex and diverse clinical datasets, including subtle features often imperceptible to the human eye. This is particularly pertinent to radiological image data, which contains crucial information for disease diagnosis, prognosis, and prediction25–28. In the realm of NAFLD, Omid Pournik et al. developed a predictive neural network model to detect cirrhosis in NAFLD patients. The model achieved a sensitivity of 66%, a false-negative rate of 34%, a specificity of 99%, and a false-positive rate of 1%29. Okanoue T et al. also developed a set of high-sensitivity predictive models using artificial neural networks to screen NAFLD and NASH patients, accurately differentiating between mild, moderate, and advanced fibrosis in NASH30. Choi et al. developed a deep learning model using CT images to predict liver fibrosis, achieving AUCs of 0.96, 0.97, and 0.95 for diagnosing F2, F3, and F4 fibrosis stages, respectively31. Most relevant to our research topic is the work by Kun W. et al., who developed a deep-learning elastography radiomics predictive model using convolutional neural networks to assess liver fibrosis staging. Their model achieved an AUC of 0.85 (95% CI: 0.81–0.89) for detecting fibrosis stages ≥F232. In this study, we employed a NAFLD animal model, enabling the acquisition of comprehensive liver tissue samples for precise pathological diagnosis. This approach circumvents the limitations inherent in patient biopsies, where the small sample size can potentially lead to inaccurate pathological diagnoses33. We leveraged the convolutional ResNet-18 network, a convolutional neural network architecture incorporating residual structures, which offers several advantages over other deep learning networks. These advantages include the ability to mitigate the degradation problem in deep networks, efficient model complexity suitable for resource-constrained scenarios, and ease of training with reduced susceptibility to the vanishing gradient problem. Even with basic optimizers like SGD, it can achieve good results within fewer training cycles26,34,35. Our study found that the joint deep learning model based on the convolutional ResNet-18 architecture, which integrates 2D ultrasound, SWE, and clinical features, significantly outperformed the deep learning models using 2D ultrasound alone or SWE alone in predicting fibrotic-NASH (≥ F2 fibrosis). Additionally, we have included in the attachment the results of models based solely on clinical features, SWE values, and 2D + SWE deep learning model, all of which demonstrated inferior performance compared to the combined model. Although several clinical parameters such as body weight, ALT, AST, and SWE showed statistically significant group differences between fibrotic and non-fibrotic NASH in Table 1, models relying on these features alone yielded limited predictive capability. This inconsistency can be explained by several factors. First, statistical significance in univariate group comparisons does not necessarily translate into strong discriminative power for individual-level prediction, as biochemical and metabolic indices often exhibit overlapping distributions across fibrosis stages. Second, SWE measurements capture the mean elasticity within a limited region of interest and are sensitive to factors such as operator pressure, tissue heterogeneity, and inflammatory or steatotic changes, which can obscure fibrosis-specific stiffness variations. Third, clinical indicators primarily reflect systemic metabolic or inflammatory states and fail to capture the spatial distribution and textural heterogeneity of fibrotic lesions that are encoded within imaging modalities. Therefore, single-modality clinical or SWE-based models lack sufficient structural and contextual information to robustly distinguish fibrosis severity. In contrast, the combined 2D + SWE + clinical deep learning model integrates complementary information—linking biochemical context with morphological and biomechanical cues—thus achieving superior accuracy, generalizability, and clinical interpretability. The combined model achieved a three-fold cross-validation accuracy of 0.880 and an AUC of 0.879, surpassing the performance of a previously reported deep-learning elastography radiomics model. Moreover, the combined model exhibited high specificity, with an average three-fold cross-validation specificity of 0.91. Additionally, the model showed a high positive predictive value (PPV, 0.8) and negative predictive value (NPV, 0.91), indicating its effectiveness in detecting fibrotic NASH cases. This model has the potential to aid clinicians in prioritizing patient evaluation and adopting a more cost-effective approach to assessing NASH status. This study has several limitations that must be acknowledged. Firstly, the data used in this research were derived from animal models, which may not fully capture the intricate nature and variability of NAFLD in human patients. To enhance the applicability of the findings, it would be advantageous to extend the investigation to encompass a larger and more diverse sample of human participants. The incorporation of human data is crucial for future research to evaluate the model’s performance in clinical settings. Additionally, as this is a single-center study, future research should utilize independent datasets from various sources or populations to conduct external validation and assess the model’s robustness and generalizability. Conclusions In this study, we developed and validated a novel multimodal deep learning model, ResHy-Net, integrating 2D ultrasound, real-time shear wave elastography (SWE) images, and key clinical features to predict fibrotic nonalcoholic steatohepatitis (fibrotic-NASH, ≥F2). In three-fold cross-validation, the combined model achieved a mean accuracy (ACC) of 0.880 and a mean area under the receiver operating characteristic curve (AUC) of 0.879, significantly outperforming the single-modality models using only 2D ultrasound (ACC 0.776, AUC 0.736) or only SWE (ACC 0.776, AUC 0.776), demonstrating that multimodal information fusion substantially enhances predictive accuracy and discriminative power. Further evaluation across sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) showed that the combined model attained an average specificity of 0.91, PPV of 0.80, and NPV of 0.91, indicating robust performance in minimizing false negatives and false positives. Grad-CAM visualization revealed that the model automatically focuses on liver tissue regions associated with significant fibrosis and that clinical variables (e.g., body weight, total cholesterol, and SWE values) contribute substantially to its decisions, confirming the interpretability and clinical relevance of multimodal feature fusion. Clinically, a noninvasive, repeatable tool for fibrotic-NASH screening is crucial for early diagnosis and timely intervention. The ResHy-Net model not only demonstrated efficacy in animal experiments but also lays a technological foundation for subsequent clinical deployment. By seamlessly integrating with routine B-mode and SWE ultrasound workflows, the model can provide real-time risk assessment on ultrasound workstations, assisting physicians in accurately identifying high-risk patients requiring pharmacological intervention, thereby optimizing care pathways and reducing reliance on liver biopsy. This study’s limitations include its reliance on an SD rat model, necessitating external validation in multicenter human clinical cohorts. Additionally, the current model does not incorporate serum biochemical panels or genomic data; future work could explore additional modalities to enhance generalizability and individualized prediction accuracy. Next steps will focus on building a large-scale, multicenter clinical sample database, conducting prospective studies and cost-effectiveness analyses to advance the model toward clinical translation. In summary, the proposed multimodal deep learning model exhibits outstanding performance in predicting fibrotic-NASH and offers a powerful tool for early identification and clinical decision-making in nonalcoholic steatohepatitis patients, with promising prospects for broad clinical application. Supplementary Information Below is the link to the electronic supplementary material. Supplementary Material 1 Supplementary Material 2 Author contributions Author Contributions: (I) Conception and design: F. X.; (II) Provision of study materials : J.W.and C.Z.(III) Collection and assembly of data: K.W.and F.X; (IV) Data analysis and interpretation: F.X.andY.W.; (V) Manuscript writing: F.X; (VI) Final approval of manuscript: All authors.F.X. and K.W. have contributed equally to this work. Funding This work was supported by the institutional research project fund of the Second People’s Hospital of Wuhu City, Anhui Province (JC2023B05). Data availability Data is provided within the manuscript or supplementary information files. Data types and formats: - The types of data include raw experimental data, processed data. - Data are available in Excel formats.Supporting materials: - Additional supporting materials such as codebooks and analysis scripts are available from the corresponding author upon request.Contact information: - For any queries or further information, please contact Dr. Wang at wjl980134@163.com. Declarations Competing interests The authors declare no competing interests. Ethics approval and consent to participate All animal experiments conducted in this study were approved by the Institutional Animal Care and Use Committee of the Second People’s Hospital of Wuhu, Anhui Province (Approval No. 2024-KY-112). Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Fei Xia and Kun Wang contributed equally to this work. References Associated Data Supplementary Materials Supplementary Material 1 Supplementary Material 2 Data Availability Statement Data is provided within the manuscript or supplementary information files. Data types and formats: - The types of data include raw experimental data, processed data. - Data are available in Excel formats.Supporting materials: - Additional supporting materials such as codebooks and analysis scripts are available from the corresponding author upon request.Contact information: - For any queries or further information, please contact Dr. Wang at wjl980134@163.com.