Development and validation of a deep learning model for severe mitral stenosis detection from chest X-rays Authors: Bo Li, Kankan Zhao, Ang Liu, Wenqing Xu, Yun Tang, Kai Yang, Linlin Dai, Xiuyu Chen, Shihua Zhao, Chaowu Yan Published: 2025 ABSTRACT AbstractBackgroundAlthough chest X-rays (CXRs) are widely used, diagnosing mitral stenosis (MS) based solely on CXR findings remains challenging in some cases.ObjectiveThis study aimed to develop a deep learning-based artificial intelligence (AI) model to detect MS using CXR.MethodsIn this retrospective study, 515 posteroanterior CXR images were analysed, including 285 from patients with MS and 230 from healthy controls. The dataset was randomly divided into training, validation and test datasets at a 7:2:1 ratio. An AI model was formulated by using the training dataset, and model performance was evaluated on the validation and test datasets using the area under the receiver operating characteristic curve (AUC), precision, recall, F1-score and accuracy. Saliency maps were generated to visualise the regions prioritised by the model.ResultsThe model achieved an AUC of 0.99 on the validation dataset, with a precision of 0.96, recall of 0.96, F1-score of 0.96 and accuracy of 0.96. On the test dataset, the model achieved an AUC of 0.99, with a precision of 0.95, recall of 0.94, F1-score of 0.94 and accuracy of 0.94. Saliency maps highlighted regions consistent with known radiographic features of MS.ConclusionThe developed deep learning-based AI model demonstrated high performance in detecting MS from CXR. This approach may provide a convenient and accessible screening tool for MS, particularly in resource-limited areas. WHAT IS ALREADY KNOWN ON THIS TOPIC Chest X-rays are widely used in routine health screenings; however, the diagnosis of mitral stenosis (MS) based solely on chest X-rays can be challenging in certain cases. Chest X-rays are widely used in routine health screenings; however, the diagnosis of mitral stenosis (MS) based solely on chest X-rays can be challenging in certain cases. WHAT THIS STUDY ADDS We developed an artificial intelligence-based model for detecting MS from chest X-rays, achieving promising diagnostic performance. We developed an artificial intelligence-based model for detecting MS from chest X-rays, achieving promising diagnostic performance. HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY Our study may provide a convenient and accessible screening tool for MS, particularly in resource-limited areas. Our study may provide a convenient and accessible screening tool for MS, particularly in resource-limited areas. Introduction Mitral stenosis (MS) is one of the most common mitral valve disorders, accounting for approximately 12% of all valvular diseases requiring hospital referral.1 2 The condition is primarily caused by either rheumatic heart disease or degenerative changes. Rheumatic fever is the predominant cause of MS globally, making the disease highly prevalent in low- and middle-income countries.3 4 Epidemiological studies from Asia have reported prevalence rates ranging from 0.68 to 11 per 1000 individuals.5 Meanwhile, the incidence of degenerative MS has also been rising steadily in parallel with population ageing in developed countries.6 Transthoracic echocardiography (TTE) is the initial and primary imaging modality used for evaluating patients with MS, providing comprehensive assessment of valve morphology, chamber size and haemodynamics.7 However, it requires specialised expertise and is often limited in resource-poor areas of developing countries. In contrast, chest X-ray (CXR) is often the first imaging step in medical examinations of the chest as it is quick, accessible and involves relatively low radiation exposure.8 In patients with MS, CXR may reveal findings such as left atrial enlargement, prominent pulmonary arteries, right ventricular enlargement and signs of pulmonary venous hypertension (eg, pulmonary congestion or interstitial oedema). Nonetheless, the diagnostic accuracy of CXR is highly dependent on the experience of the radiologists, and radiographic signs may be subtle or absent in some cases. Deep learning is a rapidly evolving branch of artificial intelligence (AI) that uses various neural network architectures capable of autonomously learning and extracting relevant features from raw data.9 10 Unlike traditional machine learning methods that rely on manual feature engineering, deep learning-based models can automatically extract meaningful features from target data. This capability makes deep learning especially well-suited for classifying and quantifying complicated and even unknown features.10 11 In recent years, deep learning has seen increasing application in the analysis of CXRs,1216 such as distinguishing between normal and abnormal findings,17 diagnosing pulmonary tuberculosis18 and detecting the presence of pulmonary hypertension.8 However, studies using deep learning to investigate MS remain limited. In this study, we developed a deep learning-based AI model to detect MS from CXRs. Furthermore, we employed saliency maps to visualise the features learnt by the AI model. Methods Study design In this retrospective study, we developed, validated and tested a deep learning-based model to detect MS using digital CXRs. CXRs were collected from patients with MS who underwent percutaneous balloon mitral valvuloplasty at our institution. We simultaneously collected a cohort of healthy individuals as controls. CXRs were labelled as either MS or healthy control. We then established a classification neural network model using deep learning techniques. Subsequently, we visualised the regions of the CXRs on which the model focused using saliency map. Given the retrospective design of the study and the complete deidentification of all patient data, the requirement for informed consent was waived. Study participants and image acquisition CXR were retrospectively collected from patients with MS, which was initially diagnosed by echocardiography and subsequently confirmed following percutaneous balloon mitral valvuloplasty performed at our institution between 2013 and 2017. All images were acquired in the posteroanterior view with the patient in a standing position. For patients with multiple CXRs, the image closest in time to the preoperative echocardiographic examination was selected for analysis. Exclusion criteria included: (1) a history of prior cardiothoracic surgery, interventional procedures or cardiac device implantation; (2) a history of malignant tumours; (3) pulmonary infections or lesions; (4) CXRs obtained via bedside (portable) imaging; (5) CXR with artefacts or poor image quality and (6) moderate to severe thoracic scoliosis. We also collected a cohort of healthy individuals who underwent CXR for routine health examinations, with no evidence of cardiac enlargement, pulmonary congestion or valvular disease on CXR and echocardiography, and without any known cardiovascular or pulmonary disorders, to serve as the control group. In total, 515 CXRs were included in the study, comprising 285 CXRs from patients with MS and 230 from healthy controls. Ground truth labelling and data partition All CXRs from patients with MS were labelled as MS-positive, while those from healthy controls were defined as MS-negative. All labelled CXRs were randomly divided into training, validation and test datasets at a ratio of 7:2:1. Partition on a participant basis ensured there was no overlap of images or participants among the respective datasets. Model development and evaluation We developed a deep learning model based on the ConvNeXt architecture to differentiate MS from health control. The ConvNeXt network is a convolutional neural network with a modernised architecture chosen for its strong performance in image classification tasks. The model was trained for 200 epochs using the Adam optimizer, which adaptively adjusts learning rates for each parameter to accelerate convergence. To evaluate the diagnostic performance of the model, we calculated standard classification metrics including precision, recall, F1-score, accuracy and area under the receiver operating characteristic curve (AUC) on both the validation and test datasets. Saliency map-based region of interest visualisation To visualise the regions of interest prioritised by the trained AI model, saliency maps were generated individually for each CXR from both MS-positive patients and healthy controls. We employed Gradient-weighted Class Activation Mapping (Grad-CAM) and guided backpropagation to produce case-specific visual explanations. Specifically, a class activation map was computed by applying global average pooling to the final convolutional layer of the trained ConvNeXt model. The resulting pooled outputs were weighted according to their corresponding learnt class-specific importance. These weights were then applied back to the feature maps to produce the saliency maps. Finally, each saliency map was superimposed on the original CXR to enable class-discriminative visualisation, highlighting the regions that contributed most to the model’s prediction. Statistical analysis Age is represented as mean ± SD, while sex is represented as number (%). The receiver operating characteristic (ROC) curve was used to compute the AUC. The AUC, precision, recall, F1-score and accuracy were assessed to evaluate the model. All analyses were performed using R V.4.2.3, with statistical inferences being made using a two-sided significance level of 5%. Result Datasets Figure 1 shows the patient enrolment flowchart for this study. A total of 515 CXRs were analysed. Table 1 summarises the demographic characteristics of the study cohorts. The training dataset included 360 CXRs, comprising 194 from patients (mean age±SD, 43.1±11.0 years) and 166 from healthy individuals (mean age±SD, 34.5±13.2 years). The validation dataset included 103 CXRs, comprising 61 from patients (mean age±SD, 42.3±11.9 years) and 42 from healthy individuals (mean age±SD, 31.7±11.7 years). The test dataset included 52 CXRs, comprising 30 from patients (mean age±SD, 44.1±11.4 years) and 22 from healthy individuals (mean age±SD, 32.8±13.2 years). The concomitant valvular diseases in patients with MS are presented in table 2. Table 1Patient characteristicsTraining dataset (n=360)Validation dataset (n=103)Test dataset (n=52)PatientsHealthPatientsHealthPatientsHealthCXRs, n19416661423022Sex, female165 (85.1)99 (59.6)51 (83.6)22 (52.4)23 (76.7)14 (63.6)Age, years43.1±11.034.5±13.242.3±11.931.7±11.744.1±11.432.8±13.2CXR, chest X-ray. Patient characteristics CXR, chest X-ray. Table 2Concomitant valvular lesions in patients with mitral stenosisTraining dataset (n=194)Validation dataset (n=61)Test dataset (n=30)Mitral regurgitation* Mild52 (26.8)12 (19.7)10 (33.3) Moderate8 (4.1)5 (8.2)2 (6.7)Aortic stenosis* Mild16 (8.2)2 (3.3)1 (3.3) Moderate3 (1.6)0 (0)0 (0)Aortic regurgitation* Mild33 (17.0)7 (11.5)4 (13.3) Moderate17 (8.8)4 (6.6)1 (3.3)Tricuspid regurgitation Mild26 (13.4)9 (14.8)5 (16.7) Moderate22 (11.3)5 (8.2)4 (13.3) Severe4 (2.1)0 (0)0 (0)Data are presented as number (%).*Patients with concomitant mitral regurgitation, aortic stenosis or aortic regurgitation had only mild or moderate lesions; no severe cases were observed. Concomitant valvular lesions in patients with mitral stenosis Data are presented as number (%). Patients with concomitant mitral regurgitation, aortic stenosis or aortic regurgitation had only mild or moderate lesions; no severe cases were observed. Study flowchart. CXR, chest X-ray; MS, mitral stenosis; PBMV, percutaneous balloon mitral valvuloplasty. Model development The model was trained using the training dataset for 200 epochs. During the training process, the model was saved at the epoch with the lowest validation loss, and the corresponding model parameters were selected as the best-performing model. Model evaluation Table 3 shows the performance of the model on the validation and test datasets. On the validation dataset, the model achieved a precision of 0.9616, recall of 0.9612, F1-score of 0.9610, accuracy of 0.9612 and an AUC of 0.9961. On the test dataset, the model achieved a precision of 0.9476, recall of 0.9423, F1-score of 0.9416, accuracy of 0.9423 and an AUC of 0.9924. Figure 2 displayed the ROC curves for the model in detecting MS. Figure 3 illustrates the confusion matrices for both the validation and test datasets, summarising the model’s classification performance across all classes. In each matrix, rows represent the true labels, while columns represent the predicted labels. Saliency maps were generated for true-positive patients and healthy controls in both the validation and test datasets. Figure 4 presents representative saliency maps, demonstrating that the AI model primarily focused on the left upper region of the cardiac silhouette as well as the pulmonary artery regions at both hila. Table 3Summary of key performance metrics for the model on validation and test datasetsPrecisionRecallF1-scoreAccuracyAUCValidation dataset0.96160.96120.96100.96120.9961Test dataset0.94760.94230.94160.94230.9924AUC, area under the receiver operating characteristic curve. Summary of key performance metrics for the model on validation and test datasets AUC, area under the receiver operating characteristic curve. Receiver operating characteristic curves for model performance in identifying mitral stenosis. The area under the receiver operating characteristic curve was 0.9961 for the validation dataset (blue line) and 0.9924 for the test dataset (orange line). AUC, area under the receiver operating characteristic; ROC, receiver operating characteristic. Confusion matrices for the validation dataset (A) and the test dataset (B). The numbers indicate the number of CXRs in each label. The background colour becomes progressively darker as the number increases. CXR, chest X-ray; HC, healthy control; MS, mitral stenosis. Visualisation of the model’s attention using saliency maps to classify healthy controls (A) and MS (B) from CXRs. The red and yellow areas, generated by Grad-CAM, highlight the regions where the model concentrated its attention. In MS cases, the model primarily focused on the left upper region of the cardiac silhouette as well as the pulmonary artery regions at both hila. CXR, chest X-ray; Grad-CAM, Gradient-weighted Class Activation Mapping; MS, mitral stenosis. Discussion Our study developed a deep learning-based model to detect MS from CXRs. Notably, even with a relatively limited training dataset, the model maintained high performance in terms of AUC, precision, recall, F1-score and accuracy across the validation and test datasets. It achieved an AUC of 0.99 in detecting MS on both the validation and test datasets, underscoring its considerable potential as a non-invasive screening tool for MS using CXRs. The aetiology of MS is predominantly rheumatic or degenerative. Although the incidence of rheumatic fever has declined in developed countries, it remains a major healthcare concern in developing regions, particularly affecting younger populations. Degenerative MS is also increasingly observed in the elderly population of high-income countries.4 Hence, it is of great clinical significance to develop a simple, rapid and non-invasive method for screening of MS. Although TTE is effective in assessing MS, its application in low-resourced areas of developing countries is limited by the lack of equipment, the need for time-consuming procedures and a shortage of trained ultrasound physicians. In contrast, CXR is the most common and cost-effective imaging modality, but its diagnostic value is often limited by insufficient clinical awareness of MS among less experienced clinicians, leading to a relatively high rate of missed or delayed diagnoses. Our results demonstrate that CXR, combined with deep learning, holds promise as an accessible and effective method for screening MS, especially in settings lacking advanced imaging resources. Recent advances in deep learning have made automated recognition of valvular diseases increasingly feasible.1921 Ueda et al developed a deep learning model capable of identifying multiple valvular diseases, including MS.21 However, their study included a small number of MS cases, all derived from developed countries. As previously mentioned, degenerative MS is more prevalent in elderly populations in high-income countries. Our study was conducted in a developing country, where rheumatic fever remains the predominant cause of MS. Our study demonstrates that a deep learning-based AI model exhibits good discriminatory performance even in populations predominantly affected by rheumatic MS. We believe that our findings not only complement the work of Ueda et al, but may also be more applicable as a screening tool for MS in developing countries. Saliency maps were used to visualise areas that the model predicted to be most indicative in the CXR of each observation in the current study.22 23 In our study, saliency maps revealed hotspots not only in the left atrial region but also in the pulmonary artery regions at both hila. MS leads to elevated left atrial pressure, resulting in left atrial enlargement, pulmonary venous congestion and pulmonary hypertension, which may further progress to right ventricular enlargement. On CXR, pulmonary hypertension often presents as enlargement of the central pulmonary arteries, particularly at both hila.24 25 The saliency maps demonstrated that the deep learning-based model focused on regions associated with pathological changes caused by MS. It is worth noting that some patients with MS did not exhibit radiographic signs of right ventricular enlargement. Accordingly, as shown in the representative saliency maps in figure 4, our model did not primarily focus on this region. Although pulmonary venous congestion is a well-recognised radiographic feature of MS, the saliency maps in our study did not show prominent activation in the pulmonary vasculature. We speculate that, on one hand, this may be because our model was trained solely to differentiate between patients with MS and healthy controls, where left atrial enlargement and bilateral pulmonary artery dilation at the hila were sufficient for accurate discrimination; on the other hand, some patients with MS indeed did not exhibit obvious signs of pulmonary venous congestion on their CXRs. A comprehensive model trained on a variety of cardiac conditions associated with pulmonary venous congestion, hyperaemia and oligemia may be able to learn and use pulmonary vascular features. This represents a potential direction for future research. It is also noteworthy that, because patients with MS were generally older, concomitant valvular lesions were almost inevitable in this real-world study, particularly secondary tricuspid regurgitation. Nevertheless, all patients in this cohort underwent percutaneous balloon mitral valvuloplasty alone without any concomitant valve surgery. Accordingly, the severity of any concomitant valvular disease did not exceed the moderate level, except for secondary tricuspid regurgitation. Finally, although clinical manifestations were not incorporated in this study, our findings suggest that CXRs alone, when analysed by a deep learning-based model, can provide valuable diagnostic insights. This highlights the growing potential of AI tools to enhance imaging-driven cardiovascular diagnosis. Limitation This study has several limitations. First, the retrospective nature of this study may introduce inherent bias. Second, our study performed in a tertiary referral centre and all included patients had undergone percutaneous balloon mitral valvuloplasty may introduce selection bias. However, all patients in our study had a definitive diagnosis of MS confirmed by interventional procedures, and we believe that this characteristic of the dataset may actually enhance the model’s ability to identify patients with clinically significant MS who are more likely to require intervention. Third, there were some differences in age and sex distribution between patients with MS and healthy controls. However, this reflects the real-world characteristics of MS, and since age and sex were not included as input variables, their impact on model performance was likely minimal. In addition, this was a single-centre study, and the results have not yet been externally validated. To address this limitation, the dataset was randomly split into training, validation and testing subsets (7:2:1), providing an independent test set for unbiased evaluation of generalisation performance. Future studies involving larger, multicentre cohorts are needed to enhance the generalisability of the findings. Nevertheless, our model demonstrated excellent discriminative performance despite the relatively small dataset. Conclusion In summary, we developed a deep learning-based AI model capable of detecting MS from CXR. The model demonstrated high performance and shows potential as a valuable tool to assist in the diagnosis of MS, particularly in resource-limited settings where access to advanced imaging modalities such as CT or echocardiography and trained physicians may be limited. Further prospective, multicentre studies with larger and more diverse patient populations are warranted to validate and expand the clinical utility of this approach.