Interpretable machine learning methods based on oscillometry and electric modeling for the diagnostic of respiratory dysfunction in silicosis Authors: Jorge Luís Machado do Amaral, Cíntia Moraes de Sá Sousa, Caroline de Oliveira Ribeiro, Paula Morisco de Sá, Agnaldo José Lopes, Pedro Lopes de Melo Published: 2026 ABSTRACT PurposeSilicosis, the most dangerous and common lung illness associated with breathing in mineral dust, is a significant health concern. Although Respiratory Oscillometry and electrical models are powerful methods to evaluate the respiratory system, they are not used in silicosis because the interpretation is not simple. This study aims to harness the power of machine learning (ML) to enhance the accuracy and interpretability of oscillometric parameters in silicosis.MethodsData was obtained from 109 volunteers (60 in the training and 49 in the validation groups). Some supervised ML algorithms were selected for tests: K-Nearest Neighbors, Logistic Regression, Random Forest, CatBoost (CAT), Explainable Boosting Machines (EBM), and a deep learning algorithm. Two synthetic data generation algorithms were also applied.ResultsInitially, this study revealed the most accurate oscillometric parameter: the resonant frequency (fr, AUC = 0.86), indicating a moderate accuracy (0.70–0.90). Next, original oscillometric parameters were used as input in the selected algorithms. EBM (AUC = 0.93) and HyperTab (AUC = 0.95) demonstrated the best performance. When feature selection was applied, HyperTab (AUC = 0.94), EBM (AUC = 0.94), and Catboost (AUC = 0.93) emerged as the most accurate results. Finally, external validation resulted in a high diagnostic accuracy (AUC = 0.96). Machine learning algorithms introduced enhanced accuracy in diagnosing respiratory changes associated with silicosis.ConclusionHyperTab and EBM achieved a high diagnostic accuracy range, and EBM explains the importance of the features and their interactions. This AI-assisted workflow has the potential to serve as a valuable decision-support tool for clinicians, enhancing their decision-making process and ultimately leading to improved accuracy and efficiency.Clinical trial numberNot applicable.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12911-026-03568-0. Introduction The oldest, most dangerous, and most common lung illness associated with breathing in mineral dust is silicosis. This disorder is one of the most common occupational diseases globally and can arise in various occupational activities, including cement manufacturing, ceramic and metal industries, shipbuilding, and maintenance [1, 2]. According to Leung, Yu, and Chen [1], radiological findings are typically used to identify this illness, while Cox, Rose, and Lynch [3] have suggested that high-resolution computed tomography (CT) is the most effective technique for evaluating respiratory disorders caused by the environment and at work. It is worth noting that while CT is most effective, there are issues regarding the potential for larger radiation doses and their associated costs. On the other hand, X-Rays scans cannot be used to diagnose a particular disease due to radiological similarities [4]. Spirometry is frequently used to evaluate pulmonary function in silicosis patients. However, this procedure necessitates high patient compliance, as patients must execute maximal forced expiratory maneuvers, which may alter airway features due to changes in bronchial tone. Furthermore, some patients cannot perform these exams due to the need for the cited difficult maneuvers [5]. System Identification aims to build detailed mathematical models of dynamic systems based on their input-output relationships. One approach that has been evaluated to examine the respiratory system biomechanics is the forced oscillation technique (FOT), also known as respiratory oscillometry [6, 7]. It is a specific case of system identification, in which the characteristics of a system (in this case, the respiratory system) are determined based on how the system responds to an external input (pressure waves) [8]. The method is noninvasive and assesses the respiratory system resistance and reactance using exams performed under spontaneous ventilation. These are practical advantages in occupational settings where individuals may hesitate to undergo more invasive and complex tests. Linking detailed oscillometry data with electrical models offers a powerful way to understand respiratory diseases better, enabling clinicians and researchers to analyse the intricate relationships between airflow, pressure, and lung mechanics with greater detail [8]. The extended Resistance-Inertance-Compliance (eRIC) model enhances sensitivity in detecting lung function abnormalities, facilitating the distinction between obstructed and non-obstructed airways [9, 10], enabling early identification of pulmonary diseases [11, 12], and providing a quantitative assessment of lung function impairment [13]. As observed in the previous discussions, respiratory Oscillometry is founded on principles from electrical engineering. Thus, although obtaining respiratory impedance values is relatively straightforward, interpreting these data can be challenging for clinicians [14]. In the diagnostic setting, proper analysis requires specialized training and experience, making it a complex task for professionals not well-versed in the technique [15]. Artificial Intelligence (AI) methods have a long history of contributions to respiratory medicine, dating back to the 1980s [16]. The diagnosis of respiratory disorders has significantly improved with the use of machine learning (ML) techniques in conjunction with oscillometric parameters in chronic obstructive pulmonary disease (COPD) [17, 18], asthma [19], early COPD [15, 20, 21], small airway dysfunction [15, 22], sarcoidosis [23], systemic sclerosis [24], and cystic fibrosis [25]. This technology simplifies and enhances medical decision-making by supplementing human expertise with valuable insights. Explainable ML methods provide the transparency needed for clinicians to trust AI-assisted diagnoses and treatment recommendations [23]. However, no studies have employed machine learning or explainable methods to enhance the diagnosis of respiratory abnormalities in patients with silicosis. Therefore, this work uses machine learning and explainable methods to help the medical team investigate and diagnose respiratory alterations in silicosis based on data obtained by oscillometry and respiratory modeling. This investigation itself presents a few challenges. The first one is the size of the training dataset (sixty volunteers), so we propose the use of synthetic data generators to increase the number of examples that are going to be used for training the machine learning algorithms. The second one is to employ deep learning algorithms in relatively small datasets. Numerous domains, including computer vision [26], natural language processing [27], video analysis [28], and reinforcement learning [29], have already seen significant success with deep learning. Deep learning techniques are less common in tabular data analysis than in other domains. It turns out that neural networks’ performance on tabular data is inferior to that of other, much simpler methods, even though they were initially designed with this goal in mind [30, 31]. The remainder of this article is organized as follows. The next section provides an overview of machine learning approaches to aid the diagnosis of silicosis. The Methods section presents the dataset, the forced oscillation technique, and its parameters, along with a description of the measurement protocol. This section also introduces the evaluated classifiers and describes the methods used for performance evaluation, classifier comparison, feature selection, and experimental design. The fourth section presents the results, and the fifth section discusses the results with respect to the search for the best classifier and parameters for detecting respiratory abnormalities. The sixth section presents the limitations. Finally, the conclusion section summarizes the primary outcomes of this investigation and proposes future steps in this research topic. Recent works that apply machine learning or deep learning to aid the diagnosis of silicosis follow three main approaches: imaging-based deep learning, blood Biomarker-Based machine learning models, and the use of non-invasive and functional diagnostics. In the first approach, the usual strategy is to rely on the use of deep learning models to classify chest X-rays or CT scans for the detection of silicosis. Priego-Torres and her colleagues developed a high-performance deep learning pipeline utilizing chest X-ray data [32]. The model was quite precise in the staging of the disease. Nonetheless, distinguishing between basic silicosis and progressive extensive fibrosis, the disease’s evolved and complex form, offered some challenges, particularly during the transitional stage, when judgment can be very subjective. Despite these difficulties, the models obtained an accuracy of around 81% and ROC AUC scores of nearly 0.93. Aksoy, Atila, and Arslan [33] have demonstrated that deep learning networks and transfer learning algorithms can accurately detect silicosis by classifying CT images. These results show that Xception outperformed the other algorithms and was the most successful in the automatic detection of silicosis, achieving an accuracy rate of 97.29%. SHIVAANIVARSHA & KAVIPRIYA [34] utilized deep learning algorithms to identify CT images for the detection of silicosis. The VGG classifier achieved an accuracy of 99.4% in detecting silicosis. In another publication, the authors proposed a method for silicosis detection using transfer learning techniques on X-ray radiographs [35]. They tested several well-known deep transfer learning models and found that the best model, based on ResNet50 with additional layers, achieved an F1 score of 0.82. Additionally, they employed the Grad-CAM technique to produce heatmap images that highlight the regions from the X-ray images that the deep learning model considered crucial to detect silicosis. It is essential to note that due to radiological similarities, X-ray scans cannot be used to definitively diagnose a particular disease [4]. CT scans provide detailed images and can detect diseases early, but they require higher radiation doses and are more expensive. The second approach employed routine blood tests to extract silicosis biomarkers. In a retrospective case-control study, the authors identified eight silicosis biomarkers from routine blood cell, biochemical, and coagulation parameters. They selected D-dimer (DD), Albumin/Globulin (A/G) ratio, lactate dehydrogenase (LDH), and white blood cells (WBC) for constructing a logistic regression model for silicosis diagnostics [36]. The model had a prediction accuracy of 0.94 with an AUC of 0.98. In another study, a total of 21 biomarkers were quantified, including enzymatic biochemical parameters, white blood cell (WBC) counts, and percentage subsets, as well as WBC-derived ratios. Five classification algorithms were trained and validated using the selected biomarkers identified by the proposed feature selection approach. The resulting models have demonstrated significant performance, enabling accurate classification between the studied groups (Healthy Control, Simple Silicosis, and Pulmonary Massive Fibrosis) with remarkable accuracy. The third approach focuses on breath and functional diagnostics. Baker and his collaborators [37] suggest using atmospheric pressure chemical ionization mass spectrometry (APCI-MS) to examine volatile organic compounds (VOCs) in exhaled breath from 31 patients with silicosis and 60 healthy controls. The authors utilized six interpretable machine learning (ML) models with Shapley additive explanations (SHAP) to categorize the data and identify the most critical VOC features that contribute to model accuracy. The extreme gradient boosting classifier outperformed all other classifiers, with an AUC of 0.933 for the top ten SHAP features. While it is a non-invasive method, it requires the use of liquid chromatography, which may not be available everywhere. Spirometry demonstrates limited sensitivity in detecting early-stage lung function impairment, and in certain advanced cases, measured values may still fall within the established reference range [38]. Taking into consideration the shortcomings of the cited methods, oscillometry powered by machine learning models can offer an additional non-invasive diagnostic approach that is both rapid and acceptable to patients. Results Studied volunteers in the training set There were no significant differences between the analysed and the control groups regarding the anthropometric parameters (Table 1). A significant reduction in all spirometric parameters was observed in patients with silicosis. Table 1Subject demographics and spirometric characteristicsControl(n = 30)Silicosis(n = 30) p Age (years)51.3 ± 15.950.4 ± 13.8nsHeight (cm)171.5 ± 7.9168.7 ± 8.5nsWeight (kg)74.8 ± 11.870.0 ± 13.9nsBMI (kg/m2)25.4 ± 3.624.6 ± 4.3nsGender (M/F)(22/8)(23/7)-Exposition time (years)-15.4 ±-Beginning of symptoms (years)-8.6 ± 8.5-Smoking (pack-years)00-FEV1 (L)3.5 ± 0.62.3 ± 0.8< 0.0001FEV1 (%)101.8 ± 12.070.0 ± 20.0< 0.0001FVC (L)4.3 ± 0.83.5 ± 1.1< 0.001FVC (%)101.6 ± 12.584.8 ± 21.0< 0.002FEV1/FVC (%)90.0 ± 13.467.6 ± 11.6< 0.0001FEF25-75 (L)3.8 ± 1.11.7 ± 0.7< 0.0001FEF25-75 (%)109.8 ± 33.248.0 ±< 0.0001 Table 1Subject demographics and spirometric characteristicsControl(n = 30)Silicosis(n = 30) p Age (years)51.3 ± 15.950.4 ± 13.8nsHeight (cm)171.5 ± 7.9168.7 ± 8.5nsWeight (kg)74.8 ± 11.870.0 ± 13.9nsBMI (kg/m2)25.4 ± 3.624.6 ± 4.3nsGender (M/F)(22/8)(23/7)-Exposition time (years)-15.4 ±-Beginning of symptoms (years)-8.6 ± 8.5-Smoking (pack-years)00-FEV1 (L)3.5 ± 0.62.3 ± 0.8< 0.0001FEV1 (%)101.8 ± 12.070.0 ± 20.0< 0.0001FVC (L)4.3 ± 0.83.5 ± 1.1< 0.001FVC (%)101.6 ± 12.584.8 ± 21.0< 0.002FEV1/FVC (%)90.0 ± 13.467.6 ± 11.6< 0.0001FEF25-75 (L)3.8 ± 1.11.7 ± 0.7< 0.0001FEF25-75 (%)109.8 ± 33.248.0 ±< 0.0001 Subject demographics and spirometric characteristics Forced oscillation parameters Figure 1 displays the resistance and reactance curves obtained from the respiratory oscillometry measurements. The box plots seen in Fig. 2 show an increase in the mean values of fr, Z4, R20, R4-R20, R12, Ax, R12, R, Rp, Rt) in patients with silicosis in comparison with the control group. The test group showed decreased mean values for Cdyn, C, and I compared to the control group. Fig. 1Resistance (above) and reactance (bellow) curves as a function of frequency obtained from the respiratory oscillometry measurements Resistance (above) and reactance (bellow) curves as a function of frequency obtained from the respiratory oscillometry measurements Fig. 2Box plots describing values from the test group compared to the control group in the training set: Resonant frequency (fr, A), dynamic compliance (Cdyn, B), respiratory impedance modulus (Z4, C), resistance in 4 Hz (R4, D), 20 Hz, (R20, E), the difference between R4 and R20 (R4-R20, F), resistance in 12 Hz (R12, G) and area under the reactance curve (Ax, H). Parameters obtained from the electric model are also showed: central resistance (R, I), peripheral resistance (Rp, J), total resistance (Rt, K), respiratory inertance (I, L) and compliance (C, M) Box plots describing values from the test group compared to the control group in the training set: Resonant frequency (fr, A), dynamic compliance (Cdyn, B), respiratory impedance modulus (Z4, C), resistance in 4 Hz (R4, D), 20 Hz, (R20, E), the difference between R4 and R20 (R4-R20, F), resistance in 12 Hz (R12, G) and area under the reactance curve (Ax, H). Parameters obtained from the electric model are also showed: central resistance (R, I), peripheral resistance (Rp, J), total resistance (Rt, K), respiratory inertance (I, L) and compliance (C, M) Experiment 1: Evaluating the diagnostic accuracy of individual FOT parameters Figure 3 summarizes the results of the first experiment, showing moderate diagnostic accuracy for 10 parameters (0.70 ≤ AUC ≤ 0.90). The oscillometric parameters fr and I showed the best performance (AUC = 0.83). Fig. 3Diagnostic accuracy values obtained in the first experiment, considering only the oscillometric parameters Diagnostic accuracy values obtained in the first experiment, considering only the oscillometric parameters The supplementary material includes the ROC curves and tables with the AUC, standard error, sensitivity and specificity and the confidence intervals of each FOT parameter (Fig. S1 and Table S1). Experiment 2: Evaluating the diagnostic accuracy of FOT combined with machine learning Figure 4 shows the AUC for each classifier and the BFP obtained in this experiment, considering the three different situations to generate synthetic data: (1) where no synthetic data was generated (NONE); (2) indicating the use of a machine learning model (FAST ML); and (3) where synthetic data was generated with GAN (CTGAN) Fig. 4Diagnostic accuracy values obtained for each classifier, the best FOT parameter (BFP), and the three situations used to generate synthetic data. NONE indicates that no synthetic data was generated, FAST ML describes that data was generated with a machine learning model, and CTGAN signifies that the data was generated with GAN Diagnostic accuracy values obtained for each classifier, the best FOT parameter (BFP), and the three situations used to generate synthetic data. NONE indicates that no synthetic data was generated, FAST ML describes that data was generated with a machine learning model, and CTGAN signifies that the data was generated with GAN FAST ML and CTGAN methods are a way to limit the over-fitting due to the sample size. They are performed only on the training data, boosting the training data size to 150 samples. The pipeline preprocessing used in this experiment includes only a standard scaling method to ensure that all the inputs have a mean equal to zero and a unit variance. This is important because it represents the inputs on the same scale. Models such as Logistic Regression, KNN or Neural Networks are sensitive to the scale of the variables. Without standardization, attributes with larger values can dominate the model, disproportionately influencing the result. HyperTab and EBM, the top-performing models for the three synthetic data scenarios, achieved excellent AUCs of 0.95 and 0.93, respectively. This success was further highlighted by KNN and CAT, which also achieved an AUC ≥ 0.9, when no synthetic data was generated. In the FAST ML scenario, HyperTab and RF achieved the best performance, with AUC scores of 0.92 and 0.89, respectively. In the CTGAN scenario, EBM, CAT, RF, and HYPERTAB were the top performers with AUC ≥ 0.9, further reinforcing the experiment’s success. Experiment 3: Evaluating the diagnostic accuracy of the best FOT parameters combined with Machine Learning Besides the scaling, the preprocessing pipeline also employs a recursive feature selection (RFE), where one first trains the model with all features and then determines the feature importance. The method continues to remove the less critical features until a drop in performance occurs. At this point, the remaining features are the most important. RFE usually employs Logistic Regression or Decision Trees to assign feature importance. Here, we used Logistic Regression because it is fast and straightforward, and it allows regularization to penalize less essential features. After feature selection, the AUCs of the best-performing FOT parameter (Fr) and six other classifiers are shown in Fig. 5 (K-NN, LR, RF, EBM, CAT, and HyperTab). Considering the NONE case, KNN, EBM, CAT, and RF were the most effective, with an AUC of ≥ 0.9. In the FAST_ML situation, KNN (AUC = 0.90) was the only one that achieved AUC ≥ 0.9. In CTGAN, four classifiers achieved an AUC of ≥ 0.9: RF, CAT, EBM, and HyperTab. In general, EBM and HyperTab outperformed other models, achieving the highest overall AUC (0.94). Fig. 5Diagnostic accuracy values obtained using the studied classifiers with feature selection Diagnostic accuracy values obtained using the studied classifiers with feature selection Figure 6 illustrates the importance of features and their interactions, while Figs. 7 and 8 provides a local explanation for typical normal and diseased volunteers, respectively. Fig. 6Importance of features and their interactions Importance of features and their interactions Fig. 7Example of the local explanation for a typical normal individual Example of the local explanation for a typical normal individual Fig. 8Example of the local explanation for a typical patient with silicosis Example of the local explanation for a typical patient with silicosis Tables 2 and 3 present Se at a higher Sp (Sp = 90%) and Se at a moderate Sp (Sp = 75%) in experiments 2 and 3, respectively. Table 2Sensitivity(%) at 90% SpBFPKNNLRRFEBMCATHyperTabNONE60.073.340.066.780.060.086.7Exp. 2FAST_ML60.066.6740.053.353.353.360.0CTGAN60.070.053.360.073.360.060.0NONE60.073.340.060.080.060.086.33Exp 3FAST_ML60.073.346.773.346.733.340.0CTGAN60.015.053.353.366.666.680.0 Table 2Sensitivity(%) at 90% SpBFPKNNLRRFEBMCATHyperTabNONE60.073.340.066.780.060.086.7Exp. 2FAST_ML60.066.6740.053.353.353.360.0CTGAN60.070.053.360.073.360.060.0NONE60.073.340.060.080.060.086.33Exp 3FAST_ML60.073.346.773.346.733.340.0CTGAN60.015.053.353.366.666.680.0 Sensitivity(%) at 90% Sp Table 3Sensitivity(%) at 75% SpBFPKNNLRRFEBMCATHyperTabNONE66.790.373.386.793.380.0100.0Exp. 2FAST_ML66.784.673.380.073.373.3100.0CTGAN66.781.173.393.386.793.380.0NONE66.775.873.386.786.780.086.7Exp 3FAST_ML66.787.873.380.080.073.373.3CTGAN66.771.773.393.3100.093.380.0 Table 3Sensitivity(%) at 75% SpBFPKNNLRRFEBMCATHyperTabNONE66.790.373.386.793.380.0100.0Exp. 2FAST_ML66.784.673.380.073.373.3100.0CTGAN66.781.173.393.386.793.380.0NONE66.775.873.386.786.780.086.7Exp 3FAST_ML66.787.873.380.080.073.373.3CTGAN66.771.773.393.3100.093.380.0 Sensitivity(%) at 75% Sp External validation Table 4 shows the biometric and spirometric characteristics of the validation set. Table 4Subject demographics and spirometric characteristics in the validation setControl(n = 24)Silicosis(n = 25) p Age (years)46.9 ± 10.647.4 ± 12.2nsHeight (cm)164.4 ± 8.0165.2 ± 7.7nsWeight (kg)69.7 ± 13.264.9 ± 10.9nsBMI (kg/m2)25.6 ± 3.623.8 ± 3.7nsGender (M/F)(11/13)(23/2)-Smoking (pack-years)00-FEV1 (L)3.1 ± 0.72.0 ± 0.9< 0.0001FEV1 (%)104.8 ± 19.065.9 ± 26.5< 0.0001FVC (L)3.8 ± 0.93.1 ± 1.2< 0.05FVC (%)103.1 ± 17.482.2 ± 29.0< 0.01FEV1/FVC (%)101.5 ± 10.181.1 ± 17.3< 0.0001FEF25-75 (L)3.3 ± 1.01.5 ± 0.9< 0.0001FEF25-75 (%)108.1 ± 34.945.0 ±29.3 Table 4Subject demographics and spirometric characteristics in the validation setControl(n = 24)Silicosis(n = 25) p Age (years)46.9 ± 10.647.4 ± 12.2nsHeight (cm)164.4 ± 8.0165.2 ± 7.7nsWeight (kg)69.7 ± 13.264.9 ± 10.9nsBMI (kg/m2)25.6 ± 3.623.8 ± 3.7nsGender (M/F)(11/13)(23/2)-Smoking (pack-years)00-FEV1 (L)3.1 ± 0.72.0 ± 0.9< 0.0001FEV1 (%)104.8 ± 19.065.9 ± 26.5< 0.0001FVC (L)3.8 ± 0.93.1 ± 1.2< 0.05FVC (%)103.1 ± 17.482.2 ± 29.0< 0.01FEV1/FVC (%)101.5 ± 10.181.1 ± 17.3< 0.0001FEF25-75 (L)3.3 ± 1.01.5 ± 0.9< 0.0001FEF25-75 (%)108.1 ± 34.945.0 ±29.3 Subject demographics and spirometric characteristics in the validation set Using the trained EBM model on the validation set achieved an AUC of 0.96. This value is slightly higher than that obtained using the test set in Experiment 2 (AUC = 0.93) and Experiment 3 (AUC = 0.94). Methods Ethical issues and training dataset This cross-sectional work was developed by a collaboration between an R&D laboratory and a teaching tertiary hospital in accordance with the Declaration of Helsinki. Demographic data, including age, height, and weight, were collected from each subject during the procedures. All the volunteers had to sign an informed consent form to be included in this study. The research steps are shown in Fig. 9. Fig. 9Flowchart of the study Flowchart of the study Individuals without a history of lung disease and with pulmonary function tests compatible with normality were evaluated as the control group. The silicosis group consisted of patients from the outpatient clinic who were diagnosed according to international standards [39]. In the training group (Fig. 1), exams were conducted on 30 volunteers in the control group and 30 volunteers with silicosis in the test group, resulting in a primary dataset of 60 instances. Exclusion criteria in this group were a history of other lung disease, COVID-19, or smoking. We intentionally used data from non-smoking patients during system development to allow the developed system to diagnose smokers and non-smokers patients. While the presence of smoking-induced respiratory changes could arguably simplify diagnosis, we sought to develop a more broadly applicable system. Forced oscillation measurements and derived parameters The dataset used in this work was obtained by an oscillometry system developed in our Laboratory [40], according to standard recommendations [7]. Respiratory impedance (Zrs) is a complex variable calculated as the ratio of the Fourier transform (F) of the small (approximately 2cmH2O) excitation pressure (P) and the resulting airflow (V’) at the studied frequencies [Zrs(f) = F(P)/F(V’)]. Three 16-second measurements were performed, and their average was calculated. Tests free from artifacts presenting a coefficient of variation ≤ 10% among measurements were included [7]. To prevent alterations in bronchial muscle tone caused by the maximum effort required in spirometry, these analyses were conducted before spirometry. Additionally, only tests with a coherence function ≥ 0.9 across the entire frequency range were accepted to minimize the impact of spontaneous breathing on the measurement of respiratory impedance [41, 42]. Resistive characteristics were evaluated at 4 Hz, 12 Hz and 20 Hz (R4, R12 and R20, respectively), along with the variation between R4 and R20 (R4-R20). Reactance was assessed through several parameters: the dynamic compliance (Cdyn), the resonance frequency (Fr), the area under the reactance curve (Ax), and the impedance modulus at 4 Hz (Z4). Cdyn, which reflects the dynamic compliance, was calculated from the reactance at 4 Hz (Cdyn = 1/2πfX4). The resonance frequency, defined as the point where respiratory reactance becomes zero, indicates the respiratory system’s homogeneity. Ax was determined by the area under the curve between the lowest frequency (4 Hz), its corresponding reactance (X4), and Fr. The respiratory system mechanical load was assessed using the modulus at 4 Hz (Z4), as it encompasses the respiratory load associated with resistive and elastic characteristics. Following the process outlined in Fig. 1, the methodology proceeds by employing an electrical model of the respiratory system to acquire more detailed biomechanical data. Electrical model of the respiratory system Figure 10 illustrates a simplified electrical model of the oscillometric instrument and the extended resistance-inertance-compliance (eRIC) model, which represents the respiratory system as two-compartment subsystems (central and peripheral) in series [9]. Resistance (R), inductance (L), and capacitance (C) are analogous to respiratory resistance, inertance, and compliance, respectively, while Rp represents the resistance in the periphery of the lungs. Fig. 10Simplified electric model of the oscillometric instrument describing the excitation pressure (P) and resulting airflow (V´), and the two-compartment (central and peripheral) integer-order model (eRIC) used to analyze respiratory impedance. Resistance (R), inductance (I), and capacitance (C) are analogs of respiratory resistance, inertance, and compliance. Rp represents the peripheral resistance Simplified electric model of the oscillometric instrument describing the excitation pressure (P) and resulting airflow (V´), and the two-compartment (central and peripheral) integer-order model (eRIC) used to analyze respiratory impedance. Resistance (R), inductance (I), and capacitance (C) are analogs of respiratory resistance, inertance, and compliance. Rp represents the peripheral resistance Model parameters were estimated within the LABVIEW™ 2018 environment using ModeLIB, developed in our laboratory. A Levenberg-Marquardt algorithm was used in this process to determine the nonlinear model coefficients that yielded the best least-squares fit to the input data. As shown in Fig. 1, the basic data set for the training group were initially obtained using oscillometry and modeling methods in the initial group (scenario 1). The next step in this methodology focus on strategies for synthetic data production (scenarios 2 and 3). Synthetic data generation Synthetic data can be used to supplement, enhance, and occasionally even replace real data when training ML models. It also removes the risk of exposure that comes with sharing data, allowing ML and other software systems that depend on data to be tested. Several methods exist for producing synthetic data. Two methods were employed in this work: the first creates synthetic data using a machine learning model [43]. The correlation between the original data columns. Additionally, it will respect specific statistical parameters like mean, variance, maximum, and minimum values (scenario 2). The generative adversarial network (GAN) is the foundation of the second technique (scenario 3). This architecture shows two neural networks: a discriminator network and a generator network. While the latter is trained to determine whether the data is artificially generated, the former is trained to generate artificial data. The rivalry between these networks enables the GAN to mimic any data distribution [44]. As described in Fig. 1, in the model selection step we compared these three scenarios to evaluate if synthetic data generation strategies would improve diagnostic accuracy: Scenario 1: Basic dataset (60 instances, 30 controls + 30 patients),Scenario 2: 180 instances (150 train set + 30 test set), 120 instances created using a machine learning model.Scenario 3: 180 instances (150 train set + 30 test set), 120 instances created using GAN. Scenario 1: Basic dataset (60 instances, 30 controls + 30 patients), Scenario 1: Basic dataset (60 instances, 30 controls + 30 patients), Scenario 2: 180 instances (150 train set + 30 test set), 120 instances created using a machine learning model. Scenario 2: 180 instances (150 train set + 30 test set), 120 instances created using a machine learning model. Scenario 3: 180 instances (150 train set + 30 test set), 120 instances created using GAN. Scenario 3: 180 instances (150 train set + 30 test set), 120 instances created using GAN. Figure 1 illustrates that these three scenarios were used in several ML models, which are briefly described in the next section. Machine learning algorithms A subfield of AI, machine learning enables computers to learn without explicit instruction [45]. Its methods work best with problems that lack a deterministic solution, where data is utilized to enable algorithms to determine the link between variables automatically. Prior studies have demonstrated the value of combining oscillometric features with machine learning methods to investigate and support the diagnosis of respiratory illnesses [19, 24, 46]. Extensive model testing showed that ensemble methods yielded outstanding results. In the present study, we conduct more experiments and further investigate two distinct ensembles: the Explainable Boosting Machine (EBM) and the Catboost (CAT). The following ML algorithms were also evaluated: Logistic Regression (LR), K-Nearest Neighbor (KNN), Random Forest (RF) [47], and HyperTab, a deep learning approach to small tabular datasets. The first three algorithms have already been briefly detailed in past investigations from our group [15, 46]. In the following, we present a concise overview of the two algorithms not included in our previous work. Full details, please refer to the cited references. Categorical Boosting (CatBoost) [48] is another implementation of gradient boosting that uses binary decision trees as base predictors. It presents similar and sometimes superior speed and accuracy to algorithms in the same class, such as XGBoost [49] and LightGBM [50]. One of the main differences between CatBoost and other algorithms is its implementation of symmetric trees (or oblivious trees). Oblivious means that the same splitting criteria are used throughout the tree level. These trees are balanced, less prone to overfitting, and significantly speed up model execution at test time. Also, it performs gradient boosting differently, using ordered boosting. CatBoost randomly creates an artificial timestamp for each data point if the data does not have a timestamp. It calculates the residual for each data point using the model trained on the other data points up to that point. Thus, different models are trained to calculate subsequent residuals for the other data points with data that the model has yet to see. CatBoost divides the dataset into random partitions and applies ordered boosting to it. By default, the algorithm creates four permutations. With this randomness, it is possible to avoid further overfitting the model. As proposed by Nori et al. [51], the Explainable Boosting Machine (EBM) is a glass-box model that preserves intrinsic interpretability while utilizing cutting-edge performing algorithms, such as boosting and bagging. This model’s primary objective is to construct a Generalized Additive Model with interactions (GA2M) by employing a low learning rate in a round-robin training approach, where one feature is considered at a time. This method outperforms the original GAM in two significant ways. First, it can record the significance of feature interactions or the combined effect of two relevant features on the model. Its general is given in the form of (Eq. 1):1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\:\left(E\left[y\right]\right)={\beta\:}_{0}+\sum\:{f}_{j}\left({x}_{j}\right)+\sum\:{f}_{ij}\left({x}_{i},{x}_{j}\right)\:\:$$\end{document} Where g is the link function, fj and fij are called smooth functions. This approach brings two significant improvements compared to the original GAM [52]. Initially, it can document the importance of feature interactions or the joint impact of two model-relevant features. In the context of respiratory physiology, feature interactions are ubiquitous. For example, changes in airway resistance directly impact lung volume due to air trapping. This condition is often disregarded because most explainability algorithms rely on local linear approximations. Retaining interpretability, while increasing the model’s accuracy, is made possible by capturing the interaction between two features. The second improvement is FAST, a powerful tool for determining and categorizing the level of interaction between each pair of variables. EBM is a condensed version of the model presented by Yin and his collaborators [53]. It is straightforward to evaluate the relative contributions of individual features and pairwise interactions thanks to the additive structure of this model. As a result, local and global interpretations can be obtained using this method. By computing the result of each smooth function given the feature values, we can interpret the local behavior of the model and understand the influence of individual physiological parameters. Similar to logistic regression feature weights, the mean of smooth functions provides a global interpretation. A hypernetwork-based method called HyperTab addresses the issue of classifying tiny tabular datasets [54]. HyperTab builds an ensemble of target networks by leveraging a hypernetwork. A hypernetwork is a neural network that produces the weights of another neural network, known as the target network. Every target network has been designed to process a particular lower-dimensional data view. To obtain lower-dimensional perspectives, a subset of features from the original data is chosen randomly, known as data augmentation. Compared to other tabular data analysis techniques, HyperTab offers several advantages. First, tiny datasets with a far smaller sample size than the number of features can be handled with ease. This procedure enables HyperTab to effectively perform non-domain-specific tabular data augmentation by increasing the quantity and diversity of the training data through feature subsetting, which yields multiple perspectives on each sample. Secondly, due to its independence from specific attributes, HyperTab can be very robust. It is an ensemble where different target networks are trained with different sets of attributes, so it does not compromise its performance if a problematic attribute exists. HyperTab can move forward without requiring any additional tuning if target networks that rely on this particular attribute are removed. With a clear understanding of the used algorithms, the subsequent section (Fig. 1) focus on strategies for evaluating the best decision support system. Experimental design The best system was identified in a study comprising three experiments. The first experiment evaluated the accuracy of each oscillometric parameter in detecting respiratory changes associated with silicosis correctly. All eleven original oscillometric and modelling parameters were applied to ML algorithms in the second experiment. Four of the six chosen classifiers were implemented using Scikit-learn, a Python library. The other two classifiers were also implemented in Python in Hypertab libraries [54] and InterpretML [51]. Since AUC is one of the more widely used metrics in medicine [55] and provides a better method of evaluating classifiers than accuracy [56], it was selected as the performance measure. These values were classified according to Greiner et al. [57] (0.70 < AUC < 0.90, moderate; AUC > 0.90, high accuracy). We split the dataset into training and test sets using the hold-out procedure before any preprocessing of the inputs. Each contains thirty measurements to assess how well machine learning models perform on small datasets. Hyperparameter tuning is a crucial step in model selection, and we applied cross-validation (CV) only to the training data. The Optuna library [58] presents a more efficient strategy for fine-tuning hyperparameters while maintaining the same interface as gridsearchCV, a popular hyperparameter tuning strategy in the Scikit-learn library [59], which explores all possible combinations of hyperparameters. Table 5 presents the classifiers and their respective chosen hyperparameters for tuning. All experiments were conducted using a free Google Colab account, accessible to nearly everyone with Internet access. Table 5Hyperparameters values for tuning in each used classifierClassifierHyperparameters chosen for tuningHyperparameter ValuesLRRegularization Constant(0.0001 -100)KNNNumber of neighbors(1–5)RFNumber of base estimatorsMax depth of base estimators(1000–2000)(1–3)EBMInteractionsMax binsLearning rate(1–10)(10–100)(10-4-1)CATLearning rateMax depth of base estimatorsInteractions(10-4-1)(1–3)(10–100)HyperTabHidden Dims1,2,4,8,10 Table 5Hyperparameters values for tuning in each used classifierClassifierHyperparameters chosen for tuningHyperparameter ValuesLRRegularization Constant(0.0001 -100)KNNNumber of neighbors(1–5)RFNumber of base estimatorsMax depth of base estimators(1000–2000)(1–3)EBMInteractionsMax binsLearning rate(1–10)(10–100)(10-4-1)CATLearning rateMax depth of base estimatorsInteractions(10-4-1)(1–3)(10–100)HyperTabHidden Dims1,2,4,8,10 Hyperparameters values for tuning in each used classifier Number of base estimators Max depth of base estimators (1000–2000) (1–3) Interactions Max bins Learning rate (1–10) (10–100) (10-4-1) Learning rate Max depth of base estimators Interactions (10-4-1) (1–3) (10–100) It is important to emphasize that rigorous quality control of oscillometry exams reduces the presence of outliers. A similar rigorous quality control is also used in spirometry exams. As a result, the studied groups did not present outliers, so it was not necessary to use outlier detection and analysis techniques. Also, there was no need for missing value management; the feature preprocessing employed is the standardization, and no data imbalance analysis was performed because the dataset was balanced. In the third experiment, we aimed to improve algorithm performance by selecting a reduced subset of the initial oscillometric parameters. Based on a wrapper strategy, which identifies input features maximizing average AUC, we employed a Logistic Regressor Classifier for feature selection. To mitigate potential overfitting from this process, and given the small dataset size, two-fold cross-validation was used. The BFP from the first experiment was then compared against the six other classifiers applied in the second and third experiments. As shown in Fig. 1, the methodological description concludes with external validation, which is described in the following section. External validation The system was externally validated using a completely independent data set (Fig. 1). Our University Hospital receives patients from all over Rio de Janeiro, and patients with silicosis come from very different activities such as mining, construction, quarrying and stone crushing, foundry work, sandblasting, and glass and ceramics manufacturing. Therefore, this new data comes from a different source than the training data, ensuring a true test of generalization. We utilized the best model from another dataset, which comprised 24 individuals in the control group and 25 patients with silicosis in the test group. These groups followed the same acceptability criteria used in the training groups. There is only one difference, as in clinical practical conditions, the presence of smoking is familiar in these patients, and smoking was allowed in this group. It is important to emphasize that synthetic data was used solely to evaluate the effect of size augmentation in the training data set. This approach was not used in the validation set. Discussion Four principal findings emerged from this study: (1) initially, it was observed that the use of machine learning algorithms in conjunction with oscillometry and electrical modeling data may achieve an accurate, fast, non-invasive diagnosis of respiratory abnormalities in silicosis; (2) automatic classifiers could increase silicosis diagnosis accuracy; (3) EBM provides clinically important explanations about the model’s diagnosis and (4) external validation confirmed the high accuracy obtained using the previous test set, when we first split the dataset in training and test set. Table 2 shows that the studied groups were homogeneous in the training and test sets, with no significant biometric differences. All comparisons of the pulmonary function parameters presented reduced values in patients with silicosis, which is in close agreement with the pathophysiology of silicosis [1]. The findings in Figs. 3 and 5 are consistent with previous oscillometric analyses [60]. They are also consistent with the structural relationships observed in patients with silicosis between the computed tomography densitometry values and oscillometric parameters [61]. A similar association between eRIC model parameters and structural abnormalities [62] may explain the observed increases in R, Rp, and Rt (Fig. 5I and J, and 5K, respectively) and the decreases in I (Fig. 5L) and Cdyn (Fig. 5M). In the first experiment, Fr and I were the FOT parameters that presented the best individual performance (Fig. 6, AUC = 0.83), achieving moderate accuracy. Experiment 2 utilized all FOT parameters as attributes (Fig. 7). HyperTab achieved the highest overall performance with an AUC of 0.95, while KNN, RF, EBM, and CAT also demonstrated high performance with AUCs of 0.9 or greater. EBM, and HyperTab maintained moderate sensitivity (70–90%) at a specificity of 90% (Table 3). Experiment 3 (Fig. 8) used the best FOT parameters as input in all studied classifiers. This choice of parameters coincided with the feature selection conducted by recursive feature elimination (RFE) employing logistic regression as a base classifier. Six parameters were selected: Fr, R4-R20, Ax, Rp, Rt, and I. The EBM and HyperTab algorithms had the highest performance (AUC = 0.94). Considering the synthetic data generation, CTGAN was the most useful since it enhanced the ML models’ performance. The Synthetic Data Vault library presents a function to evaluate the quality of the generated synthetic data. CTGAN achieves a higher quality (86.39%) compared to FAST-ML (85.43%). Additionally, this can be explained by comparing the distribution using a Kolmogorov-Smirnov test for goodness of fit. In the selected parameters, the observed p-value varies from a minimum of 0.41 (Ax) to a maximum of 0.97 (Rt), indicating that we cannot reject the null hypothesis that the original data and the augmented data with CTGAN came from the same distribution. This suggests that CTGAN performs better in generating synthetic data for these features. In Experiment 3, considering 90% specificity, KNN, EBM and HYPERTAB achieved moderate Se. Regarding 75% specificity (Table 4), RF, EBM, and CAT obtained high sensitivity (≥ 90%) in experiments 2 and 3, indicating ML methods’ efficacy in detecting respiratory changes in silicosis. Regarding overfitting, although that dataset is small, EBM and HyperTab did not present a high degree of overfit when compared to RF. If one defines the amount of overfit as a difference between the performance in the training set and the test set, it is possible to compare those classifiers. In the Experiment 2 in the worst case, the difference was 0.1 for RF, 0.09 for EBM and 0.08 for HyperTab. In Experiment 3, in the worst case, the difference was 0.9 for RF, 0.06 for EBM and 0.13 for HyperTab. However, it is worth mentioning that when HyperTab is trained with the augmented dataset by CTGAN, the difference decreases to 0.04, which indicates a benefit for using CTGAN. This decrease in the difference was also observed in RF (0.08). EBM and HyperTab presented the best scores in the experiments (Figs. 7 and 8) with or without data augmentation. However, EBM has one advantage over HyperTab: the ability to explain the importance of the features and their interactions. The EBM model considers I the most important feature (Fig. 9), followed by fr&I. The importance of fr and I found by EBM, agrees with the performance of individual FOT parameters observed in Fig. 3. The importance of Rp, R12, and R4-R20 reflects the difficulty in breathing related to the airway´s total resistance, especially in the peripheral airway (Rp). The biomechanical interpretation of Z4 relates to the work required to overcome resistive and elastic forces, as reflected in the pressure-volume curve of the lungs. Also, it is interesting to note that the top five of the seven features agree with the best features found by feature selection. Another finding is the importance of the interaction of I with Rt (total resistance). These interactions align with the eRIC model, which has the resistance in series with the inertance and C (compliance). These parameters can be used to define fr and Z4. It also aligns with the findings in [62]. EBM can also explain the model’s decision about an individual. For example, Fig. 10 illustrates the local explanation for an individual in the control group. A positive feature contribution means that the feature value contributes to the model’s decision that this individual has respiratory alterations. A negative contribution indicates that the feature value adds to the model’s decision that this individual does not have respiratory alterations. For this example, the parameter I has the highest negative contribution, followed by fr. Only Ax and Z4 have a small positive contribution. Most negative contributions explain why the model considers that this individual does not have respiratory alterations. If one looks at Fig. 5, the negative contribution of fr is demonstrated by its value, which is located between the first quartile and the third quartile in the control group. The I value is close to the first quartile of the control group. In Fig. 10, we can see the explanations for a person with silicosis. The most positive contributions come from fr and I. fr is equal to 24.16, which is close to the third quartile for the silicosis group. I value is equal to 0, so it is close to the extreme values of the silicosis group. We performed external validation using an independent dataset of typical patients to evaluate model generalizability and robustness in real-world clinical applications. During this study stage, the trained EBM model achieved high diagnostic accuracy (AUC > 0.90). Interestingly, this accuracy was slightly higher than that obtained in the test set in Experiment 2 (AUC = 0.93) and in Experiment 3 (AUC = 0.94). Mean values for FEV1 (%), FEV (%), and FEF25-75 (%) of 70.0, 84.8, and 48.0, respectively, were observed in the training set (Table 2). In the validation set, these mean values were slightly reduced to 65.9, 82.2, and 45.0, respectively (Table 5), describing the presence of increased abnormalities in this group. In patients with silicosis, the presence of smoking introduces additional respiratory abnormalities that, make the diagnosis of respiratory changes simpler. However, not all silicosis patients are smokers, so the system needs to be developed to identify this more difficult condition. It is important to emphasize that we were careful to use non-smoking patients during the system development phase (Table 2). After developing the system in these adverse conditions, we kept the same conditions during external validation and obtained highly accurate results. In clinical practical conditions, the presence of smoking is common in this class of patients. As smoking introduces an additional factor in terms of respiratory abnormalities [1, 63], it is expected that in these conditions, the performance would be even better. Therefore, in the context of the study’s goals, the results obtained during external validation confirmed the high clinical potential of the proposed system. The accuracy obtained during external validation was similar to that recently reported by Wang et al. [64], who achieved AUCs of 0.90 and 0.87 in a system for diagnosing respiratory diseases based on computed tomography (CT) and chest X-ray, respectively. It is also similar to that obtained recently by Zhou et al. [65] in a model for automated lung lesion segmentation and early prediction of acute respiratory distress syndrome, in which the AUC reached 0.90. Our model also presents similar results to those achieved by Torres and her colleagues (AUC = 0.93) [32]. Baker et al. recently [37] investigated the use of machine learning methods and volatile organic compounds measurements to diagnose silicosis in groups of 31 patients and 60 healthy controls. The highest performance achieved was AUC = 0.933, providing a high accuracy. Comparing with the best performance obtained in this study during external validation, it can be observed that an even greater accuracy was obtained (AUC = 0.96). Nowadays, spirometry is the standard diagnostic tool in respiratory physiology. Interpreting these exams for respiratory disease diagnosis requires integrating pattern recognition with clinical context. Topalovic et al. [66] found that AI-based software more accurately matched expected pattern interpretations of spirometric exams (82%) than pulmonologists (75%). The accuracy obtained during external validation in the present study using the trained EBM model (AUC = 0.96) was higher than that obtained in the cited study. From the systems identification point of view, spirometric exams may be considered a step excitation method in which the patient produces the signal under analysis. Step responses directly reveal a system’s transient behavior, which is crucial in applications where settling time, overshoot, and stability are critical considerations. In the particular case of spirometry, step excitation offers simplicity and direct insight into transient response, characterized by the forced expiratory volume in one second (VEF1), a key parameter in this analysis [67]. However, step responses are more susceptible to noise interference, potentially obscuring the system’s underlying behavior. A long research and development history has established robust techniques for analysing system responses to sinusoidal excitation [68]. Accurately identifying frequency response characteristics based on sinusoidal excitation, combined with techniques like Fourier analysis, effectively filters out noise, providing a clearer picture of the system’s true response. Its versatility makes it a valuable tool across various engineering and scientific disciplines, from mechanical and electrical to chemical and biological. In the specific case of the respiratory system, the use of external excitation allows a simple examination allowing the analysis in conditions where traditional pulmonary function is not applicable, including abnormalities in premature babies [69], sleep disorders [70], critically ill patients [71] and during cardiac surgery [71]. However, some significant limitations must be overcome before this important technological innovation can be widely used clinically. First, as noted by recent consensus guidelines [72], diagnostic simplicity is crucial for busy non-specialist clinicians. The complexity of interpreting oscillometric data can delay diagnosis and treatment, particularly in settings with limited access to specialists and time or resources for extensive training. This study describes an accurate diagnostic support system that may improve the medical services offered to patients with silicosis, simplifying the use of oscillometry and improving the diagnostic of respiratory abnormalities in these patients. A critical discussion of this study’s limitations is warranted. Firstly, although the proposed system had good discrimination in external validation and may be used by other health systems for case-identification, discrepancies in feature representation may arise due to differences in the characteristics of the patient cohorts. Patients were collected from one state in Brazil (Rio de Janeiro), potentially introducing selection bias and limiting the generalizability of our findings to other regions. Interested readers can assess the generalizability of the findings to their patient populations by reviewing the anthropometric descriptions and the inclusion and exclusion criteria outlined in this paper for the training and validation sets. Secondly, our study was limited by a relatively small sample size, which may impact the generalizability of the findings. Future studies with larger and more diverse cohorts are needed. Another noteworthy point is that the system real-life performance in clinical settings requires further evaluation. Future studies should assess its integration into clinical workflows and its impact on patient outcomes. Despite its preliminary nature, this analysis contributes significantly to the ongoing discussion of machine learning in respiratory diseases [64, 65] by providing novel data for silicosis, a disease currently unexplored with machine learning approaches. Whole-breath impedance measurements, as used in this study, provide a global assessment of lung mechanics. In contrast, within-breath analysis offers the potential to capture finer-grained, dynamic changes in impedance throughout the respiratory cycle [73]. Future research incorporating within-breath impedance parameters is recommended. Conclusion Using FOT parameters alone, it was demonstrated a limited diagnostic accuracy for detecting respiratory changes in silicosis, with AUC values below 0.9 for all parameters. Achieving high diagnostic accuracy (AUC ≥ 0.9) required combining machine learning classifiers with synthetic data generation algorithms, specifically FAST_ML and CTGAN. CTGAN, particularly when combined with feature selection, yielded superior results. Notably, HyperaTab, a deep learning algorithm designed for small datasets, performed well both with and without synthetic data augmentation. EBM also provides good results (AUC ≥ 0.9). In addition, it gives the importance of the features and their interactions and allows us to investigate further why the model gives a particular result to the subject. External validation achieved high diagnostic accuracy (AUC > 0.90). These findings demonstrate the potential of machine learning, combined with oscillometry and electric models, to accelerate medical workflows and improve the diagnosis of respiratory abnormalities in silicosis. Further research and validation are needed to translate this potential into clinical practice. Electronic Supplementary Material Below is the link to the electronic supplementary material. Supplementary Material 1 Supplementary Material 1