Inconsistencies in predictive models based on exhaled volatile organic compounds for distinguishing between benign pulmonary nodules and lung cancer: a systematic review Authors: Zhixia Su, Xiaoping Yu, Yuhang He, Taining Sha, Hong Guo, Yujian Tao, Liting Liao, Yanyan Zhang, Guotao Lu, Guangyu Lu, Weijuan Gong Published: 2024 ABSTRACT AbstractBackgroundThere is a general rise in incidentally found pulmonary nodules (PNs) requiring follow-up due to increased CT use. Biopsy and repeated CT scan are the most useful methods for distinguishing between benign PNs and lung cancer, while they are either invasive or involves radiation exposure. Therefore, there has been increasing interest in the analysis of exhaled volatile organic compounds (VOCs) to distinguish between benign PNs and lung cancer because it’s cheap, noninvasive, efficient, and easy-to-use. However, the exact value of breath analysis in this regard remains unclear.MethodsA PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)-oriented systematic search was performed to include studies that established exhaled VOC-based predictive models to distinguish between benign PNs and lung cancer and reported the exact VOCs used. Data regarding study characteristics, performance of the models, which predictors were incorporated, and methodologies for breath collection and analysis were independently extracted by two researchers. The exhaled VOCs incorporated into the predictive models were narratively synthesized, and those compounds that were reported in > 2 studies and reportedly exhibited consistent associations with lung cancer were considered key breath biomarkers. A quality assessment was independently performed by two researchers using both the Newcastle-Ottawa Scale (NOS) and the Prediction Model Risk of Bias Assessment Tool (PROBAST).ResultsA total of 11 articles reporting on 46 VOC-based predictive models were included. The majority relied solely on exhaled VOCs (n = 44), while two incorporated VOCs, demographical factors, and radiological signs. The variation in the sensitivity, specificity, and AUC indicators of the models that incorporated multiple factors was lower compared with those of the models that relied solely on exhaled VOCs. A total of 84 VOCs were incorporated. Of these, 2-butanone, 3-hydroxy-2-butanone, and 2-hydroxyacetaldehyde were identified as key predictors that had significantly higher concentrations in the exhaled breath samples of patients with lung cancer. Substantial heterogeneity was observed in terms of the modeling and validation methods used, as well as the approaches to breath collection and analysis. Many of the reports were missing certain key pieces of clinical and methodological information.ConclusionsAlthough exhaled VOC-based models for predicting cancer risk might be a conceivable role as monitoring tools for PNs risk, there has been little overall change in the accuracy of these tests over time, and their role in routine clinical practice has not yet been established.Clinical trial numberPROSPERO registration number CRD42023381458.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12890-024-03374-2.Keywords: Pulmonary nodules, Lung cancer, Volatile organic compounds, Breath biomarkers, Systematic review Introduction Lung cancer is the second most common cancer and the leading cause of cancer-related deaths worldwide [1, 2]. Approximately half of patients with lung cancer are diagnosed at an advanced stage, missing the optimal treatment window. This contributes significantly to the high mortality rate associated with the malignancy [3]. Therefore, the early detection and diagnosis of lung cancer are of great significance to both clinical practice and general public health. Chest radiography and sputum cytology have been used for lung cancer screening since the 1970s. However, the sensitivity levels of these modalities are low [4, 5]. Over the past several decades, the US National Lung Cancer Screening Trial (NLST) and several other trials in Europe have demonstrated that low-dose computed tomography (LDCT) scan effectively reduces lung cancer mortality by facilitating earlier-stage diagnoses [6–9]. There is a general rise in incidentally found pulmonary nodules (PNs) requiring follow-up due to increased CT use [10]. However, only a small fraction of these nodules are actually lung cancer [11–14]. Therefore, it is essential to develop efficient and easy-to-use techniques for distinguishing between benign PNs and lung cancer, as this is crucial for guiding clinical decision-making. Breath analysis, a simple and noninvasive approach, has shown great potential in diagnosing various pulmonary diseases [15]. Breath is a rich medium comprising gas-phase organic and inorganic compounds, as well as aerosols [16]. In the gas phase, there are hundreds of volatile organic compounds (VOCs) of diverse chemical natures that may be present in trace quantities [16]. Although no specific compounds have been identified whose presence or absence in exhaled breath can reliably indicate lung cancer to date, various prediction models based on exhaled VOCs can accurately classify lung cancer patients and healthy volunteers [17–19]. However, the value of breath analysis to distinguish between benign PNs and lung cancer remains unclear. Published studies on predictive models for distinguishing between benign PNs and lung cancer based on exhaled VOCs have reported conflicting results. First, there is some inconsistency regarding the types and quantities of breath biomarkers incorporated into the models. For example, Peled et al. first developed a diagnostic model based on a single compound (1-octene) to distinguish patients with benign vs. malignant PNs, in their prospective trial [20]. By contrast, a team from the University of Louisville developed PN predictive models by incorporating four compounds: 2-butanone, 3-hydroxy-2-butanone, 2-hydroxyacetaldehyde, and 4-hydroxy-2-hexenal [21–23]. Based on these studies, researchers from Zhejiang University established predictive models by incorporating 19 VOCs to distinguish between benign PNs and lung cancer [24]. It is worth noting that diagnosing lung cancer using a single VOC is challenging, highlighting the importance of incorporating multiple VOCs to conduct more accurate predictions [25, 26]. Second, the performance metrics of existing models have been inconsistent. Two clinical studies have demonstrated that exhaled VOCs can be used to distinguish lung cancers confirmed by biopsy analysis from suspicious PNs observed on repeated LDCT, with high sensitivity and acceptable specificity [23, 27]. However, Liao et al. found that predictive models based solely on exhaled VOCs are not sufficient to accurately identify patients with lung cancer and those with benign PNs and reported that the performance of those models must be improved by combining them with additional factors such as demographic characteristics and radiological findings [28]. This review summarizes the current knowledge regarding exhaled VOC-based predictive models for distinguishing between benign PNs and lung cancer and assesses the overall value of these models in this regard. Methods This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting checklist [29]. The protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO) (registration number CRD42023381458). Search strategy A PRISMA-oriented systematic search was performed in PubMed, Embase, Web of Science, Cochrane Library, the Chinese National Knowledge Infrastructure (CNKI), China Biology Medicine disc (CBM), Wanfang, and CQVIP. We initially searched databases from their inceptions until 23 July 2022, with an update on 30 September 2023. The search was conducted through the combination of the Medical Subject Headings (MeSH) terms and keywords ‘lung neoplasms’ ‘lung cancer’ ‘multiple pulmonary nodules’ ‘solitary pulmonary nodule’ ‘volatile organic compounds’ ‘breath’ and ‘exhaled’. A list of the detailed search strategy used is described in the Appendix 1. Inclusion and exclusion criteria We included studies that reported on the development and validation of exhaled VOC-based risk-predictive models to distinguish between benign PNs and lung cancer. Studies were included if they were conducted in patients with LC confirmed via biopsy analysis, as well as in patients with benign PNs confirmed via biopsy analysis or detected by repeated radiological scans [30]. The included studies needed to identify and report specifically which VOC biomarkers were used. The predictors incorporated into the models were permitted to have been either exhaled VOCs alone or combined with other factors such as demographic characteristics and radiological signs. Articles published in English or Chinese were included, without any restrictions on study design. Studies that analyzed exogenous VOCs or compounds in breath condensate and biofluids such as serum, urine, feces, and gastric content were excluded. Studies were also excluded if they were not carried out in humans or were not relevant to distinguishing between benign PNs and lung cancer. Studies limited to identifying risk predictors, reviews, letters, comments, and conference abstracts were also excluded. Study selection The electronic reports identified were imported into the reference manager Endnote and duplicates removed. Screening title and abstract of studies identified was performed by two researchers (ZXS and GYL) independently, and then full texts were reviewed to determine eligibility for inclusion. During this procedure, potential studies from the reference lists of original articles on this issue were screened and reviewed thoroughly for eligibility. Data extraction and analysis A pre-defined table was designed to extract the variables through a panel discussion with experts and epidemiologists. Information was extracted from each study includes: (1) the characteristics of study (title, first author, publication year, study country, study design, and study participants); (2) the characteristics of prediction models (candidate variables of the models, variables incorporated into the models, statistical method, modeling method, internal validation, external validation, sensitivity, specificity, accuracy, and area under the receiver operating characteristics (ROC) curves); (3) detailed methodologies (breath test environments, patient physiological conditions, sample collection methods, and VOC analysis and identification). Data extraction was independently performed by two researchers (ZXS and GYL). When encountering disagreements, our research team would discuss the article and reach a consistent agreement. Quality assessment The adapted Newcastle-Ottawa Scale (aNOS) used as a tool for risk of bias assessment of the included articles (Appendix 2) [31]. Moreover, the models’ applicability to the intended population and setting were assessed by the Prediction Model Risk of Bias Assessment Tool (PROBAST) (Appendix 3) [32, 33]. Two researchers conducted a critical appraisal of the studies, with any disagreements re-solved through consensus. Results Characteristics of the included studies A total of 2,288 studies were identified. Among them, 2,240 were published in English, while 48 were in Chinese. In addition, five additional studies have been preliminary added according to their titles through other sources, including reference and website [28, 34–37]. After removing 1303 duplicates, a total of 990 studies underwent title and abstract screening, before 450 studies were screened for full-text. A total of 439 studies were excluded for various reasons, as outlined in Appendix 8. Our final analysis included 11 studies (Fig. 1 and Appendix 7). These studies reported 46 VOC-based predictive models that had been used to distinguish a total of 597 patients with benign PNs from 1,700 patients with lung cancer. The number of participants involved in establishing the predictive models ranged between 72 and 768 per study. Of the 11 total studies, seven (63.64%) were conducted in the US and four (36.36%) were carried out in China. Nine (81.82%) were cross-sectional studies, one was a prospective trial, and one was a cohort study. The characteristics of the included studies are listed in Table 1. Selection process of PRISMA flow diagram Table 1Characteristics of the included prediction model studiesStudyYearCountryStudy designSample sizeStudy participantsBPN[n (%)]Patient characteristic informationSettingPatients characteristicsDiagnostic methodsPeled, N. [20]2012Americaprospective trial7253 patients with MPN, 19 with BPN19(26.39)the University of Colorado Cancer Center and Denver Veterans Affairs Medical Center47 NSCLC (30 adenocarcinomas, 13 squamous cell carcinoma, 2 large cell carcinoma, and 2 poorly differentiated carcinoma), 6 SCLCHistologic diagnosis or serial CTBousamra, M., 2nd [21]2014Americacross-sectional147107 patients with LC, 40 patients with BPNs40(27.21)NR97 NSCLC, 8 SCLC, 1 combined SCLC and NSCLC, 1 carcinoidBronchoscopy, CT-guided biopsy, or surgical biopsy/serial CT scansFu, X. A. [22]2014Americacross-sectional12997 patients with MPN, 32 patients with BPN32(24.80)NR88 NSCLC (33 adenocarcinoma, 32 squamous cell carcinoma), 9 SCLCBiopsy or resectionLi, M. X. [61]2015Americacross-sectional11985 patients with MPN, 34 patients with BPN34(28.57)the James Graham Brown Cancer Center at the University of Louisville78 NSCLC (31 adenocarcinoma, 33 squamous cell,14 Others), 7 SCLC;2 pneumonia, 8 granuloma, 3 inflammation, 2 histoplasmosis, 2 epithelial cells, 1 foamy macrophages, 16 unknownCT, pathological diagnosesSchumer, E. M. [23]2015Americacross-sectional221156 patients with LC, 65 patients with BPNs65(29.41)NR103 stage 0-II, 53 stage III-IVPathologic/tissue diagnosis or repeated CTPhillips, M. [27]2019Americacross-sectional8065 patients with MPN, 15 patients with BPN15(18.75)NRNRLDCT, BiopsyShaohua Xie [49]2020Chinacross-sectional147104 with MPN, 43 patients with BPN43(29.25)Sichuan Cancer HospitalNRCT, Pathologic diagnosisPengqiang Liao [28]2021Chinacross-sectional768629 patients with LC, 139 patients with BPN139(18.10)Sichuan Cancer Hospital; Nanchong Central Hospital; Longquanyi CDCNRSurgery or biopsyChen, X. [24]2021Chinacross-sectional230160 patients with MPN, 70 patients with BPN70(30.43)Sir Run Run Shaw Hospital149 NSCLC (117 adenocarcinomas,32 squamous cell carcinoma), 10 SCLCCT, X-ray, pathological diagnosesRai, S. N. [58]2022Americacross-sectional221156 patients with LC, 65 patients with benign PNs65(29.41)healthy control: from patient family membersthe James Graham Brown Cancer Center and Jewish Hospital at the University of Louisville103 stage 0-II, 53 stage III-IVCT, PET scans; biopsy or surgically resected specimensDing, X. [60]2023Chinacohort studyPKUPH Cohort: 100CHPKU Cohort: 63PKUPH Cohort: 49 lung cancer patients and 51 benign pulmonary nodulesCHPKU Cohort: 39 lung cancer and 24 benign nodulesPKUPH Cohort: 51(51.00)CHPKU Cohort: 24(38.10)PKUPH Cohort: the Department of Thoracic Surgery, Peking University People’s Hospital;CHPKU Cohort: the Cancer Hospital of Peking UniversityPKUPH Cohort:stage I; 49 adenocarcinomCHPKU Cohort:stage I-III; 19 adenocarcinom, 2 quamous cell carcinoma, 2 small cell lung cancerCT, pathological diagnosesNR: Not report; MPN: Malignant pulmonary nodules; BPN: Benign pulmonary nodules; LC: Lung cancer; NSCLC: Non-small cell lung cancer; SCLC: Small cell lung cancer; CT: Computerized tomography; PET: positron emission tomography; PKUPH Cohort: Peking University People’s Hospital as the discovery cohort; CHPKU Cohort: the Cancer Hospital of Peking University Characteristics of the included prediction model studies healthy control: from patient family members the James Graham Brown Cancer Center and Jewish Hospital at the University of Louisville PKUPH Cohort: 100 CHPKU Cohort: 63 PKUPH Cohort: 49 lung cancer patients and 51 benign pulmonary nodules CHPKU Cohort: 39 lung cancer and 24 benign nodules PKUPH Cohort: 51(51.00) CHPKU Cohort: 24(38.10) PKUPH Cohort: the Department of Thoracic Surgery, Peking University People’s Hospital; CHPKU Cohort: the Cancer Hospital of Peking University PKUPH Cohort: stage I; 49 adenocarcinom CHPKU Cohort: stage I-III; 19 adenocarcinom, 2 quamous cell carcinoma, 2 small cell lung cancer NR: Not report; MPN: Malignant pulmonary nodules; BPN: Benign pulmonary nodules; LC: Lung cancer; NSCLC: Non-small cell lung cancer; SCLC: Small cell lung cancer; CT: Computerized tomography; PET: positron emission tomography; PKUPH Cohort: Peking University People’s Hospital as the discovery cohort; CHPKU Cohort: the Cancer Hospital of Peking University Breath collection and analysis methods The most commonly used methodology for collecting exhaled breath involved the use of Tedlar bags (n = 8 studies, 72.73%), although the sample volume ranged between 450 and 1,000 mL. All of the studies used MS-based techniques such as gas chromatography-mass spectrometry (GC-MS; n = 5, 45.45%), Fourier-transform ion cyclotron resonance mass spectrometry (FT-ICR-MS; n = 5, 45.45%), and high-pressure photon ionization time-of-flight mass spectrometry (HPPI-TOFMS; n = 1, 9.1%). Notably, solid-phase microextraction (SPME; n = 3, 27.27%) and thermal desorption (TD; n = 2, 18.18%) were the most frequently used methods for pre-concentrating the samples. Regarding the identification of the chemical structures of the VOCs, four of the studies (36.36%) referred to the National Institute of Standards and Technology Library (NIST), while two (18.18%) relied on MS techniques and retention times. The factors that influenced exhaled VOC concentrations were divided into three categories: breath test environments, patient physiological conditions, and breath collection and analysis methods. A total of seven studies (63.64%) described their breath test environments, while seven (63.64%) focused on the physiological conditions of the patients analyzed. All of the included studies discussed their breath collection and analysis methods. Detailed information regarding these factors is presented in Table 2. Table 2Summary of factors reported to influence volatile organic compounds within exhaled breathStudyEnvironmental considerationsPatient-related factorsBreath collection methodsMethod of analysisAmbient airSpecific room for breath samplingDetection contaminationPhysiological conditionsSettingSampling periodSample collection and extraction methodsBreath portionVolumeRouteFlow rateStability of VOCsAnalysis platformSensitivityRepeatabilityVOC identificationPeled, N. [20]NRNRNRNRthe University of Colorado Cancer Center and Denver Veterans Affairs Medical Center, AmericaNRMylar bag + SPMEall750 mLmouthNRNRGC-MSNRNRNRBousamra, M., 2nd [21]collected.NRNRNRNRNRTeldar bagall1 LNRNRNRFT-ICR-MSNRNRNRFu, X. A. [22]collected.NRNRNRNRbefore resection1 L Tedlar bagall1 LNRNRNRFT-ICR-MSNRNRconfirmed by FT-ICR-MS/MSLi, M. X. [61]collected.clinic roomTedlars bags and syringes were tested free of contamination.no diet controlsthe James Graham Brown Cancer Center at the University of Louisville, AmericaNR1 L Tedlar bagallNRNR5 mL/minNRFT-ICR-MSNRNRNRSchumer, E. M. [23]NRNRNRNRNRNRTedlar bagall1 LNRNRNRFT-ICR-MSNRNRNRPhillips, M. [27]NRNRNRNRNRNRa dual-bedded sorbent trap + TDall1 LmouthNRat least 3 yearsGC-MSNRreplicate assays at two independent laboratoriesaccording to mass spectrometry library NISTShaohua Xie [49]collected.a ventilated roomNRSit over 10 min, avoid food and smoke for at least 8 h. No deep inhalation before sampling, no ventilation through nasal and no second breaths during sampling.Sichuan Cancer Hospital, Chinabefore biopsy or resection500 mL Tedlar bag + Bio-VOC breath sampler + SPMElast portion450mLmouthNR1 weekGC-MS35–200 m/zNRaccording to mass spectrometry library NIST 08Pengqiang Liao [28]NRNRNRParticipants were required not to eat for 8 h before breath collection, and rest for at least 10 min in a well-ventilated room; Deep breathing before sampling and nasal ventilation and secondary breathing during sampling were not allowed.Sichuan Cancer Hospital; Nanchong Central Hospital; Longquanyi CDCbefore resection500 ml Tedlar bag + SPMElast portion450mLmouthNRwithin 7 daysGC-MS35-200amuNRaccording to mass spectrometry library NIST and RIChen, X. [24]collected.a specific controlled roomNRAvoid high fat food any antioxidant supplements for at least 24 h. Fasting for 12 h, then stayed in a specific room. Next day before the breakfast, the participants arrived in a specific controlled room for breath sample collection.The participants didn’t brush teeth, and gargled with plain tap water 15 min before.Sir Run Run Shaw Hospital, ChinaNRTDlast portion1 LmouthNRNRGC-MS45–500 m/zRepetitive tests were performed on collected air and breath samples.according to mass spectrometry library NIST 05 and NIST 05 sRai, S. N. [58]collected.a clinic exam roomNRNRhealthy control: patient family membersthe James Graham Brown Cancer Center and Jewish Hospital at the University of LouisvilleNR1 L Tedlar bagall1 Lmouth5 mL/minNRFT-ICR-MSNRNRNRDing, X. [60]collected.a fixed roomNRParticipants fasted for at least 6 h.Participants were asked not to ingest spicy food, alcohol, or coffee the night before exhaled breath collection.Participants first gargled with pure waterPKUPH Cohort: the Department of Thoracic Surgery, Peking University People’s Hospital;CHPKU Cohort: the Cancer Hospital of Peking Universitybefore PET-CT scanning and the morning before surgeryTedlar bagallNRmouthNRwithin 4 hHPPI-TOFMSm/z < 500NRdetected by HPPI-TOFMSNR: Not report; GC-MS: Gas Chromatography Mass Spectrometry; LC: Lung cancer; SPME: Solid phase microextraction; NIST: the National Institute of Standards and Technology Mass Spectrometry library; FT-ICR-MS: Fourier Transform Ion Cyclotron Resonance Mass Spectrometry; TD: thermal desorption; PTR-MS: Proton Transfer Reaction Mass Spectrometry Summary of factors reported to influence volatile organic compounds within exhaled breath healthy control: patient family members the James Graham Brown Cancer Center and Jewish Hospital at the University of Louisville Participants fasted for at least 6 h. Participants were asked not to ingest spicy food, alcohol, or coffee the night before exhaled breath collection. Participants first gargled with pure water PKUPH Cohort: the Department of Thoracic Surgery, Peking University People’s Hospital; CHPKU Cohort: the Cancer Hospital of Peking University NR: Not report; GC-MS: Gas Chromatography Mass Spectrometry; LC: Lung cancer; SPME: Solid phase microextraction; NIST: the National Institute of Standards and Technology Mass Spectrometry library; FT-ICR-MS: Fourier Transform Ion Cyclotron Resonance Mass Spectrometry; TD: thermal desorption; PTR-MS: Proton Transfer Reaction Mass Spectrometry Incorporated predictors The number of predictors incorporated into the models ranged between 1 and 35 (Table 3). Out of 46 total predictive models, 44 (95.64%) were solely based on VOC biomarkers. Only one model (2.18%) incorporated both VOC biomarkers and patient ages. Another model (2.18%) considered VOC biomarkers, age, and radiological signs (including nodule size, count, type, and spiculation). Table 3Predictors, methods, and performance of the assessed development modelsStudyStudy participantsIncluded variablesStatistical methodModeling methodValidation methodNumberSensitivity (%)Specificity (%)Accuracy (%)AUCVOCsno-VOCsPeled, N. [20]53 patients with MPN, 19 with BPN1-octeneNRWilcoxon/Kruskal– Wallis testDFAInternal validation: LOOCV1① 86%① 96.00%① 88.00%① 0.986Bousamra, M., 2nd [21]107 patients with LC, 40 patients with BPNs2-butanone; 3-hydroxy-2-butanone; 2-hydroxy-acetaldehyde; 4-hydroxyhexenalNRWilcoxon test + ANOVANRNR9① NR; ② 21%; ③ 60%; ④ 84%; ⑤ 92%; ⑥ 28%; ⑦ 70%;⑧ 89%; ⑨ 95%① NR; ② 100%; ③ 94%; ④ 74%; ⑤ 45%; ⑥ 100%; ⑦ 95%; ⑧ 74%; ⑨ 45%① -⑨ NR① 0.8;② -⑨ NRFu, X. A. [22]97 patients with MPN, 32 patients with BPN2-butanone; 2-hydroxyacetaldehyde; 3-hydroxy-2-butanone; 4-hydroxyhexenalNRWilcoxon testNRNR1① 89.80%① 81.30%① 87.60%① NRLi, M. X. [61]85 patients with MPN, 34 patients with BPNhydroxyacetaldehyde; 2-butanone; 4-hydroxy-2-hexenal; a mixture of 2-pentanone and pentanal; 3-hydroxy-2-butanoneNRKruskal– Wallis testPLS + SVM + RF + LDA + QDAInternal validation: LCOGV14① 100%; ② -⑭ NR① 81.8%; ② -⑭ NR① 0.946; ② 0.892; ③ 0.892; ④ 0.865; ⑤ 0.811; ⑥ -⑭ NR① -⑤ NR;⑥ 0.901; ⑦ 0.821; ⑧ 0.694; ⑨ 0.793; ⑩ 0.625; ⑪ 0.696; ⑫ 0.793;⑬ 0.66; ⑭ 0.874Schumer, E. M. [23]156 patientd with LC, 65 patients with BPNs2-butanone, 3-hydroxy-2-butanone, 2-hydroxyacetaldehyde, 4-hydroxyhexanalNRNRNRNR3① 93.6%; ② 76.9%; ③ 40.4%① 44.6%; ② 78.5%; ③ 90.8%① -③ NR① -③ NRPhillips, M. [27]65 patients with MPN, 15 patients with BPN1,4-butanediol; 2-pentanamine, 4-methyl-; 2-propanamine; 3-butenamide; 4-penten-2-ol; acetamide, 2-cyanoalanine; N-methylglycine; octodrineNRNRfuzzy logistic regressionBlinded validation; Internal validation: cross-validation2① 80.1%;② NR① 75%;② NR① 80.5%; ② NR① 0.88;② 0.80Shaohua Xie [49]104 with MPN, 43 patients with BPNCyclopentane; Pentane, 3-methyl-; ethylbenzene; N, N-di(methyl)formamideaget-test + Mann-Whitney U test + χ 2 test + Fisher exact probabilitybivariate logistic regressionNR1① 80.80%① 60.50%① NR① 0.781Pengqiang Liao [28]629 patients with LC, 139 patients with BPNα -Pinene; 2,3,6-trimethyl-Heptane; 2-Methylnaphthalene; Tetrachloroethylene; Naphthalene; Naphthalene, 1-methyl-; Furan, tetrahydro-2,2,5,5-tetramethyl; p-Cresol; 1-Propene-1-thiol; 5-Hepten-2-one,6-methyl-; Undecane; Methylvinylketone; Dimethadione; 3-Pentenoicacid,4-methyl; 4-Methoxyphenol; Propanoicacid,2-methyl-,butylester; N,N-Dimethylacetamide; tert-Butanol; Methylenechloride; Guaiacol; AceticAcid; benzene; PropionicAcid; Oxetane,2,2-dimethyl; Acetone; Butanoicacid; Cyclohexane and 3 unknown compoundsage, emphysema, tumor size, tumor type, spicule sign—GA-SVM + Binary logistic regressionInternal validation:5-fold cross-validation2① 51.06%;② 78.7%①68.29%; ②68.3%①54.15%②75.56%①0.65; ②0.776Chen, X. [24]160 patients with MPN, 70 patients with BPN1,2-Dichloroethane; Benzene; 2,4-Dimethylhexane; Heptane; Di-tert-butylperoxide; Tetrachloroethylene; Heptanal; Propylcyclohexane;2-Nonenal, (2E); Cyclohexane, heptyl; 1,4-Methanoazulen-9-ol, decahydro-1,5,5,8a-tetramethyl-,(1R,3aR,4 S,8aS,9 S);2,4-Hexadiyne; Butyricacid; 2-methylbutyricacid; Cyclononasiloxane, octadecamethyl-(8CI,9CI); Octadecamethylcyclononasiloxane; Benzoicacid,3,5-bis(1,1-dimethylethyl); METHYL10-METHYLUNDECANOATE; p-TerphenylNRNRlogistic regression + ANNInternal validation:5-fold cross-validation4①-④ NR①-④ NR①-④ NR① 0.809; ② 0.799;③0.756; ④ 0.779;Rai, S. N. [58]156 patients with LC, 65 patients with benign PNsButyraldehyde; Hexanal;2-Heptanone; Octanal; Undecanal; Butyricacid; Aceticacid; Acrolein; 4-hydroxyhexanal; 4-Hydroxynonenal; Cyclopentanone; DicyclohexylketoneNRBoot-SVM-RFESVMInternal validation: 5-fold cross-validation7①-⑦NR①-⑦NR① 76.98;② 76.07;③ 76.25;④ 78.48;⑤ 76.72; ⑥ 77.65;⑦ 77.92① -⑦NRDing, X. [60]PKUPH Cohort: 49 lung cancer patients and 51 benign pulmonary nodulesCHPKU Cohort: 39 lung cancer and 24 benign nodulesacetaldehyde, 2-hydroxyacetaldehyde, isoprene, pentanal, butyricacid, toluene,2,5-dimethylfuran, cyclohexanone, hexanal, heptanal, acetophenone, propylcyclohexane, octanal, nonanal, decanal, and2, 2-dimethyldecaneNRt-test + Fisher’s exact testRFExternal validation2① 82.1%;② 71.8%① 92.3%;② 76.9%① 84.6%;② 73.1%①0.872; ②0.744①-⑭: model 1 to 14; NR: Not report; Forward SDA: Forward stepwise discriminant analysis; LOOCV: Leave-one-out cross-validation; Fisher SDA: Fisher stepwise discriminant analysis; DFA: Discriminant factor analysis; ANOVA: Analysis of variance; ANN: Artificial neural network; Fisher DA: Fisher discriminant analysis; GA-SVM: Genetic algorithm-support vector machine; PLS: Partial least squares; SVM: Support vector machine classification models; RF: Random forest algorithm; LDA: Linear discriminant analysis; QDA: Quadratic discriminant analysis; LGOCV: Leave group out cross-validation Predictors, methods, and performance of the assessed development models ① 0.8; ② -⑨ NR ① 80.1%; ② NR ① 75%; ② NR ① 0.88; ② 0.80 ① 51.06%; ② 78.7% ①54.15% ②75.56% PKUPH Cohort: 49 lung cancer patients and 51 benign pulmonary nodules CHPKU Cohort: 39 lung cancer and 24 benign nodules ① 82.1%; ② 71.8% ①-⑭: model 1 to 14; NR: Not report; Forward SDA: Forward stepwise discriminant analysis; LOOCV: Leave-one-out cross-validation; Fisher SDA: Fisher stepwise discriminant analysis; DFA: Discriminant factor analysis; ANOVA: Analysis of variance; ANN: Artificial neural network; Fisher DA: Fisher discriminant analysis; GA-SVM: Genetic algorithm-support vector machine; PLS: Partial least squares; SVM: Support vector machine classification models; RF: Random forest algorithm; LDA: Linear discriminant analysis; QDA: Quadratic discriminant analysis; LGOCV: Leave group out cross-validation A total of 84 VOCs were incorporated into the predictive models. The most commonly reported VOCs were 2-hydroxyacetaldehyde (n = 5 studies), 2-butanone (n = 4), 3-hydroxy-2-butanone (n = 4), and butyric acid (n = 3). Among these VOCs, the concentrations of 2-hydroxyacetaldehyde, 2-butanone, and 3-hydroxy-2-butanone were reported to have been significantly higher in the exhaled breath samples of patients with lung cancer. The associations between butyric acid concentration and lung cancer were not reported among any of the included studies. Table 4 presents a comprehensive summary of all of the VOCs that were reported on in ≥ 2 of the studies. The VOCs that were reported on in only one study are listed in Appendix 4, along with their chemical classes and CAS registry numbers. Table 4Volatile organic compounds reported in at least two studiesCompound nameFrequencyLevel of VOCsChemical ClassesCASLC> BPNBPN> LC2-hydroxyacetaldehyde5YesAldehydes141-46-84-hydroxyhexenal2YesAldehydes109710-37-4Acrolein2YesAldehydes107-02-84-hydroxy-2-nonenal2YesAldehydes75899-68-2Acetaldehyde2NRNRAldehydes141-46-84-hydroxyhexanal2YesAldehydes109710-36-3Heptanal2NRNRAldehydes111-71-7Hexanal2YesAldehydes66-25-1Octanal2YesAldehydes124-13-02-butanone4YesKetones78-93-33-hydroxy-2-butanone4YesKetones513-86-0Butyric acid3NRNRAcids107-92-6Acetic Acid2NRNRAcids64-19-7Propylcyclohexane2NRNRAlkanes1678-92-8Benzene2NRNRAromatic compounds71-43-2Tetrachloroethylene2NRNRHaloalkanes127-18-4NR: Not Reported; CAS: Chemical abstracts service; LC: lung cancer; BPN: benign pulmonary nodules Volatile organic compounds reported in at least two studies NR: Not Reported; CAS: Chemical abstracts service; LC: lung cancer; BPN: benign pulmonary nodules Modeling methods Table 3 presents detailed information regarding the modeling methods used in the included studies. The most commonly used modeling methods were machine learning—which included support vector machine classification models (n = 9, 19.57%), logistic regression analysis (n = 6, 13.04%), random forest algorithm (n = 3, 6.52%), discriminant analysis (n = 3, 6.52%), artificial neural network (n = 2, 4.35%), and partial least squares analysis (n = 1, 2.17%). The statistical modeling methods used for 22 of the models (47.83%) were unclear. Predictive performance Each study included in our analysis reported at least one measure of predictive performance (Table 3). Nine of the studies reported classification measures of 20 predictive models. Of these, the sensitivity and specificity values of nine of the models (45%) were reported to both be > 70%. Six of the studies reported index of prediction accuracy values that ranged between 54.15% and 94.6%. Eight of the studies assessed discrimination using AUC values, which ranged between 0.625 and 0.986. Notably, none of the studies reported on calibration measures. The variation in terms of sensitivity, specificity, and AUC values of the models that incorporated multiple factors was lower compared with those of the models that were based solely on exhaled VOCs. Among the models based solely on exhaled VOC biomarkers, the model with the highest sensitivity (100%) exhibited a specificity of 81.8%—while the model with the highest specificity (100%) showed a sensitivity of 28%. When exhaled VOC biomarkers were combined with other factors, the model with the highest sensitivity (80.80%) had a specificity of 60.50%, while the model with the highest specificity (68.3%) had a sensitivity of 78.7%. The models based solely on exhaled VOCs had AUC values ranging between 0.625 and 0.986, while those that incorporated additional factors such as age and radiological signs maintained stable AUC values of 0.776–0.781. Validation approaches Out of the 46 developed models, 19 (41.3%) exclusively underwent internal validation using methods such as K-fold cross-validation (n = 13, 28.3%), leave-group-out cross-validation (LGOCV; n = 5, 10.9%), and leave-one-out cross-validation (LOOCV; n = 1, 2.2%). Two of the models (4.3%) were internally validated through K-fold cross-validation before being subjected to blinded validation at two independent laboratories. Only one study validated two models (4.3%) using a new patient sample. Study quality assessment The aNOS scores of the included studies ranged between 6 and 9 (Appendix 5). Of the 11 total studies, five were rated very good, five were rated good, and one was rated satisfactory. The one study that received satisfactory ratings did not report on the representativeness levels of their patient samples. The PROBAST tool was used to evaluate the risk of bias associated with the included studies. As shown in Fig. 2 and Appendix 6, all were found to have a high risk of bias in terms of participants, predictors, analysis, and overall domains. This high risk of bias may be attributable to the inappropriate inclusion criteria or small sample sizes of the studies. Predictors were also evaluated based on outcome information, and the selection of predictors using univariable analysis was done without reporting on measures of discrimination or calibration for model performance. Furthermore, the risk of bias in the outcome domains was unclear for all 11 of the included studies. Risk of bias and applicability assessment according to the PROBAST In terms of the applicability of the models, nine of the studies had low risks of bias in the participant domain, whereas two had unclear risks of bias. All of the models developed had unclear risks of bias in the predictors, outcomes, and overall domains. Discussion This systematic review assessed the performance of 46 predictive models for lung cancer, based on VOCs detected in exhaled breath samples, reported in 11 studies and involving a total of 2,297 patients. Although the findings indicated that there was significant heterogeneity in the predictive performances of the models in terms of distinguishing between benign PNs and lung cancer, the models that incorporated additional factors such as demographic characteristics and radiological signs showed better performance metrics. Several compounds—including 2-butanone, 3-hydroxy-2-butanone, and 2-hydroxyacetaldehyde—were identified as being the most significant VOCs for distinguishing between benign PNs and lung cancer. Despite the potential of VOC breath analysis, there has been little overall change in the accuracy of these tests over time, and their role in routine clinical practice requires further research to be established. Although the concentrations of exhaled 2-butanone, 3-hydroxy-2-butanone, and 2-hydroxyacetaldehyde have been identified as significant biomarkers for distinguishing between benign pulmonary nodules (PNs) and lung cancer, the endogenous origins of these volatile organic compounds (VOCs) remain largely unknown or speculative. In lung cancer patients, elevated levels of ketones have been observed, which may result from fatty acid oxidation known to increase the production of 2-butanone [38, 39]. A previous study has demonstrated that exhaled 3-hydroxy-2-butanone is likely an endogenous product resulting from the degradation of 2-butanone [40]. Similar research in rodents has shown that 2-butanone is oxidatively metabolized to 3-hydroxy-2-butanone, presumably by cytochrome P-450-dependent monooxygenases [41, 42]. Additionally, 3-hydroxy-2-butanone is produced during the detoxification of acetaldehyde and may participate in pulmonary redox cycling, potentially generating toxic reactive oxygen species (ROS) that can damage lung tissue [43–45]. Endogenous 2-hydroxyacetaldehyde is formed through the oxidative degradation of glucose, as well as from glycated proteins, lipid peroxidation, and the oxidation of amino acids [46]. This compound, produced by human neutrophils during phagocytosis, can be a potential source of ROS and may play an important role in tumor development and progression [46]. It is also important to note that exhaled isoprene has been identified as a marker for lung cancers [47]. Initially, it was believed to originate from hepatic cholesterogenesis; however, recent study has revealed that exhaled isoprene originates from muscular lipolytic cholesterol metabolism, as determined by the IDI2 gene [48]. Therefore, further research is needed to uncover the human metabolic origins of endogenous VOCs in lung cancer to promote clinical validation for diagnosis. Our literature review indicated that, in addition to VOCs alone, incorporating multiple factors, such as patient characteristics and radiological signs, can greatly improve the stability of predictive models based on breath samples. Although the majority of the studies included in this review established predictive models based solely on exhaled VOCs, two of the ones that exhibited superior levels of performance also incorporated patient ages and radiological signs [28, 49]. When considering additional factors, variables such as age and smoking status should be considered first, as these represent known predictors of lung cancer risk [50, 51]. Furthermore, researchers have shown that diagnosing lung cancer using a single VOC is challenging and that incorporating several VOCs may be necessary to enhance the performance of such predictive models [25, 26]. However, including too many variables may increase the risk of model overfitting and spurious relationships [52]. Without proper statistical corrections, spurious correlations can be found even in entirely nonsensical contexts, referred to as Voodoo correlations [53]. This issue is further exacerbated by a relatively small sample size, which reduces the power of statistical tests and increases the likelihood of false positives. Therefore, well-established techniques such as the Bonferroni correction or False Discovery Rate (FDR) control that can and should be used for the reducing the likelihood of spurious correlations [54, 55]. Performance of predictive models varied significantly depending on the algorithm of machine learning. The main application field of logistic regression (LR) and support vector machine (SVM) is binary classification which makes these algorithms attractive to solve these clinical tasks [56, 57]. Xie SH, Liao PQ, and Chen X et al. utilized LR algorithm to develop predictive models based on exhaled VOCs to distinguish between benign PNs and lung cancer [24, 28, 49]. The predictive performance was almost the same for these models, with acceptable AUC values. Rai et al. trained SVM to establish their relevance in lung cancer patients’ classification which also achieved an acceptable accuracy [58]. The Random forest (RF) algorithm is more flexible because it can identify a broader scope of possible relationships between the model predictors and the disease status [59]. For example, Ding, X, et al. demonstrated the efficacy of the RF models based 16 exhaled VOCs for discriminating lung cancer from benign PNs [60]. In addition, validation approaches also varied widely between studies, including K-fold cross-validation [24, 28, 58] leave-group-out cross-validation [61], and leave-one-out cross-validation [20], which makes replication of results between studies difficult. Our review further highlighted that sample collection and measurement techniques are hard to rationalize in clinical perspective. Many researchers have optimized method of exhaled breath analysis before the study of real subjects, but the results are varied. For example, some studies have required participants to fast for 6–12 h before breath collection [24, 28, 49], while Li MX. et al. conducted in patients without any diet controls [61]. In addition, some researchers have utilized Tedlar bags, Mylar bags [20–23, 61], and Bio-VOC breath samplers [49, 62] to collect breath samples. While others preferred portable breath sample collection devices or self-made collection devices [63, 64]. Only the end-tidal phase of a breath represents systemic concentrations of VOCs [65], while the lack of alveolar sampling in the included studies is biased via various confounding effects [21–23, 58, 60, 61]. Physio-metabolic and analytical confounders are to be minimized to realize actual pathophysiological effects on exhaled VOC concentrations [66]. Therefore, to standardize the procedure for exhaled breath analysis, expert opinions can be gathered through various methods, such as the Delphi process [67]. MS-based techniques are generally considered the gold standard for analyzing VOCs, owing to their ability to determine the molecular masses and possible chemical structures of individual VOCs [68]. However, this technique is subject to certain limitations. First, the analysis of VOC biomarkers using MS-based techniques requires pre-concentration and a high level of expertise, making them too expensive and complex for many clinical application [69, 70]. Moreover, pre-concentration methods may selectively enhance the signals of certain VOCs while simultaneously leading to the loss of others [20]. Second, breath samples are recommended to be analyzed within six hours of collection to best preserve sample compositions [71]. However, it is important to consider that background pollutants from the sampling equipment may potentially alter breath sample compositions as well [71, 72]. From a clinical perspective, clinicians are more interested in identifying “treatable traits” by stratifying patients based on clinically relevant qualities such as diagnosis, prognosis, and treatment response [73, 74]. In this scenario, simpler more cost-effective techniques have been considered and developed, such as the electronic nose (or “e-nose”) [75]. Many researchers have concluded that e-noses have the potential to become promising diagnostic tools in everyday clinical practices [76, 77]. Although existing VOC-based predictive models have shown acceptable levels of performance for distinguishing between benign PNs and lung cancer, the majority of the existing predictive models for distinguishing between benign PNs and lung cancer have not undergone validation, which hinders their clinical implementation. Ideally, external validations of VOC-based predictive models should be performed in large observational cohorts that have been carefully designed to be accurately representative of real-world patients with benign PNs and lung cancer [78]. To better inform clinical practices, future studies should carefully consider the heterogeneity of prediction effects and conduct model validations to develop predictive models that are of value to clinicians. Limitations This review summarized the current knowledge on exhaled VOC-based predictive models for distinguishing between benign PNs and lung cancer. However, it was subject to certain limitations worth noting. First, there was substantial heterogeneity among the included studies in terms of the methods used for their establishment, as well as for patient breath sample collection and analysis. Consequently, no quantitative meta-analysis has been conducted on the results of the studies. Second, all of the included studies were found to have a high risk of bias, likely caused by inappropriate inclusion criteria, the selection of predictors using univariate analyses, or a lack of indicators that could be used to evaluate calibration risk in the models. Third, most included studies in this review were conducted with cross-sectional study design, which conducted in patients with mid-/late-stage LC. The difference in performance of models based on breath VOCs among benign PNs and early LC demand further investigation with a large sample. In addition, eight of the studies were conducted in small groups, which may have limited the internal validity of their methods and the generalizability of their models to the general population. Finally, studies that used sensor- and pattern-based recognition technologies without reporting on the volatile biomarkers analyzed were not included in this review. Nevertheless, we believe that our report allows for a comprehensive review of exhaled VOC-based predictive models for distinguishing between benign PNs and lung cancer. Conclusion Exhaled 2-butanone, 3-hydroxy-2-butanone, and 2-hydroxyacetaldehyde might be significant breath markers for distinguishing between benign PNs and lung cancer. Moreover, predictive models that incorporate multiple factors alongside exhaled VOCs, such as demographic characteristics and radiological signs, have shown superior levels of performance compared with those based solely on exhaled VOCs. These observations highlight a conceivable role for VOCs as monitoring tools for PNs risk. However, the role of exhaled VOC-based predictive models in routine clinical practice has not yet been established, owing to various constraints associated with breath collection and analysis methods, as well as a general lack of external validation. Further investigations are therefore warranted to standardize the sample collection and measurement techniques for this approach, as well as to enhance the reliability and generalizability of such models. Electronic supplementary material Below is the link to the electronic supplementary material. Supplementary Material 1 Supplementary Material 2 Acknowledgements We thank LetPub (www.letpub.com) for its linguistic assistance during the preparation of this manuscript. Abbreviations Volatile Organic Compounds Pulmonary Nodules Preferred Reporting Items for Systematic Reviews and Meta-Analyses Newcastle-Ottawa Scale Prediction Model Risk of Bias Assessment Tool Area Under the Curve National Lung Cancer Screening Trial Low-dose Computed Tomography The International Prospective Register of Systematic Reviews The Chinese National Knowledge Infrastructure China Biology Medicine disc The Medical Subject Headings Gas Chromatography-Mass Spectrometry Fourier-Transform Ion Cyclotron Resonance Mass Spectrometry High-Pressure Photon Ionization Time-Of-Flight Mass Spectrometry Solid-Phase Micro-Extraction Thermal Desorption National Institute of Standards and Technology Library Leave-Group-Out Cross-Validation Leave-One-Out Cross-Validation European Respiratory Society Author contributions Weijuan Gong and Guangyu Lu had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.Concept and design: Weijuan Gong and Guangyu Lu. Acquisition, analysis, or interpretation of data: All authors. Drafting of the manuscript: Zhixia Su, Weijuan Gong and Guangyu Lu. Critical review of the manuscript for important intellectual content: All authors. Statistical analysis: Zhixia Su, Xiaoping Yu, Yuhang He, Taining Sha, Yujian Tao, Hong Guo, Liting Liao, Yanyan Zhang, and Guotao Lu. Administrative, technical, or material support: Weijuan Gong, Guangyu Lu, Xiaoping Yu, Yujian Tao, Hong Guo, Yanyan Zhang, and Guotao Lu. Supervision: Weijuan Gong and Guangyu Lu. Funding This study was supported by 2022 Jiangsu Provincial Science and Technology Programme Special Funds (Key R&D Programme for Social Development) [Grant no. BE2022775]; the National Natural Science Foundation of China [Grant no. 72374178]; the National Natural Science Foundation of China [Grant no. 71904165]; the Postgraduate Research and Practice Innovation Programme of Jiangsu Province [Grant no KYCX24_3852]. Data availability All data generated or analyzed in the course of this study are included in these published articles and their supplementary information files. Declarations Ethics approval and consent to participate Not applicable. Consent for publication Not applicable. Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Guangyu Lu and Weijuan Gong contributed equally to this work and should be considered co-corresponding authors. Contributor Information Guangyu Lu, Email: guangyu.lu@yzu.edu.cn. Weijuan Gong, Email: wjgong@yzu.edu.cn. References Associated Data Supplementary Materials Supplementary Material 1 Supplementary Material 2 Data Availability Statement All data generated or analyzed in the course of this study are included in these published articles and their supplementary information files.