Reporting bias is prevalent in systematic reviews and meta-analyses related to endoscopic vs. microscopic decompression: a systematic review and meta-analysis Authors: Pranit Kumaran, David McCavitt, Zachary Singh, Daniel Rusu, Aidan Lindgren, Henry Avetisian, William Karakash, Dil Patel, Jeffrey C Wang, Raymond J Hah, Ram Alluri Published: 2025 ABSTRACT AbstractBackgroundEndoscopic decompression (ED) and microscopic decompression (MD) are newer minimally invasive approaches for surgical treatment of lumbar spinal stenosis (LSS). However, the absence of large, high-quality randomized controlled trials raises concerns for the potential of bias, or spin, in studies evaluating these techniques. This study aims to analyze the prevalence of spin in abstracts of systematic reviews and meta-analyses comparing ED and MD as treatments for LSS.MethodsStudies were identified using the Preferred Reporting Items for Systematic Reviews and Meta Analysis guidelines searching PubMed, Web of Science (WOS), and Scopus. Articles included were: (I) a systematic review with or without a meta-analysis; (II) degenerative etiology; (III) human subjects; (IV) available in English. Abstracts were graded for incidence of the 15 most common types of spin, and full texts were reviewed using AMSTAR 2 classification. General demographics were identified, including study title, author, journal of publication, year of publication, level of evidence (LOE), study design, and funding. Fisher’s exact test was used to compare study metrics.ResultsTen studies were included, all of which contained at least one type of spin. Spin type 12 (“Conclusion claims equivalence or comparable effectiveness for non-statistically significant results with a wide confidence interval”) and type 3 (Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention) were the most common forms of spin, found in 5/10 (50%) of the included studies. All 10 studies received a confidence rating of “critically low” according to the AMSTAR 2 domain. There were no significant associations between incidence of spin type and year of publication, journal of publication, number of citations, LOE, funding, Clarivate impact factor, or ScopusCiteScore.ConclusionsSpin is highly prevalent in abstracts of systematic reviews and meta-analyses investigating ED versus MD. All ten manuscripts evaluated received a low quality rating according to the AMSTAR 2 domain.Keywords: Endoscopic, microscopic, spin, meta-analysis Highlight box Key findings • All included abstracts relating to endoscopic decompression (ED) versus microscopic decompression (MD) include at least one type of spin. • All included abstracts relating to ED versus MD are “critically low” quality according to AMSTAR 2 domain. What is known and what is new? • There is paucity of high-quality randomized control trials evaluating ED versus MD. • This is the first study of its kind that evaluates bias in systematic reviews and meta-analyses. What is the implication, and what should change now? • When surgeons read abstracts of systematic reviews and meta-analyses related to ED versus MD to make their clinical decisions, they should keep in mind of the high rate of bias and low quality of current existing abstracts. • A high standard of publication should be required for systematic reviews and meta-analyses to mitigate spin in future studies. Introduction Lumbar spinal stenosis (LSS), a common cause of radicular low back pain, occurs in approximately 11% of the population and is expected to continue increasing in prevalence as the population ages (1-3). It occurs secondary to degeneration of the intervertebral disc, facet joints, and ligamentum flavum, causing narrowing of the spinal canal and neural foramina, ultimately resulting in impingement on the spinal cord and nerve roots (2). Lumbar disc degeneration is often initiated by an imbalance between the catabolic and anabolic processes within the intervertebral disc (4-6). Once conservative management fails to improve symptoms, surgical management is indicated for refractory symptoms. Traditional open lumbar decompression with or without fusion is associated with increased morbidity due to soft tissue dissection and post-surgical instability due to removal of structures that stabilize the spinal column (7). Over recent decades, there has been a shift towards minimally invasive techniques, including the development of endoscopic decompression (ED) and microscopic decompression (MD). ED and MD minimize tissue trauma, postoperative pain, and recovery time relative to traditional open lumbar decompression (8). These advancements allow for smaller incisions while still providing direct visualization of the spinal canal (8). Both approaches offer their own distinct advantages. Multiple studies have reported a shorter hospital length and of stay and less intraoperative blood loss following ED compared to MD, with no difference in efficacy and safety (9,10). However, ED is associated with a steep learning curve and poses the risk for incomplete decompression (11). As physicians increasingly prioritize evidence-based medicine, staying informed on the most recent literature and guidelines is crucial. Surgeons often turn to meta-analyses and systematic reviews due to their ability to efficiently synthesize research findings. However, these studies are subject to bias, known as “spin”. Spin in scientific literature is the distortion of results by portraying an intervention more favorably than the evidence supports, often through selective inclusion of studies or reliance on studies with small sample sizes. This bias can be exacerbated when investigating novel techniques, such as ED and MD, where there is a lack of high-quality, large-scale randomized control trials available for analysis. To promote informed clinical decision-making and raise awareness of the risks of over-relying on abstract summaries, it is crucial to evaluate the prevalence of spin in study abstracts. The primary objective of this study is to identify and characterize the incidence of spin within abstracts of systematic reviews and meta-analyses comparing ED to MD in the treatment of degenerative lumbar stenosis. The secondary objective is to grade the quality of the manuscripts using the AMSTAR 2 criteria as proposed by Shea at al, which assesses the quality of systematic reviews, and investigate associations between spin incidence and study characteristics, such as journal of publication, level of evidence (LOE), funding, Clarivate impact factor, or ScopusCiteScore (12). Prior studies have shown a high prevalence of spin in other aspects of spine surgery (13-15). We hypothesize that spin will be present in >20% of abstracts of systematic reviews and meta-analyses. We present this article in accordance with the PRISMA reporting checklist (available at https://jss.amegroups.com/article/view/10.21037/jss-25-104/rc). Methods Articles were collected by two separate authors using the PubMed, Scopus, and Web of Science (WOS) databases using the following search term: (lumbar) AND (systematic review OR meta-analysis) AND (endoscopic decompression OR microscopic decompression). All search results were consolidated in EndNote X9 (Clarivate, Philadelphia, PA, USA). Following the removal of duplicates, articles were screened by two authors, with a third author resolving disputes prior to evaluating articles for spin. Training Two reviewers were trained to identify study designs and features indicative of the 15 most common types of spin, defined by Yavchitz et al., by using previously published and graded systematic reviews (16). These features were in accordance with AMSTAR 2 criteria and aimed at assessing the article’s sources of funding, presentation of results, number of authors involved in study selection and data extraction, assessment of bias, and several other factors (12). Inclusion criteria PubMed, Scopus, and WOS databases were searched for systematic reviews and meta-analyses comparing the outcomes of ED vs. MD in the treatment of degenerative LSS, from database inception to April 8, 2024. Included articles met the following criteria: (I) a systematic review with or without a meta-analysis; (II) degenerative etiology; (III) conducted only on human subjects; (IV) available in English. Data extraction After finalization of included studies, data extraction was performed independently by two reviewers. General study characteristics collected included study title, author, journal of publication, level of evidence (LOE), study design, funding, adherence to PRISMA and PROSPERO, 2024 Scopus CiteScore (Elsevier, Amsterdam, the Netherlands), and 2024 Clarivate impact factor for the studies’ corresponding journals. The Scopus Citescore (normalized to a top score of 100) and the Clarivate impact factor are both indicators of a journal’s influence, measuring how often articles from that journal are cited in other journals. LOE not explicitly stated in the articles was determined using the American Academy of Orthopedic Surgeons LOE Criteria (17). The abstracts for each included study were graded for the presence of each of the 15 most common types of spin defined by Yavchitz et al. (16). This study utilizes the same method proposed by Yavchitz et al., which involves a four-phase consensus process to classify and rank the severity of spin, including categories such as misleading reporting, misleading interpretation, and inappropriate extrapolation (16). Disagreements in gradings were discussed by both authors, and any discrepancies that could not be resolved were settled by adjudication with a third author. Each article’s full texts were then graded based on the AMSTAR 2 criteria with subsequent categorization into either “high”, “moderate”, “low” or “critically low” ratings. Statistical analysis Data was presented in a descriptive fashion. Using Fisher’s exact test, each category of spin was analyzed to determine if the presence of spin correlated significantly with year, journal, LOE, funding source or AMSTAR 2 rating. All statistical analysis was done using R (RStudio, PBC, Boston, MA, USA; Version 4.2.1). For all analyses, statistical significance was defined as an alpha of 0.05. Results The search resulted in 1,546 articles, with 696 removed as duplicates. Of the remaining articles, 10 met inclusion criteria (Figure 1) (9,10,18-25). The year of publication for the included articles ranged from 2020 to 2023. The number of citations ranged from 21 to 53, with an average of 35. The impact factor of each included study’s journal ranged from 1.4 to 3.8, with an average of 2.34. All 10 (100%) of the included systematic reviews had performed a meta-analysis and included clinical outcomes, results, or efficacy as the primary outcome measure. All included manuscripts contained a statement of adherence to PRISMA, while no studies reported PROSPERO registration. Mean 2024 Clarivate impact factor was 2.12 and mean 2024 Scopus CiteScore was 3.25. The PRISMA flow diagram for endoscopic decompression. All 10 included studies had at least one type of spin. Mean spin occurrence was 2.9 types per article, ranging from 1–5. Spin type 12 (conclusion claims equivalence or comparable effectiveness for non-statistically significant results with a wide confidence interval) and type 3 (selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention) were the most common forms of spin, found in 5/10 (50%) of the included studies. The next most common was type 4 (the conclusion claims safety based on non-statistically significant results with a wide confidence interval), found in 4/10 (40%) of included studies (Table 1). All 10 studies (100%) received a “critically low” quality rating according to AMSTAR 2, due to the presence of at least 1 critical flaw (Tables 2,3). Table 1Type and frequency of spin in analyzed studiesCategoryTypeSpin descriptionAbstracts (N=10), n [%]Misleading interpretation1The conclusion formulates recommendations for clinical practice not supported by the findingsN/A2The title claims or suggests a beneficial effect of the experimental intervention not supported by the findingsN/A4*The conclusion claims safety based on nonstatistically significant results with a wide confidence interval4 [40]9Conclusion claims the beneficial effect of the experimental treatment despite reporting bias3 [30]12*Conclusion claims equivalence or comparable effectiveness for nonstatistically significant results with a wide confidence interval5 [50]Misleading reporting3*Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention5 [50]5The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies3 [30]6Selective reporting of or overemphasis on harm outcomes or analysis favoring the safety of the experimental intervention2 [20]10Authors hide or do not present any conflict of interestN/A11Conclusion focuses selectively on statistically significant efficacy outcome4 [40]13Failure to specify the direction of the effect when it favors the control interventionN/A14Failure to report a wide confidence interval of estimates3 [30]Inappropriate extrapolation7The conclusion extrapolates the review findings to a different intervention (e.g., claiming efficacy of one specific intervention although the review covered a class of several interventions)N/A8Conclusion extrapolates the review’s findings from a surrogate marker or a specific outcome to the global improvement of the diseaseN/A15Conclusion extrapolates the review’s findings to a different population or settingN/A*, three most prevalent spin types. N/A, not applicable. Type and frequency of spin in analyzed studies *, three most prevalent spin types. N/A, not applicable. Table 2AMSTAR 2 assessment of analyzed studiesAMSTAR 2 domainCritical domainDeficient (N=10), n [%]1. Did the research questions and inclusion criteria for the review include the components of PICO?NoN/A2. Did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant deviations from the protocol?Yes10 [100]3. Did the review authors explain their selection of the study designs for inclusion in the review?NoN/A4. Did the review authors use a comprehensive literature search strategy?YesN/A5. Did the review authors perform study selection in duplicate?No2 [20]6. Did the review authors perform data extraction in duplicate?No1 [10]7. Did the review authors provide a list of excluded studies and justify the exclusions?Yes10 [100]8. Did the review authors describe the included studies in adequate detail?NoN/A9. Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?YesN/A10. Did the review authors report on the sources of funding for the studies included in the review?No7 [70]11. If meta-analysis was performed, did the review authors use appropriate methods for statistical combination of results?YesN/A12. If meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis?No1 [10]13. Did the review authors account for RoB in primary studies when interpreting/discussing the results of the review?Yes1 [10]14. Did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review?No2 [20]15. If they performed quantitative synthesis did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?Yes2 [20]16. Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?No2 [20]N/A, not applicable; PICO, Patient, Intervention, Comparison, Outcome. AMSTAR 2 assessment of analyzed studies N/A, not applicable; PICO, Patient, Intervention, Comparison, Outcome. Table 3Comprehensive study demographic, spin, and AMSTAR 2 characteristicsArticle No.JournalYearLevel of evidence Funding source†Adherence to PRISMA?Clarivate impact factor journal 2024?Scopus CiteScore 2024?Total sum of spinAMSTAR category‡1 Global Spine Journal 20224112.45302 Experimental and Therapeutic Medicine 20204212.71.9403 Asian Spine Journal 20233211.14.7104 Medicine (Baltimore 20213111.62.1305 European Journal of Orthopaedic Surgery and Traumatology 20204111.73.6406 World Neurosurgery 202031123.9107 Annals of Palliative Medicine 20214211.92408 Experimental and Therapeutic Medicine 20234212.81.9209 Neurospine 20224113.45.35010 Medicine 20234211.62.120†, 0= undeclared; 1= no external funding; 2= yes external funding. ‡, 3= high, 2= moderate, 1= low, 0= critically low. Comprehensive study demographic, spin, and AMSTAR 2 characteristics †, 0= undeclared; 1= no external funding; 2= yes external funding. ‡, 3= high, 2= moderate, 1= low, 0= critically low. Statistical analysis revealed no significant associations between incidence of spin type and year of publication, journal of publication, number of citations, LOE, funding, Clarivate impact factor, or ScopusCiteScore. Discussion In this study, we sought to identify the incidence of spin in systematic reviews and meta-analyses comparing ED to MD. Our analysis revealed that all 10 included studies exhibited at least one type of spin. Mean spin occurrence was found to be 2.9±1.3 types per article, with a range of 1–5. The most prevalent types of spin were type 3 and type 12, both appearing in 5/10 (50.0%) of studies. Furthermore, all 10 studies received a “critically low” quality rating according to AMSTAR 2. Notably, while 100% of the studies claimed adherence to PRISMA guidelines, none reported PROSPERO registration. One of the most frequently reported spin types was type 3, characterized by the “Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention.” For instance, in a meta-analysis, the discussion highlighted that ED “resulted in better pain control in the early postoperative period, both in the lower back and legs, as well as shorter operative and shorter hospitalization times.” (22). However, the results showed mixed findings in terms of Oswestry Disability Index (ODI) scores, with ED having superior ODI scores at 12 months but inferior scores at 6 months compared to the control, a finding not reflected in the conclusion. This selective emphasis on favorable outcomes of the experimental intervention may falsely exaggerate the benefits of ED compared to other surgical methods. Similarly, another study concluded that ED “is a better treatment for lumbar spinal stenosis, showing more effective leg pain improvement, shorter operative time, and fewer complications than microscopic decompression.” (24). However, the study also measured blood loss, which was not significantly lower in ED, a finding omitted from the conclusion. Such spin in abstracts can influence physician decision-making by overstating the positive outcomes of interventions or underemphasizing drawbacks. The other most commonly reported spin type was type 12, where the “Conclusion claims equivalence or comparable effectiveness for non-statistically significant results with a wide confidence interval.” For example, one study concluded that ED had comparable effectiveness to the control based on a non-significant difference in ODI (20). Another study stated that ED is a viable alternative minimally invasive surgery with no significant difference in efficacy and safety in terms of ODI and postoperative complications (9). A recent review found this trend was common, with many studies lacking justification for non-significant findings and failing to interpret confidence intervals properly. As a result, articles simply stated the null findings without interpretation and then focused their discussion and analysis solely on statistically significant outcomes (26). This type of spin is concerning because null findings should be interpreted, analyzed, and justified as rigorously as significant findings to ensure the validity of results. Our notable finding that all 10 (100%) studies were graded as “critically low” quality according to AMSTAR 2 may reflect the current state of ED literature. All 10 articles had at least 2 critical flaws due to poor adherence to criteria 2 (“Did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review, and did the report justify any significant deviations from the protocol?”) and criteria 7 (“Did the review authors provide a list of excluded studies and justify the exclusions?”). The poor adherence to these two criteria is concerning because they both involve the study selection process. This lack of transparency is problematic as it obscures the impact of study selection on the overall findings of the systematic review or meta-analysis. Such deficiencies hinder readers from properly assessing the validity of the conclusions. Spin is not only prevalent in endoscopic literature but has also been documented in other areas of spine surgery. For instance, one study found spin in all 34 included spine surgery manuscripts comparing outcomes of lumbar microdiscectomies with other lumbar disc herniation interventions (27). Spin has been documented in other orthopaedic subspecialties as well. Kim et al. reported spin in 17 abstracts related to superior capsular reconstruction, finding at least one form of spin in all included abstracts (28). Spin has additionally been identified in non-orthopaedic fields. Nowlin et al. found spin in 76 out of 200 systematic reviews related to melanoma therapy (29). Jellison et al conducted a study of 116 abstracts in the field of psychiatry, of which 65 (56%) reported spin (30). These findings are consistent with our study, suggesting that spin is a widespread issue across all medical literature. It is particularly concerning that, despite substantial evidence documenting its prevalence across orthopaedics and other medical specialties, the significant prevalence of spin persists. Spin in medical literature can arise from several factors. The pressure on researchers to produce a high volume of publications may lead to the presentation of findings in a more favorable light to increase their chances of journal acceptance (31,32). Additionally, word count restrictions in abstracts may result in the omission of details that do not align with other significant findings. Conflicts of interest or specific funding sources may further influence authors to frame their conclusions in a favorable manner to maintain funding (31). The authors of this study have proposed methods to mitigate spin bias. For instance, journals could implement strict adherence to reporting guidelines such as PRISMA. While the current study did not find an association between spin and adherence to PRISMA guidelines, this association has been demonstrated previously (33). Therefore, broader implementation of PRISMA guidelines, along with the inclusion of measures specifically targeting spin, may help reduce spin bias rates. Furthermore, granting journals access to the study’s statistical analysis and raw data could be an additional deterrent against spin. Training journal editors to recognize spin bias and mandating that authors revise manuscripts to address identified issues before acceptance are also crucial steps in decreasing spin within published literature. Furthermore, fostering a research culture that emphasizes transparency, quality, and integrity over sheer quantity of publications is essential. Encouraging cautious and evidence-based claims does not necessarily diminish reader engagement, as demonstrated by a randomized trial showing that research with restrained conclusions regarding correlational findings still maintained reader interest (34). Such initiatives collectively aim to enhance the reliability and impact of scientific literature. This study has several limitations. First, the relatively small number of included articles limits the power of our analysis and may have prevented the identification of additional associations. Since there are few systematic reviews comparing ED and MD, the generalizability of our study is limited. The small sample size may have caused over-reporting of spin on this specific topic. Second, the grading of spin is inherently subjective where specific wording used by authors may influence result interpretation. This inherent subjectivity of spin bias combined with authors being trained with the same papers further increases the potential of over-reporting spin. This was mitigated by involving two reviewers with a third for arbitration and adopting a review process similar to those of previous spin studies in the literature. Finally, this study only reviewed abstracts for spin, which may not fully capture the extent of spin in the full manuscripts. However, this approach reflects the tendency of physicians to base clinical decision-making on abstracts, especially when limited time in clinical settings precludes the review of full articles for a higher provision of evidence (35-37). Overall, this study revealed the presence of spin bias in all included systematic reviews and meta-analyses comparing ED versus MD. With a limited sample size, our study provides an initial insight into the prevalence of spin bias within this topic rather than definitive evidence of systemic spin. However, as more literature is published regarding ED and MD, the authors implore readers to remain vigilant for spin bias when reading systematic reviews and to analyze the full manuscript prior to making clinical decisions. Other studies have shown that spin in medical literature commonly skews results to make the experimental treatment appear more favorable, potentially causing physicians to misinterpret findings which may, in turn, impact clinical decision making and outcomes (35,38). The goal is to increase physician awareness of spin, elevate publication standards, and create a body of scientific literature that supports sound clinical decision-making and improves patient outcomes. Conclusions Spin is highly prevalent in abstracts of systematic reviews and meta-analyses investigating ED versus MD. All abstracts received a “critically low” quality rating according to the AMSTAR 2 domains. Surgeons should be mindful when using current literature related to ED versus MD when making clinical decisions about which approach to use. Supplementary The article’s supplementary files as Acknowledgments None. Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Footnotes Reporting Checklist: The authors have completed the PRISMA reporting checklist. Available at https://jss.amegroups.com/article/view/10.21037/jss-25-104/rc Funding: None. Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jss.amegroups.com/article/view/10.21037/jss-25-104/coif). J.C.W. serves as an unpaid editorial board member of Journal of Spine Surgery from April 2024 to December 2026, and has received intellectual property royalties from Zimmer Biomet, NovApproach, SeaSpine, GS Medical, and DePuy Synthes; consulting fees from Depuy Synthes and Bioretec; payment for expert testimony; and stock options in Bone Biologics, Electrocore, PearlDiver, Surgitech, and Illuminant. R.J.H. has received grant funding from ATEC and NuVasive; consulting fees from SI bone, Globus/NuVasive, Medtronic, Orthofix, Cerapedics, and Evolution Spine; and support from the North American Spine Society to attend meetings. R.K.A. has received grants from the NIH; consulting fees from Medtronic, Orthofix, and Globus; payment from Eccential Robotics for lectures and presentations; payment for med-legal consulting; travel support from USC; stock options in HIA Technologies, NeoOnc, and Globus; and research support from Max BioPharma and ATEC. The other authors have no conflicts of interest to declare. References Available at https://jss.amegroups.com/article/view/10.21037/jss-25-104/prf Supplementary Materials The article’s supplementary files as