Trial trustworthiness assessment: commentary on the Cochrane 2025 systematic review on calcium supplementation during pregnancy Authors: Maria Regina Torloni, Gabriela Cormick, Alvaro Nagib Atallah Published: 2026 ABSTRACT AbstractBackgroundHypertensive disorders of pregnancy remain a major cause of maternal and perinatal morbidity and mortality worldwide. Calcium supplementation during pregnancy is recommended by guidelines and the World Health Organization to prevent preeclampsia largely based on successive versions of a Cochrane systematic review (SR). However, the 2025 update of this SR reported that calcium supplementation may result in little to no reduction in preeclampsia risk, a conclusion that diverges from all previous versions. This change was driven primarily by the exclusion from meta-analyses of several previously included trials due to concerns about study trustworthiness.DiscussionWe critically examine trustworthiness assessments in the 2025 SR and identify inconsistencies in how criteria were interpreted and applied across trials. Using two trials as illustrative examples, we show that concerns related to ethics and effect size were applied inconsistently. We further demonstrate that the same trials were deemed trustworthy and included in another Cochrane SR updated in 2024, on calcium supplementation during pregnancy for related outcomes. Sensitivity analyses illustrate how the inclusion of these two excluded trials would significantly change the pooled effect estimates for preeclampsia in the 2025 SR. We argue that reliance on unvalidated trial trustworthiness assessment tools risks subjective exclusion of evidence, potentially impacting guidelines and policies. We recommend that the review authors reassess all trials excluded from the meta-analyses due to trustworthiness issues and conduct sensitivity analyses to transparently assess their impact on the 2025 SR conclusions and related policy recommendations.Keywords: Systematic review, Randomized trial, Trustworthiness assessment, Policy, Calcium supplementation, Preeclampsia, Cochrane Introduction Hypertensive disorders of pregnancy remain an important cause of maternal and perinatal morbidity and mortality in the 21st century, especially in low and middle-income countries (LMIC) [1, 2]. Obstetric guidelines and the World Health Organization have issued recommendations on calcium supplementation during pregnancy to prevent hypertensive disorders of pregnancy [3–6] largely based on a Cochrane systematic review (SR) first published in 2000 and subsequently updated five times up to 2018. In the sixth update of this SR, published in 2025 [7], the new author team reports that calcium supplementation does not reduce pre-eclampsia, reversing conclusions from prior versions of the same review. Notably, this latest version did not include any new trials comparing calcium supplementation versus placebo but changed the review protocol, resulting in the exclusion of 20 trials that had been included in previous versions: 11 were excluded due to changes in eligibility criteria, and 9 were placed in an “awaiting classification” category due to concerns about the trustworthiness of these trials—a new assessment introduced in this SR update. The exclusion of these trials from all meta-analyses substantially changed the main results and conclusions of the 2025 SR. The change in conclusions in the 2025 SR could have an important impact on reproductive health, affecting existing antenatal care recommendations, calcium supplementation programs, and maternal health policies aimed at preventing hypertensive disorders of pregnancy, especially in LMIC. While we commend the authors of the 2025 SR for their extensive work, we have methodological concerns regarding the process and consistency of trustworthiness assessments applied across trials, which may have important implications for the results of this review and policy recommendations. Inconsistencies in assessment of trial trustworthiness in the 2025 systematic review We present below two examples of inconsistencies in the interpretation of domains of the trustworthiness checklist across trials included in the 2025 SR on calcium supplementation to prevent hypertensive disorders of pregnancy. Trustworthiness assessment of Lopez-Jaramillo 1997 As described in the Supplementary material 4 of the 2025 Cochrane SR [7], this trial which involved 260 pregnant women was categorized as “awaiting classification” and excluded from all meta-analyses because the review authors flagged three trustworthiness concerns: (a)“insufficient information on ethics”, (b)“a very high rate of preeclampsia in both groups of healthy women (4/125 calcium group; 21/135 placebo group)”, and (c)“a very large effect size in the reduction of this outcome (RR 0.21; 95% CI 0.07 to 0.58)”. The review authors reported contacting the trial authors but receiving no response prior to review submission. With respect to ethics, the original publication by Lopez-Jaramillo et al. [8] states that participants were included in the study “after written consent was given” and that “the study protocol was approved by the Instituto Superior de Investigaciones, Faculdade de Ciencias Medicas”. This type of ethics reporting is comparable to that of other trials included in the 2025 SR, which were not flagged for ethical concerns [9, 10]. For instance, although the Villar 1990 [10] publication provided ethics information identical in nature to that of Lopez-Jaramillo 1997 (obtention of written informed consent and approval by a local ethics committee), and although the review authors also reported (in Supplementary material 2) unsuccessful attempts to contact the Villar trial authors, this did not appear to raise comparable concerns regarding “insufficient information on ethics”. Regarding the second trustworthiness concern, the incidence of preeclampsia in the calcium group was 3.2%, which cannot reasonably be considered “very high”. Although the incidence in the placebo group (15.5%) is higher than average for a general obstetric population, this trial enrolled a very high-risk cohort from Ecuador: all participants were nulliparous, younger than 17.5 years, had low dietary calcium intake (mean 610 mg/day), and lived at an altitude of 2,800 m—an additional risk factor for preeclampsia [11, 12]. In this context, a 15.5% incidence of preeclampsia in the placebo group appears plausible. Finally, the reported average relative risk of 0.21 is less extreme than that reported in the Villar 1990 trial [10] (RR 0.14, 95% CI 0.01 to 2.67) which was included in the 2025 SR without any trustworthiness concerns being raised by the review authors about its effect size. Trustworthiness assessment of Purwar 1996 The Purwar 1996 trial [13] was also placed in the “awaiting classification” category and excluded from all meta-analyses in the 2025 SR [7] because of three trustworthiness concerns: (a) “insufficient information on ethics”, (b) “insufficient information on randomization”, and (c) “a very large effect size in the reduction of preeclampsia (RR 0.17; 95% CI 0.04 to 0.77)”. As in the previous case, the review authors emailed the trial authors but did not receive a response before submission [7]. However, the Purwar 1996 publication [13] explicitly states that “the research project was approved by the hospital´s ethical committee and all subjects gave written informed consent”, indicating adherence to basic ethical standards similar to those reported in other trials included in the 2025 SR [9, 10]. Regarding randomization, the Purwar 1996 article reports that “women were assigned randomly in a double blind fashion at 20 weeks´ gestation to 1 of 2 treatment groups (calcium/placebo) using computer generated random number table”. While more detailed reporting would have been desirable, this limitation pertains to risk-of-bias assessment rather than trial trustworthiness per se. Finally, the reported effect size (RR 0.17) falls within the range of point estimates of the six trials included in the 2025 SR preeclampsia meta-analysis, which ranged from 0.14 to 0.94 [7]. These two examples illustrate discrepancies in the interpretation and application of trustworthiness criteria across trials, potentially reflecting arbitrary or unclear decision-making by review authors. Additional evidence of inconsistent trustworthiness assessments Additional evidence of inconsistency in trial integrity assessments comes from another recently updated Cochrane SR on calcium supplementation during pregnancy for other outcomes by Kongwattanakul et al.[14] In that 2024 SR, the two aforementioned trials by Lopez-Jaramillo 1997 [8] and Purwar 1996 [13] were assessed as trustworthy and included in all meta-analyses. Moreover, two additional trials placed in the “awaiting classification” category in the Cluver 2025 Cochrane SR [7] due to trustworthiness concerns (Lopez-Jaramillo 1989 [15] and Sanchez-Ramos 1994 [16]) were also deemed trustworthy and included in the meta-analyses of the Kongwattanakul 2024 Cochrane SR [14]. The fact that a trial can be classified as trustworthy in one Cochrane systematic review but untrustworthy in another on the same topic highlights the influence of reviewer judgement and tools used. Notably, the trustworthiness tools used in the two Cochrane reviews differed. While the Cluver 2025 SR [7] used the Trustworthiness in RAndomised Controlled Trials (TRACT) checklist [17] which includes 23 questions (7 related to governance, 10 to plausibility, and 6 to reporting aspects [18]), the Kongwattanakul 2024 SR [14] used the Trustworthiness Screening Tool (TST) developed by the Cochrane Pregnancy and Childbirth Group [19] which includes 11 questions (4 related to governance, 5 related to plausibility, and 2 related to reporting issues [18]). Thus, the discrepant classification of the same trials in two different Cochrane SRs could be due to two factors: (a) the inherent subjectivity of different review author teams when judging the same trial and (b) the use of different, unvalidated assessment tools. Taken together, these findings highlight the subjectivity inherent in current trial trustworthiness assessments and the risks associated with relying on unvalidated tools to determine the inclusion of studies in meta-analyses. Suggestions for the 2025 Cochrane systematic review The process of undertaking a systematic review involves a sequence of decisions. While many of these decisions are clearly objective and non-contentious, some are somewhat arbitrary or unclear [20]. Pending the availability of scientifically tested and validated tools for assessing trial trustworthiness, we propose two actions for the authors of the 2025 SR: (a) first, reassess all trials currently classified as “awaiting classification” due to trustworthiness concerns using consistent and transparent criteria across studies; and (b) then, conduct sensitivity analyses including all trials that remain in the “awaiting classification” category due to “trustworthiness concerns”. The use of sensitivity analyses in dealing with problematic studies is currently recommended by Cochrane editorial policies [21]. This approach would allow readers—including policymakers—to better understand the impact of including or excluding these studies on effect estimates for key outcomes, and decide whether existing recommendations require revision. As an illustration, Fig. 1 presents the results of a sensitivity analysis that includes the two aforementioned trials in the 2025 SR preeclampsia meta-analysis. Inclusion of these trials would change the current pooled effect estimate from 0.83 (95% CI 0.67 to 1.04; 6 studies; 15,364 women, Fig. 1a) to 0.63 (95% CI 0.45 to 0.90; 8 studies; 15,814 women, Fig. 1b). Sensitivity analysis on calcium supplementation during pregnancy. Outcome: preeclampsia. a Preeclampsia forest plot of the 2025 Cochrane systematic review. b Preeclampsia forest plot including Lopez-Jaramillo 1997* and Purwar 1996*. *Studies currently categorized as awaiting classification in the 2025 systematic review due to trustworthiness concerns Clinicians and program implementers may also be confused by the contradiction between prior versions of the review and the updated one, particularly if the methodological rationale for excluding trials from the meta-analyses is not clearly explained. Therefore, we also suggest that in the discussion section of the 2025 SR, the review authors emphasize that changes in the results of this SR compared to previous versions were due to a methodological reinterpretation, not to empirical results from new trials comparing calcium supplementation versus placebo, and further discuss its implications for recommendations and policy implementation. Concluding remarks Trial trustworthiness is an important and contemporary topic. However, there is currently no consensus on what constitutes a “problematic study” and none of the many existing methods, tools, or checklists for assessing trial trustworthiness have been scientifically tested or validated [18, 22–24]. As noted by experts and by Cochrane´s Senior Integrity Editor, the use of unvalidated tools carries risks and could lead to incorrectly labelling studies as problematic [24, 25]. Moreover, since the routine examination of the trustworthiness of randomized trials is relatively new, most SR authors will have limited experience and expertise in this area [24]. These circumstances could seriously affect evidence syntheses and impact guideline developers and policymakers who use these reviews to inform recommendations. The issue is particularly relevant for policymakers and global organizations that issue norms and standards. These stakeholders must remain vigilant to the potential for such tools to distort the evidence base and inadvertently undermine the reliability of their recommendations. In the current era of unprecedented proliferation of scientific studies, evidence syntheses still remain the cornerstone of health guidelines and policies. Systematic reviews must balance between internal validity, representativeness, and transparency. While there is a clear need to protect SR conclusions from distortion by trials with serious problems, the exclusion of trials on the basis of unvalidated or inconsistently applied criteria also poses risks. In this context, the need for robust, transparent, and validated tools to support reliable and methodologically sound syntheses cannot be overstated. Strengthening the rigor of evaluative methods to assess trial integrity is essential to safeguard the credibility of the evidence base and the trustworthiness of the decisions derived from it. The responsibility for reliably judging the trustworthiness of a trial does not rest solely with SR authors, who typically act in good faith. It is also influenced by the inherent difficulties and subjectivity involved in evaluating complex and often poorly reported trial characteristics using tools that lack clear guidance and formal validation. We therefore join others in the call for researchers to undertake validation studies of existing trial trustworthiness assessment tools to determine which tools are credible, reliable, and useful for review authors. Future research should also systematically compare these tools in terms of completeness, usability, and psychometric properties, including how consistently different reviewers (inter-rater reliability) and the same reviewer at different times (intra-rater reliability) apply the tools. Such efforts will promote more consistent and transparent decision-making by review authors and contribute to the production of more “trustworthy systematic reviews” that can better inform public health policies and clinical practice guidelines. Ensuring that the results of the 2025 Cochrane review are valid, totally inclusive, and transparently and clearly reported, goes beyond a question of evidence synthesis methodology. The results and conclusions of this systematic review should be carefully considered because of their direct implications for guideline developers, maternal health programs and global policy recommendations which may impact millions of women worldwide, especially in LMICs. Acknowledgements None. Abbreviations Confidence interval Low- and middle-income countries Pre-eclampsia Relative risk Systematic review Trustworthiness in RAndomised Controlled Trials checklist Trustworthiness Screening Tool Authors’ contributions MRT: conceptualization, data curation, formal analysis, writing original draft. GC: conceptualization, formal analysis, writing-review and editing. ANA: conceptualization, formal analysis, writing-review and editing. MRT, GC and ANA approved the final version to be published and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Funding This commentary did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Data availability All datasets supporting the conclusions of this article are included within the article or available in the cited publications. Declarations Ethics approval and consent to participate Not applicable. Consent for publication Not applicable. Competing interests Alvaro Nagib Atallah is an author of the first Cochrane systematic review (published in 2000) and participated in five of the updates of this review (published in 2002, 2006, 2010, 2014 and 2018). Maria Regina Torloni is a co-author of the 2014 and 2018 updates of this systematic review. Gabriela Cormick has no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Availability Statement All datasets supporting the conclusions of this article are included within the article or available in the cited publications.