Evaluating search approaches for a network meta-analysis: a case study of Epistemonikos Database of Trials, CENTRAL, and other sources Authors: Irma Klerings, Andreea Dobrescu, Barbara Nussbaumer-Streit, Amin Sharifan, Camilla Neubauer-Bruckner, Gerald Gartlehner Published: 2026 ABSTRACT AbstractBackgroundMEDLINE, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL), and reference lists are commonly used when searching randomized controlled trials (RCTs) for inclusion in a systematic review. Recently, a new database for RCTs, Epistemonikos Database of Trials (EDT), was launched in a pilot stage. This case study aimed to compare EDT, MEDLINE, Embase, CENTRAL, reference list checking, and database combinations for the identification of RCTs about pharmacological migraine treatments and to assess the impact of missed trials on network meta-analyses (NMA).MethodsWe assessed the following: (1) the performance (recall, precision, number needed to read [NNR], F1- and F2-scores) of database searching, reference list checking, and database combinations compared to a relative recall reference standard, (2) the impact of differing search results on the findings of NMA (overall network estimates, certainty of evidence ratings) based on RCTs with low/moderate risk of bias, and (3) the suitability of EDT for systematic searching.ResultsRecall was high for all assessed search approaches (98.3–99.4% for single-database searches without date limit, 99.4–100% for database combinations without date limit, and 93.3% for reference list checking of older systematic reviews combined with publication-date-limited database searching). However, NNR and F-Scores varied widely, ranging from an NNR of six and F2 of 0.5 for EDT alone to an NNR of 16 and F2 of 0.2 for the combination of reference list checking and database searching. Different search results led to similar network estimates and identical certainty-of-evidence assessments as the reference standard. While the search, export, and documentation functions of EDT were less sophisticated than other assessed databases/platforms, they were sufficient to conduct reproducible and comprehensive systematic searches.ConclusionsAll search approaches we tested were acceptable for an NMA of pharmacological migraine treatments; high search recall led to similar NMA results. However, the associated number of references to be screened at title/abstract level differed greatly. Specialized RCT databases (CENTRAL, EDT) were more efficient in this regard than general healthcare research databases (MEDLINE, Embase). Additionally, the pilot version of EDT proved to be a promising cost-free option for RCT searching.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12874-026-02870-9.Keywords: Systematic search methods, Identification of randomized controlled trials, Search performance, Network meta-analysis as topic, Epistemonikos Database of Trials Background Systematic literature searching is a methodological foundation of systematic evidence synthesis. The search process aims to be as comprehensive as possible while ensuring that the number of records retrieved remains within the review team’s time and budgetary resources for systematic study selection. For this reason, it is important to select appropriate information sources (i.e., databases and additional search methods) to answer the research question and review type and tailor database search strategies to achieve an appropriate balance of recall (sensitivity) and precision [1–3]. In addition to established sources in this field, the Epistemonikos Foundation has recently released a new potential source for systematic searching, the Epistemonikos Database of Trials (EDT) [4]. When planning a systematic search for randomized controlled trials (RCTs), the requirements of the Methodological Expectations of Cochrane Intervention Reviews (MECIR) [5] provide guidance for search source selection that is likely to reliably achieve a high recall: the combination of MEDLINE, Embase, Cochrane Central Register of Controlled Trials (CENTRAL), and reference lists of included studies and relevant systematic reviews. These databases are often considered the standard for comprehensive RCT identification [3], and each has been shown to have a high coverage of healthcare RCTs [6–9]. Appropriate search strategies in any of these databases will likely identify the majority of eligible RCTs, but to achieve the highest possible recall, a combination of databases and supplementary search methods (e.g., reference-list checking) is usually necessary [7, 10, 11]. When conducting a systematic search, the specific characteristics of all search methods (i.e., the databases, platforms, and supplementary search methods used) need to be considered. MEDLINE and Embase contain a vast variety of study types, making it necessary to add an RCT search filter to the search strategy. This filter should be validated; that is, its recall and precision for identifying RCTs must be known, as using a low-recall filter could undermine the performance of an otherwise sensitive search strategy [12]. CENTRAL, on the other hand, is a database for RCTs, making the application of a study design filter unnecessary, which avoids a reduction in sensitivity [1]. CENTRAL contains RCTs identified by a variety of sources, including MEDLINE (via PubMed) and Embase (via Embase.com) [13]. The Epistemonikos Database of Trials (EDT) [4] is a new bibliographic database currently in its pilot stage and designed to be “a free, comprehensive repository of RCTs in health“ [14]. While EDT is a stand-alone database, it also serves as one of the evidence repositories for Epistemonikos Sustainable Knowledge Platform (SKP) [15], a web-based platform for evidence synthesis production. EDT includes records from more than 30 electronic sources (databases, preprint servers, and trial registries) which are searched regularly and are classified automatically using the Epistemonikos Evidence Taxonomy (EET), a controlled vocabulary [14, 16]. The search interface also includes an “Automated Boolean Composer,” which provides pre-formulated search strategies for single search concepts that the user can also edit [14]. These search strategies are available for all EET concepts except very broad terms, and are developed and validated by information specialists using included studies of relevant systematic reviews as strategy development and validation sets [17]. So far, there have been no systematic evaluations of this new database concerning its suitability for systematic searching. As it is currently in its pilot stage, its functionalities may also be subject to change. Beyond database searching, backward citation searching (e.g., manual reference-list checking) is a requirement of MECIR [5] and a standard supplementary search method [18]. This search approach includes reviewing both the reference lists of trials newly identified by database searching, and of existing systematic reviews, either manually (i.e., reviewers scan the reference list and/or tables of included studies for potentially relevant titles) or by using an electronic citation index. Consequently, when authors update older systematic reviews that they did not originally conduct themselves, e.g., as part of a rapid review process, reference-list checking can be considered the primary search approach of the review [19]. While the recall of the established databases MEDLINE, Embase, and CENTRAL and citation searching has been evaluated in several studies, [7, 9, 11, 14, 20–23] no evaluation of EDT is available yet. Additionally, existing studies rarely assess the impact of search recall on the findings of evidence syntheses. Building upon our previous research [7], we assessed the suitability of different methods for the identification of RCTs and their impact on an SR and NMA of pharmacologic treatments of acute episodic migraine. Specifically, we compared search approaches using EDT, MEDLINE, CENTRAL, Embase, and reference list checking to a relative recall reference standard. Methods Objectives This study compared different approaches to systematic searching for RCTs of pharmacologic migraine treatments to a relative recall reference standard (a pooled set of relevant documents retrieved by multiple search approaches) [24]: Single-database searches of EDT, MEDLINE, Embase, or CENTRAL. Single-database searches of EDT, MEDLINE, Embase, or CENTRAL. Database combinations: MEDLINE, Embase, and CENTRAL (the MECIR-required minimum); any two-database combination. Database combinations: MEDLINE, Embase, and CENTRAL (the MECIR-required minimum); any two-database combination. The systematic search of a published systematic review, which used reference list checking of preexisting systematic reviews as the primary search approach and database searches limited by publication date as secondary searches [25]. The systematic search of a published systematic review, which used reference list checking of preexisting systematic reviews as the primary search approach and database searches limited by publication date as secondary searches [25]. It addresses the following questions: Compared to a pooled set of relevant publications, what proportion of relevant records and trials is missed by each approach? Compared to a pooled set of relevant publications, what proportion of relevant records and trials is missed by each approach? What is the impact of each approach on the title/abstract screening workload as represented by the number of references retrieved by this search approach? What is the impact of each approach on the title/abstract screening workload as represented by the number of references retrieved by this search approach? How do missed trials impact overall network estimates and corresponding certainty-of-evidence (CoE) ratings? How do missed trials impact overall network estimates and corresponding certainty-of-evidence (CoE) ratings? Is EDT suitable for systematic searching? Is EDT suitable for systematic searching? This case study is part of a larger methods project assessing the use of the Epistemonikos SKP for conducting a systematic review of pharmacological RCTs. The protocol for the entire project has been published on Open Science Framework [26] (Registration/DOI: 10.17605/OSF.IO/BJFD3). The study built on data from a systematic review (SR) and NMA on pharmacologic treatments for episodic migraine, which included 21 interventions [25]. It used the same eligibility criteria as the SR and focused on the following outcomes: pain freedom at two hours, pain relief at two hours, sustained pain freedom and sustained pain relief up to 48 h, and risk of serious adverse events. Search methods All database searches were based on eligibility criteria of the original SR [25]. Only English-language articles published in a scientific journal were eligible. Both the original searches and those conducted for this case study were designed and conducted by an experienced information specialist (IK). The original SR search process Because of the large scope of the review, the systematic search followed a three-step approach. First, we searched for relevant SRs of placebo-controlled and head-to-head trials of migraine medications. Seventeen recent SRs were identified and used for backward citation searching (manual reference-list checking) to identify eligible RCTs. In the second step, we conducted database searches in Ovid MEDLINE, CENTRAL (Cochrane Library/Wiley), and Embase.com (Elsevier). These searches were limited by publication date to identify only newer RCTs: As different SRs had assessed different interventions, we applied different start dates for sets of interventions depending on the recency of the previous reviews: 2013, 2014, 2015, 2020, and 2021. Fourteen additional reviews identified during the database searches were also used for further reference list checking. The documentation of the database searches is available in the original publication [25], and the full list of SRs used for manual citation screening is reported in Additional File 2. Results from the database searches were screened in two steps: dual screening of both abstracts and full text using DistillerSR [27]. Reference lists were checked manually at the title level by one person, and the potentially eligible studies were then added to the dual abstract and full-text screening process. Additional database searching for this study For this study, we re-ran the database searches without a date limit and conducted an EDT search on January 8, 2025. For the EDT search strategy, we translated the concepts used in other search strategies but used EDT-specific functionalities. We replaced subject headings with EET terms, and in the free-text search, we used the “Automated Boolean Composer” for all available concepts (population, interventions) without any manual adjustments to the search terms proposed by the system. To fit the eligibility criteria of the original SR, searches in MEDLINE and Embase included RCT filters, CENTRAL and EDT were limited to published articles only, and all databases except EDT were limited to English only. See Additional File 2 for the complete documentation of the search process. To arrive at the final deduplicated search results for the various database combinations, we used the Systematic Review Accelerator Deduplicator (algorithm: focused) [28]. Reference standard To create a reference standard of records that should be identified by the search approaches, we used a relative recall approach and pooled the relevant records retrieved from different sources: all eligible records identified by manual reference list checking, date limited searches of MEDLINE, Embase and CENTRAL, and by the EDT search without date limit. The relative recall approach was originally developed for search filter validation [24] and is commonly used in evaluations of the search performance of databases or database combinations for systematic searching [7, 10, 11, 20, 22] as it provides a single “gold standard” against which the performance of different search approaches can be assessed: The “relative recall” of a search approach is the proportion of relevant documents found by this approach, compared to the pooled relevant documents (e.g., included studies) retrieved by several search approaches working as a composite (e.g., all included studies of one or several systematic reviews, which used several search approaches in combination). To improve readability, we use “recall” throughout the paper to refer to the concept of relative recall based on the pooled relevant documents of multiple search approaches. Our reference standard of records for the evaluation of search performance is a set of 180 publications. The publications reported the results of 189 trials, and 174 trials were assessed as having a low or moderate risk of bias (RoB). They were the reference standard for evaluating the impact on the NMA. The list of publications and available database accession numbers is presented in Additional File 1. It also shows whether the record was initially identified by EDT or the original search process and what trial(s) it was associated with. Evaluation of search performance We assessed the relative database coverage, recall, precision, NNR, F1 score, and F2 score (see Table 1) for the following search approaches compared to the reference standard: Single-database searches without date limits: EDT, MEDLINE, Embase, CENTRAL. Single-database searches without date limits: EDT, MEDLINE, Embase, CENTRAL. Database combinations without date limits: MEDLINE+Embase+CENTRAL, CENTRAL + EDT, Embase+CENTRAL, Embase + EDT, MEDLINE+CENTRAL, MEDLINE + EDT, MEDLINE+Embase. Database combinations without date limits: MEDLINE+Embase+CENTRAL, CENTRAL + EDT, Embase+CENTRAL, Embase + EDT, MEDLINE+CENTRAL, MEDLINE + EDT, MEDLINE+Embase. Original SR search approach: A stepwise search for SRs and RCTs, using reference list checking, MEDLINE, Embase, and CENTRAL. Original SR search approach: A stepwise search for SRs and RCTs, using reference list checking, MEDLINE, Embase, and CENTRAL. Table 1Definitions of outcomes for search performance and impact of missed trialsDefinitions of OutcomesChanges in NMA results: Differences in effect estimates and 95% CI for analyses conducted using data from different search strategies.Changes in CoE assessments: Assessment of confidence for each outcome based on the references identified by each search strategy. For the original search, we used the assessments reported in the original review.Database coverage: The proportion of eligible records that are available in a database at the time of searching.F1 score: A metric combining precision and recall using their harmonic mean:F2 score: A metric combining precision and recall that gives recall twice the weight of precision:Number needed to read (NNR): The number of records that need to be screened to find one relevant article:Precision (= positive predictive value): The proportion of relevant records among those identified by a search:Recall (= sensitivity): The proportion of all eligible records correctly identified by a search:× 100FN (false negative): The number of records incorrectly classified as irrelevant in the searchFP (false positives): The number of records incorrectly classified as relevant by the search TN (true negatives): The number of records correctly classified as irrelevantTP (true positives): The number of records correctly classified as relevantAbbreviations:CIs confidence intervals, CoE certainty of evidence Definitions of outcomes for search performance and impact of missed trials Changes in NMA results: Differences in effect estimates and 95% CI for analyses conducted using data from different search strategies. Changes in CoE assessments: Assessment of confidence for each outcome based on the references identified by each search strategy. For the original search, we used the assessments reported in the original review. Database coverage: The proportion of eligible records that are available in a database at the time of searching. F1 score: A metric combining precision and recall using their harmonic mean: F2 score: A metric combining precision and recall that gives recall twice the weight of precision: Number needed to read (NNR): The number of records that need to be screened to find one relevant article: Precision (= positive predictive value): The proportion of relevant records among those identified by a search: Recall (= sensitivity): The proportion of all eligible records correctly identified by a search:× 100 FN (false negative): The number of records incorrectly classified as irrelevant in the search FP (false positives): The number of records incorrectly classified as relevant by the search TN (true negatives): The number of records correctly classified as irrelevant TP (true positives): The number of records correctly classified as relevant Abbreviations:CIs confidence intervals, CoE certainty of evidence To arrive at comparable values, we calculated recall based on the total eligible records, regardless of database coverage. For each eligible record missed by one of the database searches, we also assessed the reason for non-retrieval (e.g., due to the search strategy or due to non-availability in the database). Impact of missed trials To assess the impact of trials missed by different search approaches, we evaluated changes in the NMA results and CoE assessments (Table 1). While search performance is based on records (e.g., published articles), evidence syntheses consider trials. The trials reported in the records of the search reference standard were identified and dually assessed for RoB using the Cochrane RoB 2.0 tool [29]. For the NMA reference standard, we included only trials with low or moderate RoB. We conducted NMAs following the approach of the original SR for pain freedom at two hours, pain relief at two hours, sustained pain freedom up to 48 h, and overall adverse events [25]. For each search approach that missed trials compared to the reference standard, we performed an updated NMA using a random effects model, incorporating the trials identified from that specific search approach. We calculated risk ratios with corresponding 95% CIs for all comparisons, and absolute risk differences only for comparisons with direct evidence. All analyses were conducted in R (version 4.3.2) using the netmeta package (version 2.8-2) [30] and the meta package (version 6.5-0) [31]. To evaluate the impact of missed trials on the NMA results, we assessed differences in absolute risk differences (95% CIs) using a predefined threshold of ± 0.05 for each outcome (i.e., CI from the reference standard ± 0.05), as prespecified by the guideline panel to represent a clinically relevant difference [25]. When absolute risk differences could not be calculated, we compared results based on the direction of effect (i.e., risk ratios) and the overlap of the corresponding 95% CIs. We also evaluated how missed trials impact the corresponding CoE assessment using the same methodology as the original review. One trained reviewer involved in the original review (AD) reassessed the certainty of the evidence for each search approach result, while a second reviewer (BNS) verified the plausibility and consistency of these assessments using GRADEpro software (https://www.gradepro.org/). Evaluation of EDT for systematic searching To assess the suitability of EDT for systematic searching, we adapted the characteristics established by Bethel et al. to evaluate database-hosting platforms [32]. We assessed the process of creating a sensitive search strategy using the “Automated Boolean Composer” and considered whether established quality control measures, such as validation with test references and Peer Review of Electronic Search Strategies (PRESS), could be applied to EDT search strategies [33]. Changes from the protocol We expanded the reference standard to include additional records/trials identified by the EDT search following a relative recall approach. We assessed EDT both as a single database and in combination with other databases. In addition to the measures of search performance, we calculated the impact of missed trials on network estimates and the CoE. Results Search performance Database coverage and search performance of single-database searches Database coverage (Table 2) was uniformly high across all databases, but only CENTRAL contained all eligible records at the time of searching (January 2025). All single database searches retrieved more than 98% of the reference standard, with CENTRAL having the highest recall (99.4%). However, search precision varied greatly; EDT achieved the highest precision (17.7%), leading to the highest F2 score (0.5) and the lowest NNR (6). Table 2Performance of single-database searches without publication date limitsDatabaseEligible includesCoverageRetrieved includesSearch resultRecall (%)Precision (%)NNRF1F2EDT180179178100398.917.7%60.30.5MEDLINE180177177162598.310.9%90.20.4Embase180179178219698.98.1%120.10.3CENTRAL180180179159999.411.2%90.20.4NNR Number needed to read, EDT Epistemonikos Database of Trials Performance of single-database searches without publication date limits NNR Number needed to read, EDT Epistemonikos Database of Trials Database combinations Table 3 shows the performance of databases in combination. The cumulative database coverage of all tested combinations was 100% (180/180), as was the cumulative recall in all cases except one: The combination of CENTRAL and EDT missed 1 record that was available in both databases but not identified by either search strategy. However, this combination also had the smallest deduplicated search result, leading to the lowest NNR (9). This was followed by the combination of MEDLINE and EDT, which achieved 100% recall and an NNR of 10. Table 3Cumulative performance of combined database searches without publication date limitsDatabase combinationEligible includesCombined coverageRetrieved includesSearch result (deduplicated)Recall (%)Precision (%)NNRF1F2MEDLINE+Embase+CENTRAL18018018028311006.4160.10.3CENTRAL + EDT180180179161799.411.190.20.4Embase+CENTRAL18018018026171006.9150.10.3Embase + EDT18018018024631007.3140.10.3MEDLINE+CENTRAL18018018020321008.9110.20.3MEDLINE + EDT18018018018271009.9100.20.4MEDLINE+Embase18018018025621007.0140.10.3NNR Number needed to read, EDT Epistemonikos Database of Trials Cumulative performance of combined database searches without publication date limits MEDLINE+Embase+ CENTRAL NNR Number needed to read, EDT Epistemonikos Database of Trials Original SR search performance Table 4 shows the performance of the original search approach. SR reference-list checking and database searching limited by publication date identified 168 of 180 eligible records (Recall = 93.3%). Most records were found by screening reference lists of SRs. Table 4Recall, precision, NNR, and F scores of the original search approachSources of the original searchDate limitsDuplicates removedEligible recordsEligible identifiedSearch resultRecall (%)Preci-sion (%)NNRF1F2Reference lists (manual title screening)nono180156189986.78.2120.20.3MEDLINE+Embase+CENTRAL (date limited)yesyes1616798100.02.0500.00.1Total combined search results for databases and reference listsnopartially180168269793.36.2160.10.2NNR Number needed to read Recall, precision, NNR, and F scores of the original search approach MEDLINE+Embase+ CENTRAL (date limited) NNR Number needed to read Adding up the number of records screened during reference list screening and the database searches for newer RCTs, a total of 2697 references were screened, leading to an NNR of 16 and an F2 score of 0.2. The SRs used for reference list screening had been identified by a previous systematic database search that had retrieved 659 references. However, as this step of the search process did not aim to identify RCTs, it was not added to the total search result in this analysis. Missed records Table 5 gives an overview of the reasons records were missed by database searches without date limits. Table 5Reasons for non-retrieval by database searches without date limitDatabaseNot available in databaseNo abstract availableRecord missing relevant informationEDT (n = 2)101MEDLINE (n = 3)300Embase (n = 2)110CENTRAL (n = 1)001 Reasons for non-retrieval by database searches without date limit Non-availability was the only reason for missed records in MEDLINE. In Embase, one record was not available, and one was missed because it had no abstract in that database. One record was not available in EDT at the time of searching. Additionally, the searches of CENTRAL and EDT both missed the same record: An older publication that only used the drug code (311C90) in the title/abstract, not the generic name (Zolmitriptan) or a brand name [34]. Conversely, this record was identified in both MEDLINE and Embase. In Embase, it was indexed with the appropriate Emtree subject heading, while in MEDLINE, the generic name was stated in the “substance name” field, which was searched as part of the search strategy. While the CENTRAL record was sourced from both PubMed and Embase.com, it contained neither of those details. The EDT record only showed title and abstract; EET was not visible and could not be evaluated. SR reference-list screening missed 24 eligible records (see Table 6). Table 6Reasons for non-retrieval by reference list screeningNewer than previous SRsNot cited by previous SRsMissed by manual screeningMissed by reference list screening of eligible SRs (n = 24)12210SR systematic review Reasons for non-retrieval by reference list screening SR systematic review Twelve were published after the relevant SR and could not have been found. Two were older but not cited in any of the screened reference lists. Ten had been cited by at least one of the reviewed SRs. They were missed by the single-reviewer manual title screening and were not flagged for dual abstract screening. Since the Terminology, Application, and Reporting of Citation Searching (TARCiS) statement [35] recommends using the combined coverage of two electronic citation indexes for citation searching instead of manual screening, we assessed the coverage of Scopus (Elsevier) and citationchaser [36] for our seed references; seven reviews were either not available, or had no or incomplete references in either citation index [37–43]. One notable reason for incomplete reference lists was SRs that provided the full list of included studies only in an online supplement. Impact of missed trials on overall network estimates A total of 174 trials were assessed as having a low or moderate RoB and formed the reference standard. We considered a trial to be found by a search approach if one of its associated records was retrieved. Table 7 gives an overview of retrieved trials categorized by RoB assessment. All database combinations except EDT+CENTRAL found all 174 trials eligible for the NMA. CENTRAL alone and EDT+CENTRAL missed the same trial; EDT, Embase, and MEDLINE alone missed two trials each. The original SR search approach missed six trials. Table 7Retrieval of identified trials categorised by risk-of-bias assessment for all search approachesRisk of biasOriginal SR searchEDTMedlineEmbaseCentralMedline+Embase+CentralLow/moderate (n = 174)162 (93.1%)172 (98.9%)172 (98.9%)172 (98.9%)173 (99.4%)174 (100%)High (n = 15)11 (73.3%)15 (100%)14 (93.3%)15 (100%)15 (100%)15 (100%)Risk of bias Central + EDT Embase+ Central Embase + EDT Medline+ Central Medline + EDT Medline+ Embase Low/moderate (n = 174)173 (99.4%)174 (100%)174 (100%)174 (100%)174 (100%)174 (100%)High (n = 15)15 (100%)15 (100%)15 (100%)15 (100%)15 (100%)15 (100%) Retrieval of identified trials categorised by risk-of-bias assessment for all search approaches Figures 1, 2, 3 and 4 in Additional File 3 present a comparative summary of the analysis results from all sources and the corresponding CoE assessments. All missed trials were placebo-controlled and contributed data on two drug classes: triptans and Nonsteroidal anti-inflammatory drugs (NSAIDs). None evaluated head-to-head comparisons. In this network, the reference standard NMAs for all outcomes produced results similar to those of all search approaches, despite differences in the number of missed trials across search approaches. CoE assessments remained unchanged across all search approaches. Suitability of epistemonikos database of trials for systematic searching EDT is currently in its pilot phase and may change over time. To account for this, we assessed the search functions of EDT in January and May 2025, and observations were checked in November 2025 to account for changes. In terms of search functions, EDT allowed Boolean searching, the use of parentheses, phrases and right truncation, and the combination of multiple lines. A combination of truncation and phrase searching was not possible. Using the search functions, it was possible to translate the Ovid MEDLINE strategy to the EDT interface as a multi-line search strategy, using both free text (title-only or title/abstract) and controlled vocabulary (EET). The search results could be filtered by year and publication type (book, conference, journal, preprint, registry of trials, report, thesis, unpublished, web page). The following information was visible for database records: title, year, source (e.g. journal, preprint server, etc.), author(s), DOI, link, and abstract. EET and publication type classification were not visible at the record level. Individual entries were downloadable in RIS format. Bulk export of 100 records at once was possible directly from the EDT database as an RIS file. To export the entire search result in one go, it was necessary to send the results to the SKP evidence platform, where RIS bulk export was possible. The search strategy could be shared as a URL that contained the executable search, including any date/document type filters applied. There was also an option to export the search strategy as a “single-block strategy” text string, which could later be entered again into the search interface. See Additional File 2 for the checklist of functionalities based on Bethel et al. [32]. A feature of EDT is the “Automated Boolean Composer.” This function allows users to input a concept (e.g., “headache,” “almotriptan”) and receive a formally correct Boolean search string including synonyms in return (e.g., “headache* OR (head* AND pain*)”, “almotriptan* OR LAS31416* OR “LAS-31416” OR “LAS 31416” OR Axert*”). Automated Boolean strategies were available for all EET concepts used in the search strategy. According to a recent preprint, the search strategies are developed for all EET concepts with the exception of broad concepts “where achieving adequate sensitivity is challenging or where a strategy would not be meaningful” [17]. In our study the search queries provided for some concept by the Automated Boolean function changed in some cases between the January 2025 and November 2025. In most cases, additional search terms were added to the queries. Additional File 2 provides a complete overview of all concepts and automated query outputs used in the strategy. In terms of quality control measures, EDT records had accession numbers, but they could not be used in the search. This made the validation of the search strategy using already-known relevant references [44, 45] less accessible than in the other databases tested. Because EET terms were not visible at the record level, it was also not possible to use known relevant references for the identification of appropriate EET terms. Peer review of the search strategy by a second person was possible for EDT searches; however, the search strategy looked different for the reviewer than for the designer. The entire search strategy could be exported as a “single-block strategy,” which translated a multi-line search into a long single-line search string that was harder to read. An alternative was sharing the URL of the search strategy, which was executable and produced a multi-line search. In this case, some of the free-text lines were combined or split differently compared to the search strategy entered originally. See the screenshots in Additional File 2 for a comparison of the search strategy as originally input compared to the single-line text export and the re-running via URL. Finally, our search strategy was limited to publication type “journals,” but during screening, we noticed a number of trial register records in the search results. The inverse was not true for the other publication types; they did not include any misassigned journal articles. Discussion This case study examined the impact of different search approaches (single databases and combinations of search sources) on NMAs of pharmacologic RCTs, assessing search performance, abstract screening workload, and impact on review findings. Search recall reached > 93% in all approaches, with six database combinations achieving 100% recall based on a relative recall reference standard. The number of references retrieved, however, varied widely across the approaches, ranging from 1003 (EDT alone) to 2831 (MEDLINE + Embase + CENTRAL without date limits), leading to different title/abstract screening workloads. The size of the search result also affected the precision of the searches and the NNR. NNR ranged from six for EDT alone to 16 for both MEDLINE+Embase+CENTRAL without date limits and the original search approach. Generally, systematic searches strive for the best balance between precision and recall, while privileging recall. The smallest search result associated with 100% recall was the combination of MEDLINE and EDT (1827 references, NNR = 10, F2 = 0.4). On the other hand, our case study showed that not every reference missed by a search affected the findings of the review: In our analyses, the few records and trials that were missed by some search approaches had no impact on the findings of the NMA and CoE assessments. However, it is important to note that each of our NMAs already included a large number of RCTs relative to the few missed trials. In reviews with a smaller evidence base, the impact of missing studies may be more consequential and should therefore be evaluated. To our knowledge, this is the first evaluation of the impact of search approaches on NMAs, but the results are in line with previous studies examining the impact of MEDLINE coverage [6, 46] and search recall [7] on the results of meta-analyses: Results of meta-analyses that included a large number of trials or concerned pharmacological topics, were less likely to be affected by removing a small number of trials compared to smaller reviews and non-pharmacological topics. Additionally, combining two information sources (two databases, or a databases and reference list checking) was often sufficient to achieve a result comparable to a systematic search using a larger number of databases and additional search methods. In addition to the established databases for the identification of RCTs–MEDLINE, Embase, and CENTRAL–we also assessed the performance and usability of Epistemonikos Database of Trials, a new database. It proved to have a high coverage of eligible records, and the search strategy achieved the best balance of recall and precision (F2 = 0.5) of any source assessed. Search strategy design was sped up by the “Automated Boolean Composer,” and we were able to run a reproducible search strategy. Using this function has the potential to reduce the time needed to design a comprehensive search strategy, as it reduces the time needed to select appropriate search terms and apply the correct syntax. For example, in EDT, it is necessary to search for “LAS-31416” OR “LAS 31416” because the two terms will retrieve different records. This is not the case on most other commonly used platforms (where the use of hyphens is treated the same phrase searching with double quotations marks). Pre-designed search strategies were also available for every EET we used and were sensitive and appropriate for the topic. However, while these strategies are designed and validated by information specialists, users of the function should always review the proposed terms and make changes if necessary. Additionally, while the terms used in an individual search strategy remain unchanged and can be re-run using an individual URL, the search query provided for a concept when running the Automated Boolean Composer may change over time. When updating a search originally created with the Automated Boolean Composer, these changes would not be applied automatically. But it may be good practice to check for changes in the search strings that are provided by the automated function. Additionally, we noted some drawbacks concerning the export of search results and search histories and a lack of transparency concerning the database functionality. In contrast to EDT’s good performance, the search approach used in our original SR proved to be a less efficient option. While we originally chose the stepwise approach with the intention save time when faced with a very broad topic, using reference-list checking of previous SRs as a primary search method led to a higher number of references to be screened than most other approaches, and some eligible records were missed during the manual single-reviewer title-only screening. While using electronic citation indexes for backward citation searching would have enabled dual screening and deduplication of results, we found that some relevant SRs were either not available or used online supplements for their complete reference lists. These references were not picked up by electronic citation indexes. A recent scoping review found that primary/stand-alone citation searches appear not to be sensitive enough for SRs [23]. In our case study, reference list checking alone had a 86.7% recall, which increased to 93.3% when combined with date-limited database searches. This was still high enough for NMAs based on this search approach to produce results similar to all other search approaches we assessed. Our study has several limitations. First, this is a case study based on a single systematic review; the results are not generalizable to all network meta-analyses. Specifically, our analyses were based on a large systematic review of acute episodic migraine, which may limit the applicability of the findings to other conditions, populations, or types of interventions. The present study used the same eligibility criteria as the original review [25]: By restricting the network meta-analysis to trials with low or moderate risk of bias, the observable impact of missed trials may have been reduced. Additionally, focusing on double-blinded RCTs evaluating pharmacologic interventions may impacted on the search performance as this type of trials may be easier to identify common search approaches. For example, research has shown that coverage in commonly used databases and recall tends to be lower for nonpharmacological RCTs compared to pharmacological RCTs [20, 22, 47, 48]. Further research across different clinical areas and trial designs is needed to determine whether our observations are broadly generalizable. Second, when establishing the reference standard, we did not screen the results of the MEDLINE, CENTRAL, and Embase searches without a date limit due to time constraints. It is possible that they would have retrieved additional eligible records not found by either the original SR search or EDT. However, when comparing the date-limited database searches with the same date-ranges of EDT results, we found that the strategies identified the same references. It seems therefore unlikely that a large number of eligible records was missed by this approach. Additionally, the search strategies of the original review used the eligible records found in the first round of reference list screening to identify search terms via text analysis and to validate the search strategy. It is possible that search strategies without this solid foundation of already-known eligible records would have been less sensitive. However, given the consistent terminology used in pharmacologic interventions, it is likely that a smaller set of known records would have sufficed to achieve similar search sensitivity. Finally, EDT is currently in its pilot stage, which means that the interface, search, and export functionalities may have changed by the time this paper is published. Conclusion All search approaches we tested were acceptable for network meta-analyses of pharmacological migraine treatments under assessment. They reached a high search recall and consequently led to similar network estimates and identical CoE assessments as the reference standard. However, the associated workload for screening differed greatly. Specialized RCT databases (CENTRAL, EDT) were more efficient in this regard than either general healthcare research databases (MEDLINE, Embase) or using SR reference-list checking as the primary search source. However, while missing a small number of eligible trials may not affect the findings of NMAs or meta-analyses in some cases, there is no pre-defined cut-off for minimum recall of systematic searches. Additionally, recall can only be assessed when the eligible records are already known. While information specialists generally use simple relative recall approaches during search strategy design [44, 45], the full extent of available research will be only known at end of the systematic review process. For this reason, aiming for the highest possible recall when designing search strategies and selecting information sources is still the best way to avoid retrieval bias. On the other hand, searchers also need to consider that the size of the search result is associated with the information source used and the design of the search strategies. In our case study, using a combination of two databases was sufficient to achieve 100% relative recall while keeping the number of references retrieved comparatively small. Continued research on database coverage and average achievable recall is necessary to help informed decisions on appropriate information sources for systematic searches. In this context, the new Epistemonikos Database of Trials appears to be a promising cost-free addition to established databases for RCT searching. However, larger studies are needed to provide more generalisable insights into the relation between database selection, search yield, and impact on review findings. Supplementary Information Supplementary Material 1. Supplementary Material 2. Supplementary Material 3. Acknowledgements We thank Gabriel Rada and Javiera Peña of the Epistemonikos Foundation for providing additional information about the Epistemonikos Trials Database. We would also like to thank Claus Nowak for his statistical expertise and Petra Wellemsen for administrative support throughout this project. Abbreviations Cochrane Central Register of Controlled Trials Confidence interval Certainty of evidence Epistemonikos Evidence Taxonomy Epistemonikos Database of Trials False negative False positive True negative True positive Methodological Expectations of Cochrane Intervention Reviews Network meta-analysis Number needed to read Nonsteroidal anti-inflammatory drugs Peer Review of Electronic Search Strategies Randomized controlled trial Risk of bias Sustainable Knowledge Platform Systematic review Terminology, Application, and Reporting of Citation Searching Authors’ contributions IK and AD have conceptualized and designed the work. IK, AD, AS acquired, analyzed and interpretated data. IK and AD drafted the work. GG, AS, CNB, and BNS substantially revised the work. All authors reviewed the manuscript. Funding Internal funding from Cochrane Austria. Data availability The most important data generated or analyzed during this study are included in this published article and the additional files. Additional datasets are available from the corresponding author upon reasonable request. Declarations Ethics approval and consent to participate Not applicable. Consent for publication Not applicable. Competing interests The authors declare no competing interests. Footnotes Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Supplementary Materials Supplementary Material 1. Supplementary Material 2. Supplementary Material 3. Data Availability Statement The most important data generated or analyzed during this study are included in this published article and the additional files. Additional datasets are available from the corresponding author upon reasonable request.