From international guidelines to practice: a multimethod evaluation of BMJ Rapid Recommendations focusing on suitability for adaptation and implementation Authors: Siri Schøyen Seterelv, Samir Gupta, Andrew Kouri, Ivan D Florez, Thomas Agoritsas, Per O Vandvik, Stijn Van de Velde Published: 2025 ABSTRACT Abstract AbstractObjectivesWe aimed to assess usage and explore factors influencing the suitability of British Medical Journal (BMJ) Rapid Recommendations—a set of international guidelines meeting standards of trustworthiness—for adaptation and implementation, further defined as implementability.DesignQualitative descriptive study with a multimethod approach.ParticipantsWe used citation screening and a survey to identify groups that had used any of a sample of 14 BMJ Rapid Recommendations in adoption, adaptation, de novo guideline development or implementation.Setting12 participants from a range of country income levels were interviewed, using semistructured interviews.MethodsInterviews were analysed using thematic analysis to identify themes related to implementability of the guidelines. We compared these findings with those from a critical appraisal conducted using three different appraisal tools.ResultsBMJ Rapid Recommendations are used in high-income, middle-income and low-income countries. We identified 33 instances of use. Our thematic analysis of the interviews uncovered 14 challenges to implementability, categorised under perceived trustworthiness, comprehensibility and practical relevance. The critical appraisals gave the BMJ Rapid Recommendations high scores on validity, but low scores on implementability and measurability. The challenges found in the interviews were generally corroborated by the appraisals, but the comparison also exposed tensions such as the complex interaction between methodological rigour and perceived credibility, the balance between novelty and compatibility with current practice, and the difficulty of offering globally applicable yet locally practical guidance.ConclusionImplementability challenges may limit effective use of international guidelines globally. Recognising and reflecting on tensions between rigour, credibility and contextual feasibility may help improve their uptake and value. The next phase of this research will explore strategies to enhance guideline implementability during their development.Keywords: QUALITATIVE RESEARCH, Decision Making, Guideline Adherence, Protocols & guidelines STRENGTHS AND LIMITATIONS OF THIS STUDY The study combined interviews and appraisals for a thorough understanding of guideline implementability. The study combined interviews and appraisals for a thorough understanding of guideline implementability. Inclusion of participants from several countries provided varied perspectives and insights across diverse cultural and healthcare contexts. Inclusion of participants from several countries provided varied perspectives and insights across diverse cultural and healthcare contexts. The interview sample did not capture the viewpoints of those who considered but chose not to use the British Medical Journal (BMJ) Rapid Recommendations. The interview sample did not capture the viewpoints of those who considered but chose not to use the British Medical Journal (BMJ) Rapid Recommendations. The number of participants from low- and middle-income countries was low, and they mainly had experience from one BMJ Rapid Recommendation. The number of participants from low- and middle-income countries was low, and they mainly had experience from one BMJ Rapid Recommendation. Background While evidence-based healthcare relies on clinical practice guidelines, developing high-quality guidelines is a challenging task that requires significant resources.1 Annually, a multitude of new guidelines surface from international, national and local panels, many with significant overlap in scope. The concept of promoting international collaboration to develop fewer, higher-quality guidelines, coupled with efficiently adapting and implementing them locally, is gaining appeal as a means to reduce duplication of effort, accelerate dissemination and ultimately enhance quality of care.2 3 However, developing guidelines at an international level and applying them within local settings comes with its own set of difficulties. Although the quality of international guidelines has improved over time, there are still challenges with implementability.2 4 Implementability refers to the intrinsic characteristics of a guideline that enhance adaptation and implementation. Both adaptation and implementation are processes usually external to guideline development (table 1). Consistent with frameworks in implementation science, adaptation can be viewed as a deliberate step within the implementation process, facilitating the fit between the guideline and its context of use.5,7 Table 1Definitions of key terms used in the article.Definitions of key termsAdaptation‘A systematic approach for considering the endorsement or modification of guidelines produced in one setting for application and implementation in another as an alternative to de novo guideline development, or as a first step in the process of implementation, while preserving evidence-based principles’.82ImplementationDeliberate efforts to increase impact and uptake of successfully tested healthcare innovations.83ImplementabilityIntrinsic characteristics of a guideline that enhance adaptation and implementation.5 7GuidelinesStatements that include recommendations intended to optimise patient care. They are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options.84 Definitions of key terms used in the article. Critiques of international guidelines typically highlight too vague descriptions of targeted settings, a lack of context-sensitive recommendations and insufficient advice on aspects to consider for local adaptation and implementation.38,10 Some describe this low implementability as a ‘longstanding injustice’ of misaligned international recommendations.11 Implementability issues inherent in international guidelines can delay, hinder or prevent their uptake. Additional challenges such as complex adaptation and implementation methods, patchy use of supportive digital tools and local health system obstacles further contribute to these difficulties.10,13 This frequently leaves guideline end-users unable to make well-informed decisions, resulting in missed opportunities to improve patient outcomes and wasting of scarce resources. 2 3 11 14 Current studies on the perceived implementability of international guidelines offer valuable insights but have limitations. They either focus solely on high-income (HIC) or low- and middle-income (LMIC) settings, address only guideline factors affecting either adaptation or implementation, or rely on indirect sources instead of lived experiences. 4 15 International guidelines are produced by a wide range of institutions and global collaborations. The British Medical Journal (BMJ) Rapid Recommendations are a prime example, developed by the MAGIC Evidence Ecosystem Foundation (MAGIC) in partnership with the BMJ. This initiative seeks to accelerate user-friendly and trustworthy clinical recommendations into practice for important clinical questions. It particularly aims to minimise the time between the availability of potentially practice-changing evidence and guideline publication.16 BMJ Rapid Recommendations are published on the BMJ website, with complementary links to the MAGIC authoring and publication platform (MAGICapp) as well as one or more supporting systematic reviews. In MAGICapp, readers can explore additional information and decision aids in multilayered digital formats, and organisations are encouraged to use the digitally structured content for adaptation and implementation. BMJ Rapid Recommendations use the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system and are designed to support decision-making for individual patients and take a patient perspective.17 They have global visibility, are widely accessed online and are included in the Emergency Care Research Institute Guidelines Trust repository of trustworthy clinical practice guidelines.18 However, it is unclear how they are used worldwide and what challenges arise. This study sought to evaluate whether and how organisations use BMJ Rapid Recommendations as an example of trustworthy international guidelines, and to explore the factors affecting their implementability. We operationalised ‘use’ as the explicit consideration of BMJ Rapid Recommendations by guideline groups or other bodies as part of guideline adoption, adaptation, de novo development or implementation. Insights gained from this research will inform a follow-up study to create and assess strategies to improve guideline implementability in future BMJ Rapid Recommendations and will inform the broader efforts to improve guideline implementability. Methods Overview We employed a descriptive multimethod research design and a systematic approach to comprehensively investigate the research problem and verify our findings.19 The Tailored Implementation for Chronic Diseases (TICD)-checklist served as our primary theoretical framework for this investigation.20 This is one of the newest and most comprehensive frameworks in implementation science, building on 12 other implementation frameworks.21 TICD targets implementation of guidelines especially and is user-friendly. We used the BMJ Rapid Recommendations scientific articles as the unit of evaluation, looking at the entire guideline and the specific recommendations. The methods we used include: (1) screening citations and a survey to identify organisations that used BMJ Rapid Recommendations for adoption, adaptation, de novo development or implementation, (2) conducting semistructured interviews with members from organisations that used BMJ Rapid Recommendations and agreed to participate, (3) evaluating the implementability of selected BMJ Rapid Recommendations through critical appraisal. We deidentified and stored all personal data on a secure server and adhered to the Standards for Reporting Qualitative Research22 (online supplemental appendix 1). Researcher description Two authors (SSS, SVdV) collected and analysed the data. Both have training in qualitative research methods and self-identify as post-positivists. Their choice of paradigm and their backgrounds might influence research outcomes. As researchers based in Norway, their perspectives may be shaped by economic, cultural and systemic contexts that differ from the global guideline community. SSS, a female public health physician with experience from primary healthcare and guideline implementation, and SVdV, a male senior researcher experienced in development of trustworthy guidelines, as well as adaptation and implementation of guidelines across high and low resource settings, bring complementary skills. Both researchers are employed at Lovisenberg Diaconal Hospital and affiliated with MAGIC, the organisation responsible for the BMJ Rapid Recommendations. However, neither has served as a panel member for any of the evaluated guidelines. One researcher (SVdV) had prior working relationships with some interview participants, which could influence responses and data interpretation. However, potential bias was minimised due to the non-hierarchical nature of these relationships and the involvement of another researcher (SSS) in managing the data coding. Additionally, our analysis incorporated independent appraisals by external reviewers, triangulation with interview data and deductive coding guided by established implementation frameworks to mitigate potential subjectivity. Public involvement We engaged interest-holders—defined here as researchers, guideline developers and implementation experts—by presenting the protocol at the Guideline International Network (GIN) conference, sharing findings at the European Implementation Collaborative conference, coauthoring with non-MAGIC researchers and hosting workshops with the guideline community.23 24 For the workshops, we collaborated with the chairs of the GIN implementation working group and the adaptation working group. During the workshops, we presented the intermediate findings and actively solicited feedback on the preliminary list of themes. This feedback largely confirmed our preliminary analyses. Patients were not involved in this study. Guidelines Table 2 lists the 14 BMJ Rapid Recommendations selected for this study, covering all available guidelines (as of October 2023) since July 2018. This pragmatic selection spans various specialties to facilitate a broad exploration of use and implementability issues. For the critical appraisal, we limited the sample to the five most recent recommendations (as of March 2022), given that we expected a homogeneous result, since all BMJ Rapid Recommendations follow the same process. Table 2Overview of the 14 sampled BMJ Rapid RecommendationsGuideline nameMonth publishedPCSK9 Inhibitors and ezetimibeMay 2022Plasma exchange and glucocorticoid dosing for patients with ANCA-associated vasculitis*February 2022Medical cannabis or cannabinoids for chronic pain*September 2021A living WHO guideline on drugs for COVID-19*†September 2021SGLT-2 inhibitors or GLP-1 receptor agonists for adults with type 2 diabetes*May 2021Gastrointestinal bleeding prophylaxis for critically ill patients*January 2020Colorectal cancer screening with faecal immunochemical testing, sigmoidoscopy or colonoscopyOctober 2019Thyroid hormone treatment for subclinical hypothyroidismMay 2019Subacromial decompression surgery for adults with shoulder painFebruary 2019Dual versus single antiplatelet therapyDecember 2018Oxygen therapy for acutely ill medical patientsOctober 2018Prostate cancer screeningSeptember 2018Corticosteroid therapy for sepsisAugust 2018Patent foramen ovale closure or drug therapy for management of cryptogenic strokeJuly 2018*BMJ Rapid Recommendations that the team appraised using Appraisal of Guideline Research and Evaluation Recommendations EXcellence, The Guideline Implementability Appraisal and The Guideline Language and Format Instrument instruments. †We did not interview the panel members of this BMJ Rapid Recommendation which was governed by the WHO.BMJ, British Medical Journal. Overview of the 14 sampled BMJ Rapid Recommendations BMJ Rapid Recommendations that the team appraised using Appraisal of Guideline Research and Evaluation Recommendations EXcellence, The Guideline Implementability Appraisal and The Guideline Language and Format Instrument instruments. We did not interview the panel members of this BMJ Rapid Recommendation which was governed by the WHO. BMJ, British Medical Journal. Data collection Citation screening and survey To identify groups that have used BMJ Rapid Recommendations, one researcher (SVdV) screened PubMed and Google Scholar for references (up to October 2023) that cite the selected BMJ Rapid Recommendations. Planned processes to introduce the BMJ Rapid Recommendations in specific healthcare settings fell under this definition. The country’s economic classification was used to describe the contexts in which the BMJ Rapid Recommendations were used based on World Bank classification by income level for 2023.25 The researcher initially reviewed titles and, if uncertain about the content, also examined the abstract and full text to evaluate its relevance. To identify further use of the BMJ Rapid Recommendations, we invited all guideline panel members (n=185) from 13 of the 14 selected guidelines to participate in an online survey, conducted from February to May 2023. The survey sought to identify relevant cases using the guidelines as part of an organisation. We excluded panel members involved in the WHO guideline on drugs for COVID-19, as it was governed by the WHO, and including them would have required a separate permission process.26 Interviews We emailed invitations for semistructured interviews to one designated contact person for each identified case where BMJ Rapid Recommendations had been used by an organisation or network of organisations. In the event of the contact person wanting to participate with a colleague, we accommodated this. In the event of non-response, we issued two reminders. Two researchers (SSS, SVdV) individually conducted 30 min video-call interviews in English or Norwegian with consenting participants, using a piloted interview guide (online supplemental appendix 2) and transcribed the conversation verbatim. We piloted the interview guide with two individuals experienced in international guideline development and adaptation to ensure appropriate interview length and clarity of prompts. Interviewees gave informed consent before the recorded online interviews began. All interviews were conducted between October 2023 and April 2024. Critical appraisal Two researchers affiliated with MAGIC (SSS, SVdV) and three affiliated with external organisations (AK, IDF, SG) individually conducted the critical appraisal. Given that no single validated tool exists to capture all factors influencing implementability, we selected three distinct appraisal instruments: The Guideline Implementability Appraisal (GLIA), Appraisal of Guideline Research and Evaluation Recommendations EXcellence (AGREE-REX) and The Guideline Language and Format Instrument (GLAFI).5 27 28 Each tool offers a unique scope, addressing different implementability factors and each targets the recommendation level (see table 3 for further details about each instrument). Both AGREE-REX and GLAFI are built on the Guideline Implementability Decision Excellence Model (GUIDE-M), which has been face-validated by a large international group of experts.7 GLAFI has also undergone additional face validation. AGREE-REX’s evaluation includes assessments of construct validity and discriminant validity. Meanwhile, GLIA, as the longest-standing tool, has been assessed for both content validity and construct validity. While AGREE-REX and GLIA are designed for critical appraisal, GLAFI was designed primarily to support guideline authors during development.5 27 28 Table 3Description of the appraisal toolsAppraisal toolsAimDomainsItemsAGREE-REX27Assesses recommendationsTo optimise the quality of guideline recommendations, defined as recommendations that are clinically credible, trustworthy and implementable. Clinical applicability EvidenceApplicability to target usersApplicability to patients/populations Values and preferences Values and preferences of target usersValues and preferences of patients/populationsValues and preferences of policy/decision-makersValues and preferences of guideline developers Implementability PurposeLocal application and adoptionGLAFI28Assesses guideline and recommendationsTo optimise language and format during guideline development, thereby enhancing guideline uptake. Language Succinct and uncomplicatedActionable/effective writingConsistent use of termsPersuasive Format Standardised componentsDocument layoutDocument structureInformation visualisationGLIA5Assesses guideline and recommendationsTo help anticipate barriers to implementation success Global domain Definition of target patient populationDefinition of intended audienceDescription of target settingsCredibility of guideline organisation(s) and author(s)Strategies for implementation or tools for applicationClarity on sequence of recommendationsInternal consistency of the guidelineEasy to identify recommendationsConcise recommendations Executability Recommendations specific on what to doSufficient details on how to perform the recommended action Decidability Sufficient detail on when to apply the recommendationConsideration of all reasonable combinations of conditionsClarity on the logical relationship between recommendations Validity Explicit justification for the recommendationExplicit statement on the quality of evidence Flexibility Explicit statement of strength of each recommendationSpecification of patient characteristics requiring/permitting individualisationSpecification of practice characteristics requiring/permitting modification Effect on process of care Impact on workflowResource commitment for pilot test of recommendations Measurability Measurability of adherenceMeasurability of outcomes of this recommendation Novelty Need to acquire new knowledge or skillsConsistency with attitudes and beliefs of intended audienceConsistency with patient expectationsAGREE-REX, Appraisal of Guideline Research and Evaluation Recommendations EXcellence; GLAFI, The Guideline Language and Format Instrument; GLIA, The Guideline Implementability Appraisal. Description of the appraisal tools AGREE-REX, Appraisal of Guideline Research and Evaluation Recommendations EXcellence; GLAFI, The Guideline Language and Format Instrument; GLIA, The Guideline Implementability Appraisal. Each researcher independently applied the three tools to appraise the selected guidelines. They appraised the BMJ Rapid Recommendations based on the infographic and article on the journal website, as these are the most widely consulted components.29 Appraisers omitted the computability domain in GLIA, as we considered this to be context-specific and not answerable from a general perspective. Data analysis Interviews To describe the profiles of interviewees, we calculated absolute frequencies and proportions. To analyse interview transcripts, two researchers conducted a qualitative thematic analysis following five steps: familiarisation, applying initial deductive codes, generating inductive codes, searching for themes and defining themes, searching especially for enablers and barriers to implementability.30 We based the initial deductive codes on the frameworks of Stirman et al, Song et al and Flottorp et al.6 10 20 We then coded any material not captured by these codes inductively, added the new codes to the codebook and recoded the material.31 One researcher (SSS) coded all data, with another researcher (SVdV) double-checking results. We resolved disagreements through informant clarification or discussions with a third researcher (POV). Both researchers (SSS, SVdV) are fluent in English and Norwegian. One translated the Norwegian quotes into English, and the other conducted a back-translation to confirm accuracy. As part of the analysis, we compared themes between interviewees with experience from BMJ Rapid Recommendation guideline panels and those without. To assess data saturation, we did a code frequency count analysis (online supplemental appendix 3) and reflected on information power as described by Malterud.32 33 The research team collectively agreed on the final challenges and their definitions. We used the Atlas.ti V.23 software for the qualitative data analysis. Critical appraisal The three appraisal instruments together included 43 items. We standardised evaluations across the three instruments by classifying them as positive or negative. For AGREE-REX, scores of ≥5 were defined as positive and ≤4 as negative, consistent with Florez et al and to align with the binary yes/no format of GLIA and GLAFI.34 For each item, a negative classification was assigned if over half of the five appraisers provided a negative evaluation. To aggregate the scores, we summed the number of negative or positive evaluations per domain across the five guidelines and expressed this as a percentage of the total scores per domain for all five guidelines. Lastly, we used the framework analysis method to compare findings from appraisals and interviews to assess alignment and divergence. Results Use of the BMJ Rapid Recommendations For 11 of the 14 sampled BMJ Rapid Recommendations, we identified 33 instances where organisations either endorsed (n=2), adapted (n=7), used them to develop new guidelines (n=13) or implemented them (n=11) (online supplemental appendix 4).35,59 The most frequently used guidelines were the living WHO guideline on COVID-19 drugs (n=8) and oxygen therapy for acutely ill medical patients (n=8). Instances of usage were distributed across 12 HICs, 4 middle-income countries (MICs), 2 LICs, 2 international network initiatives of HICs and one international initiative of LMICs (figure 1). One organisation initially considered using a BMJ Rapid Recommendation to develop a new guideline but later chose to exclude it from the process.60 For three of the selected BMJ Rapid Recommendations, we did not find information relating to their use.61,63 Map showing all groups found to use BMJ Rapid Recommendations by country and income status. HICs are marked in orange, LMICs are marked in blue. BMJ, British Medical Journal; HICs, high-income countries; LMICs, low- and middle-income countries. Insights from interviews We interviewed 12 of 22 invited candidates (54%). The number of invited candidates is lower than the number of identified usage cases (n=33) because some candidates were involved in multiple cases (interviewee characteristics provided in online supplemental appendix 5). Among the 12 interviewees, 7 came from the citation screening and 5 through the survey. The survey had a 24% response rate. However, only eight individuals had experience with use of BMJ Rapid Recommendations as part of an organisation and were subsequently invited to participate along with the identified contact persons from the citation screening. The group represented a mix of geography, clinical experience and expertise in guideline development, adaptation and implementation. Interviewees had experience with BMJ Rapid Recommendations, across hospital and primary care and in both medical and surgical specialities. Eight interviewees came from European HICs, three from MICs and one from a LIC; none came from Australia or North America. Modifications Eight interviewees reported modifying BMJ Rapid Recommendations, most commonly the WHO living guideline on COVID-19 drugs. Content modifications were most frequent, with the recommendation strength altered in three cases. According to the Framework for Reporting Adaptations and Modifications to Evidence-based Interventions framework, the primary adaptation goals were to improve fit with recipient, followed by increased reach and reduced costs.6 Interviewees from LMICs cited improved feasibility, often due to resource constraints, as their main reason for modifications. In contrast, interviewees from HICs pointed to existing laws, regulations and prior provider training as key drivers for adapting the guidelines. Factors influencing implementability Interviewees identified challenges and enablers to BMJ Rapid Recommendation implementation. We synthesised 14 themes from interviews organised under three categories: trustworthiness, comprehensibility and practical relevance (table 4). Seven themes were recognised as both a challenge and an enabler. Results from the code frequency count analyses indicated saturation of themes after 9 of the 11 interviews. Table 4Presentation of the themes and categories found related to the implementability of BMJ Rapid RecommendationsThemesExplanationsTrustworthiness Methodological rigourHow the applied guideline development methods and the enforcement of conflict-of-interest rules are perceived to influence the implementability of the guidelines. Accessibility of methodological informationHow easy it is to access relevant details about the guideline development process and understand the rationale behind the international panel’s decisions. Currency of the informationHow the timeliness and currency of the information presented in guidelines or recommendations, including the frequency of new evidence searches, is perceived. Credibility of the guideline sourceHow the guideline’s credibility is perceived based on the familiarity and trust interest-holders have with both the individuals who created it and the organisations they represent.Comprehensibility Presentation of recommendationsHow the visual presentation of recommendations through text, tables and illustrations is perceived to impact the ease of reading, interpreting and recalling information, as well as influence communication among healthcare professionals and with patients. Specificity of recommendationsHow clear and specific the recommended actions in the international guideline are perceived to be. This includes the definition of patient groups and the clarity regarding who should perform what actions, where, when and how. Availability of guideline toolsWhether the guideline incorporates tools, different language versions and/or other resources perceived to facilitate its dissemination and implementation.Practical relevance Choice of clinical scopeHow the scope of the guideline is perceived to align with national or local needs, considering whether it addresses a highly specialised topic or a topic affecting a larger population, as well as the range of issues it covers for that topic. Choice of guideline perspective and frameworkHow the factors considered by the international guideline panel, and their chosen perspective are perceived to align with the needs of the organisations implementing the guideline. Acceptability and cultural appropriateness of recommendationsHow well the recommendations are perceived to align with the values, preferences and cultural norms in the settings or contexts trying to use them and how this fit influences the implementability of the guidelines. Compatibility with current practiceHow the recommendations align with or differ from existing standards of care, and how this is perceived to influence the guidelines’ implementability and the effort needed to implement them in practice. Feasibility and accessibility of the recommended interventionsWhether there are perceived challenges with the feasibility and/or accessibility of the recommended clinical action across different settings or contexts, and how the guideline addresses these challenges. Guidance on implementationWhether the guideline provides guidance on assessing suitability in different contexts, offers advice on local implementation approaches or addresses strategies for managing the unavailability of first-choice interventions. Measuring adherence and outcomesWhether the guideline includes guidance on how to assess the adherence to the recommendations or how to measure possible outcomes.BMJ, British Medical Journal. Presentation of the themes and categories found related to the implementability of BMJ Rapid Recommendations BMJ, British Medical Journal. Trustworthiness While many interviewees acknowledged the guidelines’ trustworthiness, some emphasised that clinicians’ trust in their medical societies or national experts often overshadowed their perceptions of the BMJ Rapid Recommendations. Interviewees referred to experiences with notable resistance from colleagues during implementation of BMJ Rapid Recommendations, largely due to unfamiliarity with the guideline creators. So, there are many (….) not affiliated with MAGIC who wonder a bit: Who are these people doing it (making guidelines)? Why should we trust it, in a way. Why isn’t (names local expert) involved in this? (Interview A) Some interviewees saw the partnering with WHO for the COVID-19 drug guideline as bolstering the BMJ Rapid Recommendation’s credibility. Interviewees viewed the BMJ Rapid Recommendations methodological qualities as important assets, citing them as a great product, based on meticulous work and with strict inclusion criteria for panel members. However, there was also a perspective that while strict rules to avoid conflicts of interest among panel members may increase methodological trustworthiness, they could potentially hinder the guidelines’ implementability later on. I think (…), the framework for handling and managing conflicts of interest in the BMJ Rapid Recs are rather strict, and they tend to exclude some people who could provide valuable input to the guidelines. (…) Obviously you need to draw a line, but personally I’m probably more open to some panel members having disclosures as long as they are reported and managed. (…) Because I think sometimes those specific people can actually provide very valuable input to the context, and (…) in the end, they will be some of the clinicians which will use or advocate for use of these guidelines. (Interview K) Interviewees presented a need for easier access to methodological information and called for a more frequent updating schedule for non-living guidelines. As one interviewee noted: It would be an argument from our organisation, they would say: we can’t adopt something that is five years old, we really can’t. We should just make them de novo. (Interview H) The importance of updated evidence was echoed by another interviewee, who focused on how the continuous updating of the living guideline for COVID-19 drugs increased trust. They were dynamic, and that’s the rapid review, which you were doing at that time. So, we were sure that, whatever the evidence, it is going to be there, this will be coming, as part of this ongoing process of revising these guidelines. So that part was one and the foremost which helped us to bring trust on the guidelines, to continue following these guidelines. (Interview J) Comprehensibility Interviewees praised the BMJ Rapid Recommendations’ infographics and presentations for accessibility and ease of understanding. I think that the way they present a guideline, especially with the infographics, it’s (…) more easy to use than just a flat text. (Interview C) However, some interviewees expressed concerns about the complexity of the infographics and the use of unfamiliar terms for clinicians. Many of the used terms in the infographic are difficult to understand for colleague clinicians and for medical students (eg, quality of evidence, risk of bias). (Interview I) Their concerns also included a need for guideline tools, expressing the need for additional training materials for staff, accessible summaries for patients and more language versions. Interviewees identified the guidelines’ specificity as a challenge, particularly in the recommended actions, highlighting the need for greater detail. Well, if you say consider tapering whatever, then give us a tapering strategy! Tell us exactly what that means (…). A handout that you could use in your clinical practice. So that patients could say, OK … in week one, I have to do that, in week two I have to do that. But that’s not there. (Interview D) Despite these concerns, interviewees also noted how the guidelines, particularly the infographics, could enhance communication with other healthcare personnel and patients. Practical relevance The BMJ Rapid Recommendations, except for the WHO living guidelines for COVID-19 therapeutics, focus on one or two specific clinical questions. Interviewees identified this choice of clinical scope as a challenge, as only a limited set of issues was addressed, noting the difficulty of excluding other necessary patient interventions. Some interviewees felt the recommendations often prioritised highly specialised topics over more urgent issues with potential for broader impact on a larger population. They also mentioned that it was unclear how these choices of topics were made. Obviously, there are many burning questions within each different specialty or disease, and obviously in the BMJ Rapid Rec you need to pick, within surgery, critical care and I mean and medicine and so forth. So, it’s a strength that it’s broad, but it’s also a limitation because you can also only grab a few of these burning questions. At least from my point of view, I’m not familiar how these different guidelines or topics are prioritised. (…) I think it would be nice if it was more open on how this specific decision making is done. (Interview K) The BMJ Rapid Recommendations takes the individual patient perspective instead of the population/health system perspective. While interviewees confirmed that the patient perspective was considered valuable, they also identified the absence of a health system perspective as a challenge. Some interviewees commented that they missed a consideration of acceptability and cultural appropriateness and elaborated how this could be a reason for not implementing certain BMJ Rapid Recommendations. Another interviewee indicated that the guideline development panel did not include members with bedside experience from enough different settings to ensure an implementable recommendation. Interviewees from LMICs identified accessibility to drugs or feasibility of recommended clinical actions as major barriers for implementation. One interviewee suggested that this challenge might be more prominent for international guidelines, while others stressed the need for alternative recommendations in settings where accessibility is likely problematic. Maybe just providing a comment explaining that you are considering that this may not be the best way to go in another context? I think that’s something that will be highly appreciated, because sometimes we look at the recommendation and we say OK they are recommending this, but we don’t even have this drug, and we will not have it for a year or two years. (Interview E). Participants from both HIC and LMIC sought guidance on assessing applicability and conducting local adaptation. Additionally, participants from LMIC requested advice on alternative solutions if the recommended intervention was unavailable. How to assess adherence to the BMJ Rapid Recommendations and measure patient outcomes emerged as a barrier, but interviewees expressed uncertainty about whether national or regional levels should provide advice about measurable indicators. One interviewee explained that the BMJ Rapid Recommendations were used actively to identify practices that required change, thereby improving implementation of new evidence. However, other interviewees identified the lack of compatibility with current practices as a challenge and reflected on how the BMJ Rapid Recommendations’ drive towards newness might contribute to reduced implementability. Even if the reasons behind a change in practice are acceptable to both patients and clinicians and the change is feasible, the effort required to alter practice may still be perceived as too high, thereby hindering implementation. I think that the main comment we received is that moving away from something, like using a cholesterol level to determine your further treatment options, is probably much too practice changing…. The difficulties in communicating that to patients, probably makes it very difficult to implement. (Interview D) Conversely, when the BMJ Rapid Recommendation did not differ much from current practice, some interviewees noted that the effort to implement was minimal. Implementation difficulties also arose when a new BMJ Rapid Recommendation conflicted with other existing guidelines. One interviewee described an adaptation process where discrepancies between the new recommendation and slightly older recommendations from a respected medical society made it nearly impossible to gain clinician acceptance of the new guidelines. We identified two distinct understandings among interviewees regarding the importance of a guideline’s compatibility with current practices. One informant posed the open question of whether international guidelines should recommend proportionate interventions that consider feasibility across different countries and healthcare systems or should only recommend the highest attainable care in the world. My idea of measuring quality is about, (1) getting a baseline of what’s available and then (2), being able to drive improvement. (…) if you have that mindset, then you are tempted (…) towards creating guidelines or assessment tools which are slightly aspirational, but they're also achievable, right? So, you create your guidelines, (…) so that you think this is something we could reach as the kind of highest limit. And then you push your process towards that. Whereas there’s another way of doing it, which is about doing baselines and indicating unavailability and then using that for advocacy, right? And using that to say (…) there’s no reason why…, a covid-19 patient in (names country), should have worse care than a patient in Norway, (…). We do not have this and this, and this is a reason to then advocate to fund us. So, I think in my mind those would be the two sides of the debate where you would have to negotiate. (Interview I) To further explore how challenges and enablers might materialise in guideline quality and implementability, we next applied three appraisal instruments (AGREE-REX, GLAFI and GLIA). Critical appraisal Figure 2 shows the aggregated appraisal results across the five BMJ Rapid Recommendations. Overall, there were no substantial differences between the five guidelines, with 14–19 negative evaluations out of 43 questions across the three tools. (Online supplemental appendix 6) The aggregated critical appraisal results across the five BMJ Rapid Recommendations showed negative evaluations in orange and positive evaluations in green. AGREE-REX, Appraisal of Guideline Research and Evaluation Recommendations EXcellence; BMJ, British Medical Journal; GLAFI, The Guideline Language and Format Instrument; GLIA, The Guideline Implementability Appraisal. Appraisals identified most negative evaluations in the Implementability domain (AGREE-REX) and Measurability domain (GLIA), while Clinical applicability (AGREE-REX), Format (GLAFI) and Global domain and Validity (GLIA) received positive assessments. The item Evidence in AGREE-REX and GLIA’s Validity domain received positive scores, indicating strong perceived methodological quality. In the AGREE-REX Values and preferences domain, all guidelines scored high on the item values and preferences of patients/populations but received low scores for not addressing the values of target users, decision-makers and guideline developers. AGREE-REX’s positive evaluation of patient values and preferences aligned with GLIA’s positive assessment of the guidelines’ consistency with patient expectations. The appraisals awarded positive scores for most of the nine items in the GLIA Global domain, except for the two items regarding clear definitions of the target audience and setting. Appraisers consistently rated the credibility of the guideline organisation in the Global domain (GLIA) positively across all five guidelines. GLIA’s positive scores for concise recommendations and easy to identify aligned with GLAFI’s favourable ratings for the items succinct and uncomplicated and document layout. However, GLAFI scores also highlighted the need for more actionable writing, echoing GLIA’s negative evaluations regarding the specification of patient and practice characteristics, and insufficient details on how to perform the recommended actions. Comparing appraisals and interviews Both appraisals and interviews agreed on the guidelines’ methodological qualities and helpful infographics, with some concerns about infographic complexity. Both sources also emphasised the need for more specific and actionable recommendations, pointed to feasibility and accessibility issues and noted the absence of a health system perspective as a limitation. The low appraisal scores on values of target users and decision-makers were in line with interviewees’ concerns about barriers stemming from acceptability and cultural appropriateness. Appraisals identified measurability as a challenge. While interviewees partially agreed, they also nuanced that measuring adherence or patient outcomes might be setting-specific and better developed locally. Appraisals evaluated BMJ Rapid Recommendations positively for credibility due to their trustworthiness and publication in a high-impact journal. However, interviewees highlighted challenges with end-users’ preference for trusted medical societies and unfamiliarity with the guideline group. Appraisals and interviews differed on the dimension of novelty. While appraisals rated BMJ Rapid Recommendations positively, interviewees highlighted the lack of compatibility with current practices as a distinct implementation challenge. Discussion Summary of main results BMJ Rapid Recommendations are used in both HICs and LMICs and are often modified to improve fit and feasibility. We found that BMJ Rapid Recommendations are valued for their methodological quality, inclusion of patient perspectives, timely advice and user-friendly infographics. However, we also identified challenges in the categories of perceived trustworthiness, comprehensibility and practical relevance that affect the implementability of BMJ Rapid Recommendations. Our findings also highlight the complex interplay between methodological rigour and perceived credibility, the tension between novelty and compatibility with current practice and the challenge of providing global guidance that remains practical and relevant at the local level. Methodological rigour and source credibility Although appraisers found the guidelines to be credible, interviewees stressed that credibility involves more than methodological rigour. They noted that the involvement of known, trusted organisations or experts is a determinant for guideline use and that strict conflict of interest rules may negatively affect implementability. Novelty versus compatibility Despite positive ratings for novelty in the appraisals, our thematic analysis found challenges related to the introduction of practice-changing recommendations. Such recommendations may conflict with existing guidelines, disrupt the current workflow or require a substantial effort to change practice.20 The specific aim of BMJ Rapid Recommendations—to expedite the transition from new evidence to trustworthy guidelines—may amplify these challenges compared with other international guidelines. While guidelines are essential when facing new practice-changing evidence, guidelines with highly novel practice recommendations may need to pay special attention to implementability issues. Global versus local perspectives Thematic analysis revealed two distinct understandings regarding a guideline’s mission: whether it should focus solely on defining the best possible care or also consider feasibility across diverse settings within countries and internationally. Interviewees highlighted the lack of guidance on adapting recommendations across different contexts, implementing them and addressing unavailable interventions as key implementability challenges. However, they also recognised that providing a standard for best practice can act as an advocacy tool for resourcing, and that providing comprehensive context-specific advice may be overly ambitious for international guidelines. Agreements and disagreements with other studies and frameworks The challenges identified in this study closely align with those reported in various studies concerning the implementability of international guidelines.4 10 15 34 The credibility of a guideline has been shown to affect its likelihood of uptake. In their realist review on implementability, Kastner et al described the key determinants of credibility as the range of interest-holders involved, and transparency regarding all conflicts of interests and funding sources.64 Guideline appraisal tools also emphasise interest-holder involvement and editorial independence as essential components of credibility. We find that this does not fully capture the broader understanding of credibility presented by some of our interviewees. On the other hand, implementation frameworks like the Consolidated Framework for Implementation Research (CFIR) and TICD both present a more comprehensive understanding of credibility by recognising the perceived credibility of the source as a key determinant when describing the innovation/guideline domain.20 65 Notably, a group perceived as highly credible may not necessarily produce high-quality guidelines. For instance, Florez et al’s evaluation of 161 guidelines using AGREE-REX revealed that guidelines developed by professional societies often had lower quality compared with those produced by government-supported organisations.34 In terms of tensions related to novelty, Dedios also highlighted that international recommendations often require new behaviours from clinicians and patients, and the implementability challenges this creates.4 The GUIDE-M also includes novelty as an attribute under feasibility.7 Although the TICD-checklist does not explicitly address the concept of novelty, it includes lack of compatibility as a barrier, exemplified as ‘the recommendation may disrupt the current workflow’.20 Other groups have highlighted the challenges of applying globally relevant guidelines in local contexts, and some are moving to developing resource-stratified or context-stratified recommendations.66,68 Similarly, various strategies are being tested to integrate implementation guidance into international guidelines, indicating that this need has been recognised by others.69,71 In terms of links with implementation frameworks, TICD includes barriers such as feasibility of the clinical intervention, accessibility of the intervention and implementation effort. However, it does not cover adaptation issues among its guideline factors.20 In contrast, CFIR acknowledges adaptability as an important construct within its innovation domain, defining innovation as the ‘thing’ being implemented, including guidelines, and innovation adaptability as ‘the degree to which the innovation can be modified, tailored or refined to fit local context’.65 72 In our appraisal, BMJ Rapid Recommendations scored well on integrating patient values, but suboptimally in incorporating the values of target users, policymakers and guideline developers. A study comparing 161 guidelines using AGREE-REX found that most guidelines scored low on including values, with this trend being most pronounced among international guidelines.34 Common guideline frameworks like AGREE-II, GRADE or GIN standards for trustworthy guidelines do not include the values of users and policy makers. Recent work on ‘time needed to treat’ for physicians exposed to preventive guidelines highlights the importance of including the values and perspectives of target users.73 Still, published guidance remains limited on how to incorporate these values into international guidelines or the potential benefits versus the efforts this would require. Other research has identified specific aspects related to methodological rigour that participants in this study did not mention. Reported challenges include a lack of formal methods for panel selection, difficulties in interdisciplinary collaboration and insufficient consideration of intervention complexity.4 Studies also highlight a tendency to prioritise discussions on benefits and harms, leading to the under-representation of other critical criteria when formulating recommendations.74 75 An over-reliance on expert opinion for such other criteria and limited use of non-randomised evidence has also been identified, alongside tensions between methodological experts and panel members over the applicability of evidence, and difficulties in determining the strength of recommendations.4 74 76 One plausible reason for this difference is that those studies gathered feedback from panel members, whereas we interviewed individuals involved in using the guideline. Strengths and limitations A key strength of this study is its comprehensive approach, addressing challenges in adoption, adaptation, de novo guideline development and implementation, informed by the lived experiences of diverse groups as well as objective appraisals through validated guideline tools. While previous studies have focused on specific aspects or settings, this study offers a broader perspective.4 15 67 77 Our multimethod approach, integrating critical appraisals and interviews, enabled us to uncover additional themes and nuances that a single approach might have missed.3 8 Our design is comparable to that used by Dedios et al, who combined appraisal tools and interviews to assess the adaptability of WHO guidelines, though our study applies this approach to international clinical recommendations and focuses specifically on end-user perspectives.4 While not claiming full saturation, we believe our study offers substantial information power.32 The variability within our sample provided rich material for in-depth analysis, and our findings align with established implementation frameworks. Despite the unique characteristics of BMJ Rapid Recommendations, our results are largely consistent with related studies on the implementability of international guidelines. Our code frequency analysis further suggests data saturation.33 However, BMJ Rapid Recommendations may be more widely used globally than our citation screening and survey reveal. Despite Google Scholar indexing grey literature, we may have missed unpublished applications. We also note the limited number of LMIC participants, whose experiences were largely confined to the BMJ Rapid Recommendation on COVID-19 drugs. The unique circumstances of the COVID-19 pandemic and the collaboration with WHO may not accurately represent other BMJ Rapid Recommendations. The narrow scope of BMJ Rapid Recommendations and their status outside an established guideline body could limit the transferability of our findings to more comprehensive international guidelines from other credible organisations (eg, WHO). The challenge novelty poses to guideline implementability may be particularly relevant for BMJ Rapid Recommendations, given their intent to rapidly translate new, practice-changing evidence into clinical guidance. This may also diminish generalisability to guidelines covering less time-sensitive evidence. While evidence-based practice aims to integrate the best available evidence with clinical expertise and patients’ values and preferences, incorporating new evidence into this triad can be particularly demanding. Novel findings may unsettle established expertise, disrupt clinical routines or misalign with patient expectations, making novelty a barrier worth considering for all guidelines. In our study, we used three distinct appraisal tools—AGREE-REX, GLIA and GLAFI—to appraise the implementability of guidelines. Each tool offers unique insights, yet there is overlap across some domains. This is the first study, to our knowledge, to integrate these three instruments. While each has undergone validation, their combined use has not been systematically explored. The repetition of certain factors across domains may inadvertently create a skewed perspective on their significance. Nonetheless, our approach also provides a more comprehensive evaluation of the implementability challenges than one tool would have provided. Our recruitment efforts were unable to capture participants who may have considered but ultimately refrained from reusing a BMJ Rapid Recommendation, potentially biasing our qualitative results towards more favourable assessments. Lastly, the researchers’ affiliations with MAGIC present a potential limitation. To address this, we collaborated with external researchers and engaged in interest-holder dialogue on the intermediate findings. Implications for practice and further research In an enhanced evidence ecosystem, all actors need to streamline processes and outputs to swiftly turn new evidence into trustworthy international guidelines, as the first step towards ensuring effective adaptation and implementation.14 This should facilitate delivery of actionable guidance to healthcare providers and patients, which addresses the diverse realities that they encounter.11 Our appraisal identified gaps in several implementability domains, suggesting that systematic evaluation of these factors during development could strengthen future guidelines. Using dedicated appraisal tools may help ensure that information critical for implementation is explicitly considered and clearly reported. Beyond methodological rigour, ensuring that guidelines are endorsed or adapted by widely recognised and trusted organisations could further enhance their credibility and uptake. Ensuring that guideline topics target areas of substantial clinical and system-level need may further improve their implementability and relevance to practice. This study did not formally compare implementability issues between HIC and LMIC, but indications of contextual differences emerged. Participants from LMICs uniquely reported challenges such as limited availability of recommended drugs and reduced capacity for adaptation. Future research should examine how resource context influences implementability needs and explore possible interactions between the tensions identified in this study to better understand if and how they interact. While this study shows that the BMJ Rapid Recommendations are being appreciated for being rapid, rigorous and user-friendly, areas for improvement remain. For example, a notable insight emerging indirectly from this study is that the full potential of MAGICapp to facilitate clinical use, adaptation and implementation has not been harnessed within the BMJ Rapid Recommendation process. Providing digitally structured and multilayered content in an online guideline authoring and publication platform allows all relevant content to be available, as opposed to what is possible to present in a scientific publication in PDF format.78 Digitally structured data also facilitate the presentation of the guideline data into accompanying tools to support shared decision making.79 80 This insight has direct implications for practice, as can be seen in a recent BMJ Rapid Recommendation which provides detailed guidance on implementability in the full, online version of the guideline.81 Further research should identify and evaluate strategies that enhance the implementability of guideline products themselves. Although several approaches to improving implementability have already been proposed in the literature, studies are needed to assess which strategies improve trustworthiness, comprehensibility and practical relevance, and to weigh their resource demands for development teams. To advance this work further, we will conduct a second study to explore strategies for enhancing the implementability of international guidelines across diverse healthcare settings and resource levels, while also considering the burden on guideline panels and implementers. Conclusions The extent to which international guidelines are used for adoption, adaptation, de novo guideline development and/or implementation and the challenges encountered often remain unclear. This study took a novel approach by focusing on the end-users’ perspectives of BMJ Rapid Recommendations, a type of international guideline, highlighting how efforts to improve implementability must grapple with three key tensions: methodological rigour versus perceived credibility, novelty versus compatibility and global ideals versus local feasibility. Although not all implementability challenges can be solved by those developing international guidelines, enhancing implementability may remain an elusive goal if such tensions are not acknowledged and reflected on during guideline development. Supplementary material Footnotes Funding: The E-3 project is funded by the South-Eastern Norway Regional Health Authority, project nr: 2021060, and supports a designated PhD and researcher. The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report or decision to submit the article for publication. Prepub: Prepublication history and additional supplemental material for this paper are available online. To view these files, please visit the journal online (https://doi.org/10.1136/bmjopen-2025-114609). Provenance and peer review: Not commissioned; externally peer reviewed. Patient consent for publication: Not applicable. Ethics approval: This study involves human participants and was approved by the Data Protection Officer at Lovisenberg Hospital, Oslo, Norway, ref. number 2022/2601. REK approval was not required under Norwegian law. Map disclaimer: The depiction of boundaries on this map does not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area or of its authorities. This map is provided without any warranty of any kind, either express or implied. Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details. Data availability statement Due to ethical and privacy concerns, raw data from interviews cannot be made available. However, quotations from the transcriptions have been used in the results section to support the findings. Data from the critical appraisal can be shared upon request. References Supplementary Materials Data Availability Statement Due to ethical and privacy concerns, raw data from interviews cannot be made available. However, quotations from the transcriptions have been used in the results section to support the findings. Data from the critical appraisal can be shared upon request.