Expanding methods to address RE-AIM metrics in hybrid effectiveness-implementation studies Authors: Samantha M. Harden, Karla I. Galaviz, Paul A. Estabrooks Published: 2024 ABSTRACT BackgroundDissemination and implementation science is an evolving field that focuses on the strategies and mechanisms by which scientific evidence is adopted, used, and sustained in clinical and community practice.Main bodyImplementation scientists are confronted by the challenge to balance rigor and generalizability in their work while also attempting to speed the translation of evidence into clinical and community practice. Hybrid Effectiveness-Implementation studies and the RE-AIM framework were conceptualized to address these challenges. Hybrid Effectiveness-Implementation (HEI) studies provide methods of examining the effectiveness of health promoting interventions while concurrently assessing the utility of dissemination and implementation strategies designed to enhance the application of evidence-based principles in practice. RE-AIM provides a set of planning and evaluation dimensions that can be assessed with a goal to balance internal and external validity. The purpose of this commentary is to provide clarity on definitions of each approach and how to effectively use them together to answer research questions that will advance dissemination and implementation science for health promotion.ConclusionsWe provide examples of concerted use of RE-AIM within HEI studies from the literature and focus on language to provide a clarity and consistency across research questions, designs, and settings. We share how to operationalize RE-AIM dimensions in HEI studies for both dissemination and implementation strategies. Future directions include refining, defining, and evaluating each RE-AIM dimension within hybrid studies.Supplementary InformationThe online version contains supplementary material available at 10.1186/s43058-024-00646-0. The RE-AIM framework was introduced to evaluate the public health impact of health promotion interventions by considering dimensions of reach, effectiveness, adoption, implementation, and maintenance. Hybrid Effectiveness-Implementation (HEI) studies were introduced as a typology to speed the translation of evidence-based interventions into practice.Due to the complexity of constructs and definitions, there are inconsistencies and lack of clarity on how to operationalize RE-AIM outcomes across HEI studies.This commentary includes guidance on applying the RE-AIM framework within HEI studies with specific examples from health promotion interventions. The focus of this guidance is based on key research questions and design considerations. The RE-AIM framework was introduced to evaluate the public health impact of health promotion interventions by considering dimensions of reach, effectiveness, adoption, implementation, and maintenance. Hybrid Effectiveness-Implementation (HEI) studies were introduced as a typology to speed the translation of evidence-based interventions into practice. The RE-AIM framework was introduced to evaluate the public health impact of health promotion interventions by considering dimensions of reach, effectiveness, adoption, implementation, and maintenance. Hybrid Effectiveness-Implementation (HEI) studies were introduced as a typology to speed the translation of evidence-based interventions into practice. Due to the complexity of constructs and definitions, there are inconsistencies and lack of clarity on how to operationalize RE-AIM outcomes across HEI studies. Due to the complexity of constructs and definitions, there are inconsistencies and lack of clarity on how to operationalize RE-AIM outcomes across HEI studies. This commentary includes guidance on applying the RE-AIM framework within HEI studies with specific examples from health promotion interventions. The focus of this guidance is based on key research questions and design considerations. This commentary includes guidance on applying the RE-AIM framework within HEI studies with specific examples from health promotion interventions. The focus of this guidance is based on key research questions and design considerations. Introduction Dissemination and implementation science is defined as the study of the strategies and mechanisms by which scientific evidence is disseminated and implemented in community or clinical settings to improve outcomes for a specified population [1, 2]. This can include developing and testing innovative approaches to identify, understand, and overcome barriers to the adoption, adaptation, integration, scale-up and sustainability of evidence-based interventions, tools, policies, and guidelines. Over the past 25 years fundamental challenges in dissemination and implementation science have centered on the need to balance internal and external validity, assess a broad range of outcomes, and speed the translation of evidence into clinical and community practice settings [3]. There remains a substantial translational lag time in evidence-based interventions that promote health such as those for improving physical activity, tobacco cessation, healthful eating, and weight control [4–13]. Therefore, dissemination and implementation researchers have worked to develop program, practice, and policy interventions that address (a) multi-leveled outcomes, (b) clinical and community infrastructure, (c) implementation staff characteristics and perceptions, and (d) the characteristics and perceptions of the population intended to achieve a health benefit—all while attempting to rule out alternative explanations for outcomes within complicated and complex clinical and community systems [14]. Two approaches were independently developed to plan, intervene upon, and evaluate dissemination and implementation strategies concurrently with effectiveness outcomes. These approaches are the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation, and Maintenance) and Hybrid Effectiveness-Implementation (HEI) studies [3, 15]. Both RE-AIM and HEI approaches have been widely used together, given their complementary methodological strengths. A PubMed search using the words “RE-AIM” and “Hybrid Effectiveness Implementation” yielded 322 manuscripts in the 10 years since the initial paper on HEI studies was published (2012–2022). Similarly, a search in the National Institutes of Health Reporter for 2019–2022 yielded 95 active projects that use RE-AIM to test health interventions within HEI studies across a wide range of behavioral interventions. While these two approaches have previously been used in combination [14], there remains inconsistencies and lack of clarity on how to operationalize RE-AIM outcomes across HEI study types [16]. Therefore, the purpose of this conceptual paper is to provide clarity on definitions of each approach and how to effectively use them together to answer significant research questions that will advance dissemination and implementation science and health promotion. Based on our areas of expertise, we highlight nutrition, physical activity, and obesity studies throughout this manuscript. The RE-AIM framework RE-AIM provides a set of planning and evaluation outcomes that can be assessed with a goal to balance internal and external validity while concurrently providing data that can be used to predict or determine the public health impact of interventions when delivered in real-world settings [17]. RE-AIM was developed to address the multi-leveled nature of dissemination and implementation outcomes with a dual focus on assessing health benefits as well as intervention delivery by staff and organizations [3]. Specifically, reach is defined as the absolute number, proportion, and representativeness of individuals who are willing to participate in the intervention relative to the broader eligible population (i.e., beneficiaries). Effectiveness is the impact of the intervention on the intended health outcomes for a given sample or population. This includes potential negative effects, effects on quality of life, and effects across subgroups. Adoption is the absolute number, proportion, and representativeness of settings and intervention delivery agents who are willing to initiate a program, relative to the broader eligible population of staff and settings. Implementation refers to the degree to which the intervention is delivered as intended: is the fidelity to the various elements of an intervention’s key functions or components, including consistency of delivery as intended and the time and cost of the intervention. Maintenance at the individual level is the sustained change in health/behavioral outcomes targeted by the intervention, while Maintenance at the organizational level is the sustained implementation of the evidence-based interventions as intended. Researchers using the RE-AIM framework were (and are) encouraged to test scientific questions in contexts that reflect what would occur in typical clinical or community practice [3]. Conceptualizing HEI studies HEI studies address research questions about the effectiveness of an intervention and its implementation within a single study [18]. HEI approaches distinguish between two types of interventions—one that focuses on the beneficiary’s health outcomes and one that focuses on dissemination and implementation outcomes. To distinguish these two types of interventions, with the context of HEI studies, we refer to those that target beneficiary health outcomes as the health promoting intervention while those that target dissemination and implementation outcomes as to strategies [19]. For example, considering physical activity promotion, HEI studies can focus on assessing the effects of an intervention on physical activity behavior while concurrently assessing the implementation context or the utility of implementation strategies designed to enhance the delivery of the intervention in a specific settings [15]. The main goal of HEI studies is to help determine whether a study should focus primarily on intervention effectiveness while exploring implementation factors, or primarily on implementation quality while measuring health outcomes, or both [18]. Indeed the three most common types of HEI studies focus on the following: (a) testing the effects of an intervention on health outcomes (primary outcome), while gathering information on intervention implementation (Type 1); (b) testing the effects of strategies on adoption, implementation, or sustainability of an evidence-based intervention (primary outcome), while measuring health outcomes (Type 3); and testing the effects on an intervention on health outcomes and the effects of a strategy on dissemination and implementation outcomes (co-primary outcomes; Type 2). While there are more variations of HEI studies [18], and these may also be expanded to include context as an additional independent variable [20], this discussion will focus on the three most common types described in the seminal HEI paper. Distinguishing evidence-based interventions and dissemination and implementation strategies A fundamental need in the application of HEI studies is to clearly specify and distinguish an intervention intended to result in health improvements among beneficiaries versus dissemination and implementation strategies applied at the level of setting, organization, and/or staff with a goal to improve adoption, implementation, and maintenance [21]. A functional starting point is to consider the intervention as an independent variable in the study that include comparison conditions with hypotheses related to improvements in participant health outcomes. This includes, among other approaches, theory-based interventions that target social cognitions [22] or group dynamics principles [23] to promote health behaviors in the beneficiaries that are exposed to a given intervention. Dissemination and implementation science also uses the term evidence-based intervention to highlight that in addition to being hypothesized to improve health outcomes, that the intervention has demonstrated efficacy in a single large trial or across a series of trials summarized in a review of literature [1]. Dissemination and implementation strategies can also be described as the independent variable or an observation in an HEI study that are hypothesized to directly produce or describe improvements in intervention adoption, implementation, and sustainability. Dissemination and implementation strategies include activities that address physical activity promotion indirectly by focusing on improving implementation quality and improving the fit between the delivery setting and the behavioral intervention. In general, strategies can be further described as those that focus on planning efforts (e.g., gathering data, building support, and developing partnerships in the delivery setting), educational approaches (e.g., sharing information on the evidence-based intervention), system adaptations (e.g., changes to implementer workflow), quality improvement (e.g., training staff), the use of economic incentives (e.g., incentives based on performance standards), and addressing policy (e.g., guideline development and application) [24]. Expert recommendations exist to support researchers in selecting and specifying dissemination and implementation strategies [25, 26]. The intersection of RE-AIM framework and HEI studies Considering the intended outcome of an evidence-based intervention or dissemination and implementation strategy, based on the RE-AIM dimensions, can also be helpful. Specifically, any activities that are intended to improve reach or adoption are dissemination strategies. In each case, the strategies are focused on the spread of an intervention to the population intended to see a health benefit (i.e., reach) or to the population of staff and settings intended to deliver the intervention (i.e., adoption). Activities that are intended to improve implementation and sustainability are considered implementation strategies. In these cases, strategies are focused on the quality of delivery from initial application by specific staff in a given setting (i.e., implementation) or on sustained quality of delivery (i.e., maintenance). Finally, activities intended to improve intervention health outcomes or the degree to which beneficiaries sustain those improvements are evidence-based, or health promoting, interventions. We acknowledge that there can be some nuances in distinguishing between interventions and strategies to change specific RE-AIM outcomes. Prior to the introduction of HEI designs and terminology, behavioral research was (and is) rich with examples of multi-level interventions that have historically focused on behavior change through delivery agent training [27], engagement of organizational leadership [28], systems integration [29], and partnership with community-based organizations [30]. These multilevel interventions typically include components that intend to directly improve beneficiary’s behavioral outcomes (i.e., the evidence-based intervention within HEI approaches) and indirectly improve behavioral outcomes through changes to the environment or improvements in implementation fidelity (i.e., a dissemination and implementation strategy within HEI approaches). To improve the ease of description of HEI study types that use RE-AIM, mapping processes can help to provide clarity during preparation, execution, and interpretation of a study [31]. This can help avoid issues related to underreporting or misreporting in of reach, adoption, cost, and setting level maintenance, or misclassification of reach and adoption indicators [16, 32]. These issues are often exacerbated in HEI studies due to the need to identify appropriate denominators for reach and adoption, distinguish between individual- and setting-level outcomes, and distinguish between health promoting interventions and dissemination and implementation strategies. Further, most dissemination strategies (i.e., intended to improve reach or adoption) inaccurately labeled as implementation strategies. Table 1RE-AIM operationalization across HEI study typesTYPE 1 HEIReachEffectivenessAdoptionImplementationMaintenance-iMaintenance-oIntervention testedEBIEBIEBIEBIEBIEBIUnit of analysisEBI beneficiaryEBI beneficiaryStaff / settingStaff / settingEBI beneficiaryStaff / settingIs this a primary outcome?YesYesNoNoYesNoIs this a secondary outcome?NoNoYesYesNoYesTYPE 2 HEIReachEffectivenessAdoptionImplementationMaintenance-iMaintenance-oIntervention testedEBI and D&I StrategyEBI and D&I StrategyEBI and D&I StrategyEBI and D&I StrategyEBI and D&I StrategyEBI and D&I StrategyUnit of analysisEBI beneficiaryEBI beneficiaryStaff / settingStaff / settingEBI beneficiaryStaff / settingIs this a primary outcome?YesYesYesYesYesYesIs this a secondary outcome?NoNoNoNoNoNoTYPE 3 HEIReachEffectivenessAdoptionImplementationMaintenance-iMaintenance-oIntervention testedD&IStrategyD&IStrategyD&IStrategyD&IStrategyD&IStrategyD&IStrategyUnit of analysisEBI beneficiaryEBI beneficiaryStaff / settingStaff / settingEBI beneficiaryStaff / settingIs this a primary outcome?YesNoYesYesNoYesIs this a secondary outcome?NoYesNoNoYesNoEBI evidence-based intervention; D&I Strategy Dissemination and/or Implementation Strategy; Maintenance-i or -o individual and o = organizational, respectively Table 1RE-AIM operationalization across HEI study typesTYPE 1 HEIReachEffectivenessAdoptionImplementationMaintenance-iMaintenance-oIntervention testedEBIEBIEBIEBIEBIEBIUnit of analysisEBI beneficiaryEBI beneficiaryStaff / settingStaff / settingEBI beneficiaryStaff / settingIs this a primary outcome?YesYesNoNoYesNoIs this a secondary outcome?NoNoYesYesNoYesTYPE 2 HEIReachEffectivenessAdoptionImplementationMaintenance-iMaintenance-oIntervention testedEBI and D&I StrategyEBI and D&I StrategyEBI and D&I StrategyEBI and D&I StrategyEBI and D&I StrategyEBI and D&I StrategyUnit of analysisEBI beneficiaryEBI beneficiaryStaff / settingStaff / settingEBI beneficiaryStaff / settingIs this a primary outcome?YesYesYesYesYesYesIs this a secondary outcome?NoNoNoNoNoNoTYPE 3 HEIReachEffectivenessAdoptionImplementationMaintenance-iMaintenance-oIntervention testedD&IStrategyD&IStrategyD&IStrategyD&IStrategyD&IStrategyD&IStrategyUnit of analysisEBI beneficiaryEBI beneficiaryStaff / settingStaff / settingEBI beneficiaryStaff / settingIs this a primary outcome?YesNoYesYesNoYesIs this a secondary outcome?NoYesNoNoYesNoEBI evidence-based intervention; D&I Strategy Dissemination and/or Implementation Strategy; Maintenance-i or -o individual and o = organizational, respectively RE-AIM operationalization across HEI study types D&I Strategy D&I Strategy D&I Strategy D&I Strategy D&I Strategy D&I Strategy EBI evidence-based intervention; D&I Strategy Dissemination and/or Implementation Strategy; Maintenance-i or -o individual and o = organizational, respectively Specifically, regardless of HEI type, reach, effectiveness, and maintenance of health effects are assessed at the level of the beneficiary –with effectiveness and maintenance shifting from a primary, to a co-primary, to a secondary outcome, in a HEI type 1, 2 and 3, respectively. Similarly, adoption, implementation, and maintenance are assessed at the level of the staff, setting, and/or system that is delivering the evidence-based intervention within their organization. These outcomes also shift from a secondary aim, a co-primary aim, and a primary aim in the HEI 1, 2 and 3, respectively. Of note, while assessed at different levels, reach and adoption are dissemination outcomes, while staff, setting, and system implementation and organizational maintenance are implementation outcomes. There are several examples of published studies that have employed RE-AIM in HEI studies. Examples of the use of RE-AIM within HEI type 1 studies include examining sugar-sweetened beverage reduction interventions in US adults and children curran [33, 34], and testing physical activity interventions among cancer survivors in Canada [35]. RE-AIM has also been used in type 2 HEI studies testing interventions to improve clinician counseling and patient physical activity in Mexico [36], to improve physical activity among older adults in Canada [37], and to improve young children’s eating and health behaviors in the US [38]. Finally, RE-AIM has been used in type 3 HEI studies to assess the effects of diabetes prevention programs in the Veteran Affairs health system [39, 40], and the effects of interventions on young children’s eating and physical activity behaviors [41, 42]. To better illustrate the joint application of these two approaches, we include detailed examples of studies using RE-AIM and HEI studies in nutrition, physical activity, and obesity [42–45] in Table 2. Across studies, RE-AIM outcomes are operationalized inconsistently within each HEI study type, and there is lack of clarity regarding the intervention/strategy being evaluated and the primary outcome(s) it targets. Table 2Example of current RE-AIM operationalizations within HEI study typesType 1 HEI(RAINBOW; 43)Type 2 HEI(FitEx; 44,45)Type 3 HEI(Go NAPSACC; 42)Study goalAssess the reach and effectiveness of an intervention designed to reduce body weight and depressive symptoms in adults, while gathering information on intervention, adoption, implementation and maintenance.Compare the effects of various training modalities (in person, hybrid, on demand) on intervention fidelity by delivery personnel and assess the effects of the program on beneficiary physical activity behaviors.Compare the effects of basic versus enhanced implementation support on centers’ use of healthy eating and physical activity best practices, while gathering information on children’s eating and physical activity behaviors.Intervention testedEBIEBI and D&Ia strategyD&I strategyUnit of analysisIntervention beneficiariesIntervention beneficiaries and delivery staff / settingIntervention delivery staff/settingPrimary outcomeBeneficiary BMI and SCL-20 scorePatient Physical Activity and Training AdaptationsCenter’s use of healthy eating and physical activity best practicesReachNumber, proportion, and representativeness of beneficiaries exposed to the intervention relative to the intended audienceNumber, proportion, and representativeness of physically inactive patients that agree to participateNumber, proportion, and representativeness of children that agree to participateEffectivenessChanges in BMI and SCL-20 score from baseline to 12 monthsProportion of patients meeting physical activity guidelines before and after the interventionSecondary outcome: children’s eating and physical activity behaviorsAdoptionNumber, proportion, and representativeness of settings and staff (lifestyle coach) willing to initiate RAINBOWNumber, proportion, and representativeness of staff and counties willing to initiate either FitEx or control program (ALED)Number, proportion, and representativeness of child care centers that agree to initiate the Go NAPSACCImplementationBeneficiary’s intervention dose received, fidelity of intervention sessions, and costs of intervention deliveryFidelity to FitEx core elements or ALED manualized program by delivery staffFidelity of Go NAPSACC components by child care centersMaintenancea) Attrition rates by beneficiary characteristics and maintenance of intervention effectsb) Qualitative assessment of intervention integration and sustainability in routine carea) Beneficiary physical activity post program not regularly monitored in ongoing work due to established long-term impact from original quasi experimental designb) Continued delivery of FitEx; control condition (ALED) discontinued outside of adoption trial.a) Child care center continued implementation of Go NAPSACC 6, 12, and 24 months post-interventionb) Coaches continued implementation of Go NAPSACC 6, 12, and 24 months post-interventionNote: HEI 1 and 3 are exemplar operational definitions with outcome data from the literature. The example for the RE-AIM HEI 2: We found protocol papers for type 2 HEI for health promotion, but no outcome papers to date. We use FitEx v ALED as example data, although they are not explicitly listed as HEI 2.aD&I Dissemination and Implementation Table 2Example of current RE-AIM operationalizations within HEI study typesType 1 HEI(RAINBOW; 43)Type 2 HEI(FitEx; 44,45)Type 3 HEI(Go NAPSACC; 42)Study goalAssess the reach and effectiveness of an intervention designed to reduce body weight and depressive symptoms in adults, while gathering information on intervention, adoption, implementation and maintenance.Compare the effects of various training modalities (in person, hybrid, on demand) on intervention fidelity by delivery personnel and assess the effects of the program on beneficiary physical activity behaviors.Compare the effects of basic versus enhanced implementation support on centers’ use of healthy eating and physical activity best practices, while gathering information on children’s eating and physical activity behaviors.Intervention testedEBIEBI and D&Ia strategyD&I strategyUnit of analysisIntervention beneficiariesIntervention beneficiaries and delivery staff / settingIntervention delivery staff/settingPrimary outcomeBeneficiary BMI and SCL-20 scorePatient Physical Activity and Training AdaptationsCenter’s use of healthy eating and physical activity best practicesReachNumber, proportion, and representativeness of beneficiaries exposed to the intervention relative to the intended audienceNumber, proportion, and representativeness of physically inactive patients that agree to participateNumber, proportion, and representativeness of children that agree to participateEffectivenessChanges in BMI and SCL-20 score from baseline to 12 monthsProportion of patients meeting physical activity guidelines before and after the interventionSecondary outcome: children’s eating and physical activity behaviorsAdoptionNumber, proportion, and representativeness of settings and staff (lifestyle coach) willing to initiate RAINBOWNumber, proportion, and representativeness of staff and counties willing to initiate either FitEx or control program (ALED)Number, proportion, and representativeness of child care centers that agree to initiate the Go NAPSACCImplementationBeneficiary’s intervention dose received, fidelity of intervention sessions, and costs of intervention deliveryFidelity to FitEx core elements or ALED manualized program by delivery staffFidelity of Go NAPSACC components by child care centersMaintenancea) Attrition rates by beneficiary characteristics and maintenance of intervention effectsb) Qualitative assessment of intervention integration and sustainability in routine carea) Beneficiary physical activity post program not regularly monitored in ongoing work due to established long-term impact from original quasi experimental designb) Continued delivery of FitEx; control condition (ALED) discontinued outside of adoption trial.a) Child care center continued implementation of Go NAPSACC 6, 12, and 24 months post-interventionb) Coaches continued implementation of Go NAPSACC 6, 12, and 24 months post-interventionNote: HEI 1 and 3 are exemplar operational definitions with outcome data from the literature. The example for the RE-AIM HEI 2: We found protocol papers for type 2 HEI for health promotion, but no outcome papers to date. We use FitEx v ALED as example data, although they are not explicitly listed as HEI 2.aD&I Dissemination and Implementation Example of current RE-AIM operationalizations within HEI study types a) Attrition rates by beneficiary characteristics and maintenance of intervention effects b) Qualitative assessment of intervention integration and sustainability in routine care a) Beneficiary physical activity post program not regularly monitored in ongoing work due to established long-term impact from original quasi experimental design b) Continued delivery of FitEx; control condition (ALED) discontinued outside of adoption trial. a) Child care center continued implementation of Go NAPSACC 6, 12, and 24 months post-intervention b) Coaches continued implementation of Go NAPSACC 6, 12, and 24 months post-intervention Note: HEI 1 and 3 are exemplar operational definitions with outcome data from the literature. The example for the RE-AIM HEI 2: We found protocol papers for type 2 HEI for health promotion, but no outcome papers to date. We use FitEx v ALED as example data, although they are not explicitly listed as HEI 2. aD&I Dissemination and Implementation The way forward This manuscript aims to provide clear definitions and guidance on how to operationalize RE-AIM outcomes in HEI studies. We also offer new types of HEI studies anchored on specific RE-AIM outcomes and try to guide this anchoring process using a subway map analogy expanded from Lane-Fall et al., [46] (Fig. 1). First, consider that reach and adoption reflect dissemination outcomes, while implementation and maintenance at the staff/organization/system level reflect implementation outcomes. Hence, a study can focus on examining adoption of an evidence-based intervention (i.e., dissemination outcome) or the fidelity of an evidence-based intervention (implementation outcome). Depending on the dissemination and implementation strategy employed and the target outcomes, type 2 and 3 hybrids studies might include Effectiveness and one or more of the other four dimensions of RE-AIM: reach, adoption, implementation, maintenance (Fig. 1). Fig. 1"Subway" schematic for design and outcome considerations based on hybrid effectiveness-implementation studies and the RE-AIM framework "Subway" schematic for design and outcome considerations based on hybrid effectiveness-implementation studies and the RE-AIM framework For instance, if the focus of a study is to determine the effects of a dissemination strategy on intervention reach, a hybrid effectiveness-reach study type may be used. This study would test whether such strategy (e.g., mass media campaign) improves the participation rate and representativeness of the target beneficiary population. If the focus of the study is to increase the participation and representativeness of the staff delivering the evidence-based intervention, that may be a hybrid effectiveness-adoption study. Alternatively, if the goal is to investigate the influence of specific dissemination and implementation strategies have on the evidence-based intervention longevity within a system, that would be considered a hybrid effectiveness-maintenance study. To better illustrate this, we offer a subway map to help visualize the process of selecting the most appropriate RE-AIM-HEI study type for the research question of interest. The subway line starts at the HEI stop of the Lane-Fall et al. subway map [46]. The primary outcome should guide the path taken down the subway map. If the primary outcome is intervention effectiveness (e.g., impact on health/behavioral outcomes) and the secondary outcome is a dissemination or implementation outcome, the HEI 1 lane should be followed and RE-AIM operationalized as per usual (see Table 1 for guidance on this). If the primary outcomes include a health/behavioral outcome and a dissemination or implementation outcome, the HEI 2 lane is followed. If the primary outcome is a dissemination or implementation outcome, and the secondary outcome a health/behavioral outcome, the HEI 3 lane is followed. HEI 2 and 3 lines further splinter to reflect the different RE-AIM hybrid studies we previously described depending on the focus of the primary dissemination and implementation outcome. Researchers and practitioners can use this subway map to determine which RE-AIM hybrid study is most appropriate to address their dissemination or implementation research question. Notably, Curran and colleagues stated that the HEI “definition and typology offered [in their manuscript] must be considered constructs still in evolution.” [15] This has been evident over the last ten years of applying HEI [18]. We anticipate that the complexity and application of the RE-AIM Framework and HEI study types will continue to evolve, too. For example, while less prominent in the literature to date, investigators may be interested in a hybrid effectiveness-reach study. Further still, depending on the study, a team may be using reach or effectiveness as the primary, co-primary or secondary outcome. We provided a subway illustration for these evolving study types for your consideration as well; where HeR indicates reach is a primary outcome and effectiveness a secondary outcome; HER indicates effectiveness and reach as co primary; and HEr indicates effectiveness as primary and reach as secondary. See Supplemental Figure. In sum, RE-AIM can be operationalized in all three types of HEI studies. The outcome specification approach we propose here would promote consistency in implementation research by facilitating our understanding of the impact, generalizability, and sustainability not only of evidence-based interventions, but also of dissemination and implementation strategies. As RE-AIM continues to evolve, it is well positioned to address the realist evaluation question of what interventions are effective, with which dissemination and implementation strategies, for whom, in what settings, how and why, and for how long [3, 16]. Conclusion This manuscript provides recommendations for using RE-AIM within HEI study types, and offers a new typology of hybrid studies anchored on RE-AIM outcomes. A general concern for dissemination and implementation researchers is that there is confusion about dissemination and implementation as an action versus dissemination and implementation as a science. The “science” designation implies that there is a research question, research design, and research methodologies used to understand a specific phenomenon. Frameworks such as RE-AIM and study types such as HEI are just one way to contribute to the science of implementing health interventions. Empirical and practical gaps still remain in the field and novel ways to approach them, such as the approach offered here, should continue to be developed, challenged and refined. Supplementary Information Supplementary Material 1. Supplementary Material 1.