Intensive capacity building in implementation science: an evaluation of the University of Washington Implementation Science Summer Institute Authors: Aneth Dinis, Emma Murphy, Kenneth Sherr, Rishika Mohanty, Arianna R Means Published: 2025 ABSTRACT AbstractBackgroundThe demand for implementation science training far exceeds supply, indicating a need to expand capacity building efforts. Since 2014, the Implementation Science Summer Institute at the University of Washington (ISSI-UW) has provided an annual intensive training program to strengthen capacity in implementation science. In this article, we describe an evaluation of the program to determine its impact on short and long-term outcomes.MethodsWe used the Kirkpatrick model to guide the evaluation. We administered pre- and post-surveys with 2024 trainees to evaluate changes in implementation science knowledge and skills, intent to use methods and tools, and overall satisfaction. A survey of alumni from 2014 to 2024 evaluated post-training engagement in implementation science-related activities, the course’s impact on alumni work, and its effect on professional networks. Surveys included both closed- and open-ended questions. Descriptive statistics were used to analyze demographic characteristics and present Likert-type response frequencies. Wilcoxon rank-sum tests were used to compare median differences across surveys and to determine statistical significance. Qualitative content analysis was conducted to examine open-ended responses.ResultsOf the 477 individuals from 48 countries (69% were low and middle-income countries) trained at the institute between 2014 and 2024 who were invited to participate in the alumni survey, 136 (28%) responded. Of the 72 individuals invited to participate in the 2024 pre- and post-surveys, 54 (75%) and 51 (70%) responded, respectively. Participants included global health researchers, implementers, graduate students, and donors. Findings from the 2024 pre- and post-surveys showed a significant improvement in the median responses for knowledge, skills, and intent, and respondents expressed high satisfaction. Alumni reported incorporating skills related to implementation science theories, models, and frameworks (84%), design of implementation evaluations (52%), and writing grants (50%) into their work. Alumni reported that the course had a very large (21%) or large influence on their work (32%) and noted a positive impact on networking (66%).ConclusionsThe ISSI-UW was associated with short-term improvements in implementation science knowledge and longer-term integration of implementation science methods and tools in professional work. This is an example of how universities can contribute to expanding and building capacity in implementation research and practice.Supplementary InformationThe online version contains supplementary material available at 10.1186/s43058-025-00838-2.Keywords: Implementation science, Capacity building, Training evaluation, Global health The evaluation describes how an intensive implementation science training program can strengthen capacity and sustain positive post-training results. The evaluation describes how an intensive implementation science training program can strengthen capacity and sustain positive post-training results. The training emphasizes a global health perspective, attracting participants from various backgrounds and settings. This initiative contributes to closing the capacity-building gap in implementation science training among different stakeholders in research and practice. The training emphasizes a global health perspective, attracting participants from various backgrounds and settings. This initiative contributes to closing the capacity-building gap in implementation science training among different stakeholders in research and practice. By using a well-known evaluation framework and triangulating findings from closed and open-ended questions, this evaluation exemplifies how we can use simple methods to conduct replicable evaluations in implementation science training. By using a well-known evaluation framework and triangulating findings from closed and open-ended questions, this evaluation exemplifies how we can use simple methods to conduct replicable evaluations in implementation science training. Background Scientific discoveries should be applied where they are most needed for society to achieve a complete return on investments made in research. However, a substantial gap remains in translating research findings into practice, partly due to the lengthy research pipeline [1]. A promising approach to bridging this research-to-practice gap is implementation science, which enhances knowledge about adopting and integrating evidence into practice. Implementation research is defined as the scientific study of methods to promote the systematic adoption of proven clinical treatments, practices, and organizational and management interventions into routine use [1]. In other words, implementation science aims to close know-do gaps; that is, gaps between what we know works and what we do in practice [2]. The field of implementation science has grown tremendously worldwide over the past two decades, indicating the need for increased capacity of implementers, policymakers, and institutions conducting research [3]. Implementation science education involves equipping health professionals with various levels of knowledge and skills to systematically apply evidence-based findings in practice. While some individuals may benefit from a basic exposure to IS concepts and terminology, others—such as researchers or implementation practitioners—require more advanced competencies to design, conduct, and evaluate implementation efforts. Implementation science has become a highly relevant topic in health sciences education due to its role in equipping health professionals with the knowledge and practical skills to navigate the complex systems that influence healthcare delivery, including organizational culture, policy environments, and stakeholder dynamics [4, 5]. Additionally, its emphasis on context-specific strategies also supports the development of equitable interventions tailored to the needs of diverse populations [6]. Current efforts to strengthen capacity in implementation science take many forms, including academic and government institutions, centers and departments dedicated to the field, organizing scientific conferences and meetings, and developing and providing training programs [7]. Despite substantial growth in the number of training opportunities, demand for training in implementation science still far exceeds supply, especially in low- and middle-income countries (LMICs), where the burden of preventable disease is highest but there are fewer implementation science training programs [8, 9]. This mismatch between supply and demand highlights the need for an expanded capacity-building approach that involves implementation partners across various fields [8, 10] and includes increased engagement with implementation practitioners and policymakers [11]. Moreover, most implementation science courses primarily emphasize implementation research, but there is also a need for increased education in implementation practice. Furthermore, few implementation science capacity-building programs leverage existing partnerships to create collaborative learning networks to support implementation science approaches across practice settings [11]. A scoping review of academic initiatives to advance implementation practice identified that most programs rely on structured didactic instruction, are offered in person or online, use expert consultation and mentorship to support implementation, and include practical application activities. The target audience is primarily health or behavioral health researchers and practitioners, and all studies reported favorable short-term outcomes [11]. Another mapping review in dissemination, implementation and translation science noted heterogeneity in the competencies targeted, limited consistency in the use of frameworks to guide capacity building, most trainings were conducted in the USA and then in Canada, the length of trainings ranged from two days to two years, and it was noted frequent absence of explicit health equity orientation in capacity-building programs [12]. Interestingly, one of the most used tools in implementation science is theories, models, and frameworks, yet theoretical concepts guide few capacity-building programs in implementation science [11, 12]. The primary goal of any capacity-building program is to achieve long-term, sustainable impact across various contexts, individuals, and teams. Implementation science-focused capacity-building programs further aim to increase the number of evidence-based interventions implemented and evaluated while reducing the time needed for integration into real-world practice. While the logic for capacity building is broadly accepted, important gaps remain. It’s unclear which modalities are most effective in particular contexts; there is a need to develop competencies aligned with LMIC contexts; few programs rigorously assess downstream institutional or systems impact; and adaptation to resource-constrained settings is underexplored. Building capacity in implementation science drives institutional and policy transformation by embedding evidence-informed decision-making into organizational systems. As more professionals gain implementation science skills, institutions become better at identifying barriers, adapting interventions, and using data for continuous improvement. This collective expertise fosters learning-oriented cultures and informs policies that accelerate the translation of research into sustainable, scalable, and contextually relevant programs. In this study, the Implementation Science Summer Institute at the University of Washington (ISSI-UW) seeks to share its experience in building capacity in implementation science and address some of the gaps cited above. We describe ISSI-UW’s eleven years of experience training participants from around the globe in implementation science. Our assessment has two main objectives. The first is to evaluate short-term outcomes, including changes in implementation science knowledge, skills, intent to use methods and tools, and overall satisfaction. The second is to assess long-term outcomes, including alumni self-reported professional network expansion and post-training engagement in implementation science-related activities. Methods Course description The ISSI-UW is a one-week in-person course launched in 2014 at the University of Washington. It is led by professors from the Department of Global Health Implementation Science Program and guest lecturers with expertise in specific methodological topics. The goal of the course is to provide participants with an intensive introduction to implementation science. While it has varied over the years, the most recent iteration of the course included 43.5 h of instruction. Since its inception, the ISSI-UW has provided introductory content to individuals new to implementation science. Participants review the history of implementation science as a formal discipline, learn about core scientific methods and tools, and use case studies to identify and examine implementation successes and failures. In 2023, the ISSI-UW launched an additional advanced training option given the evolving needs of implementation researchers and practitioners. The advanced training option is available to students who have already participated in the ISSI-UW or other introductory implementation science training opportunities. Participants in the advanced training program learn about complex applications of implementation science methods through readings, discussions, and application exercises. An overview of curriculum topics in the introductory (fundamentals) and advanced tracks is presented in additional file 1. The focus and depth of topics taught in the course have changed over the years in response to advances in the field of implementation science. Despite these changes, the course emphasizes teaching scientific methods for identifying implementation gaps and strategies for improvement, designing studies, and conducting process and impact evaluations. This is achieved by mixing didactic material with content that demonstrates the application of implementation science methods and tools in practice. Course content includes interactive lectures, case studies, readings, and group discussions from a global health perspective. There is an effort to show participants real-world examples of how implementation science has been used to address urgent issues across different settings, highlighting the importance of designing approaches tailored to each context to identify and solve problems. Additionally, the guest lecturers are intentionally chosen to represent multiple perspectives and experiences in the application of implementation science to global health programs. Evaluation design We used the Kirkpatrick model for evaluating training programs to guide our evaluation of the ISSI-UW [13–15]. This widely used model adopts a multi-dimensional approach by assessing not only the immediate reactions of participants but also their learning, behavior change, and ultimate impact on organizational results, thereby ensuring a holistic understanding of the program’s effectiveness. The model comprises four levels or domains: reaction, learning, behavior, and results. Our evaluation included measures from all four domains (Fig. 1). Evaluation outcomes and data sources based on the Kirkpatrick model By definition, level 3 of the Kirkpatrick model—Behavior – examines the extent to which a change in behavior has occurred because the participant attended the training program [14, 16]. Given that it was not feasible to directly observe that, we use the intent to change behavior as a proxy measure in this study. Descriptive quantitative methods were used to describe participants and outcomes; nonparametric statistical tests were used to evaluate differences and the significance of the outcomes; and basic qualitative content analysis was used to examine open-ended responses to survey questions. Data collection The data used in this evaluation were collected from the following sources: program administrative data, a pre- and post-survey of 2024 trainees, and an alumni survey for all course cohorts from 2014 to 2024. The surveys were emailed to participants who accessed them via a link. All 72 of the 2024 trainees were invited to take the pre- and post-surveys. We invited the 477 course alumni to take the alumni survey using the email contacts we had available. The pre-survey was sent on August 18, 2024, and the post-survey on September 17, 2024. The alumni survey was open from October 17 to November 17, 2024. Two reminder emails were sent to alumni on October 25 and November 01, 2024. Both surveys included Likert-type, multiple-choice, and open-ended questions. The survey responses were anonymous and collected online using REDCap (Research Electronic Data Capture) software [17, 18]. The pre-survey included demographic questions, including country of residence, age group, primary job title, and education. Additionally, we inquired about the participants’ experience level in implementation science and the areas in which it was applied. Eleven questions assessed knowledge level, eight questions assessed skills in using implementation science tools and methods, and eight questions assessed participants’ engagement or intention to engage in implementation science-related activities. The post-survey contained the same questions as the pre-survey, in addition to six questions to assess satisfaction and three open-ended questions about the overall quality of the course, the most valuable session for participants, and suggestions for improvement. The alumni survey included the same demographic and general questions as the pre- and post-surveys. However, ten questions were added to assess post-training engagement activities related to implementation science. Additionally, we asked to what extent the ISS-UW course influenced the participants’ work and whether it helped expand their professional networks. Outcomes We evaluated five outcomes based on the Kirkpatrick model (Fig. 1). Each outcome was evaluated using a set of questions based on the course objectives and lecture topics. Additional File 2 includes the specific questions included in the surveys to measure knowledge, skills, and intent outcomes. Knowledge We created 11 questions using a four-point Likert scale (1 = no knowledge, 2 = beginner, 3 = intermediate, and 4 = advanced) to assess basic implementation science topics, including implementation strategies, implementation outcomes, theories, models and frameworks, study designs, stakeholder engagement, organizational readiness, economic evaluation, and grant writing. Skills The survey applied a set of eight questions on a four-point Likert scale (1 = no skill, 2 = beginner, 3 = intermediate, and 4 = advanced) to assess skills, including specifying strategies, selecting theories, models, and frameworks, integrating appropriate research methods, conducting stakeholder engagement analyses, selecting organizational readiness assessment techniques, and writing grants. Intent To assess the intention of changing behavior, we included a set of eight questions related to designing studies and economic evaluations, conducting stakeholder analyses, writing grants and applying for funding, training others in implementation science-related topics, publishing manuscripts, and integrating policy using a four-point Likert scale (1 = no plans to start, 2 = involved in the past but no plans to continue, 3 = involved in the past and plan to continue, and 4 = no experience but plan to start). Satisfaction Using a five-point Likert scale (1 = poor, 2 = fair, 3 = good, 4 = very good, and 5 = excellent), we assessed the overall course organization, clarity of goals and objectives, quality of lectures, content amount, relevance, and usefulness. Long-term effects To assess long-term effects of the training, we included binary questions (yes or no) related to the course’s influence on expanding the alumni participants’ professional network, influencing organizational or individual work, and engaging in implementation science-related activities (e.g., grant writing, training others, publishing or reviewing manuscripts, designing evaluations and policies, or pursuing advanced training in implementation science). Open-ended questions were included for participants to describe other activities, specify training activities, and comment on how the training influenced their work and expanded their professional network. Data analysis We used descriptive statistics to analyze data on participants’ demographic characteristics. Responses from the Likert-type questions are presented as percentages. Using the Wilcoxon rank sum test, we compared the unpaired medians of responses from the pre- and post-surveys, with the level of significance defined as P < 0.05. Statistical analysis was conducted in R software version 4.2.2. A complementary qualitative analysis was performed by applying a content analysis to the open-ended questions in the surveys [19]. One researcher (AD) read the text to identify codes and keywords in the responses, coded the text inductively, grouped the codes into categories and subcategories, and quantified them using frequency counts. To strengthen analytic rigor, we triangulated qualitative data derived from the content analysis of open-ended survey responses with quantitative descriptive findings, allowing for cross-validation and a more nuanced understanding of participant perspectives. The qualitative analysis was performed in ATLAS.ti version 25.0.1. Positionality statement This evaluation was conducted by a team based in a U.S academic institution. We acknowledge that our institution’s location, disciplinary training, and access to resources shape our perspectives and approaches to evaluating capacity-building efforts. As evaluators situated in the Global North, we recognize potential power asymmetries inherent in partnerships that involve participants from multiple countries. Our interpretation of findings is influenced by these positionalities, as well as by cultural and contextual lenses through which we understand implementation and training outcomes. Results Descriptive statistics From 2014 to 2024, the ISSI-UW trained 477 participants from 48 countries. The majority of trainees were from North America (n = 239, 50.1%), followed by sub-Saharan Africa (n = 124, 25.9%), Latin America and the Caribbean (n = 37, 7.8%), Europe and Central Asia (n = 32, 6.7%), East Asia and Pacific (n = 22, 4.6%), South Asia (n = 22, 4.6%), and the Middle East and North Africa (n = 1, 0.2%). Of the 48 countries, 69% (n = 33) were LMICs according to the World Bank definition. The number of trainees increased gradually from 2014 to 2019; however, the course was not held in 2020 and 2021 due to the COVID-19 pandemic. ISSI-UW resumed in 2022 with fewer participants, but participation increased again between 2022 and 2024. The majority of trainees between 2014 and 2024 were from the United States (n = 233, 48.8%), followed by Kenya (n = 32, 6.7%). Table 1 outlines the demographic characteristics of survey respondents. Of the 72 trainees invited to participate in the 2024 pre- and post-surveys, 75% (n = 54) and 70% (n = 51) responded. Of the 477 alumni invited to participate in the 2014–2024 alumni survey, 28% (n = 136) responded. These 136 alumni are from 36 distinct countries, of which 64% are LMICs. The age groups of participants were relatively evenly distributed for the pre- and post-surveys. In contrast, the majority of alumni survey respondents were aged 35 to 44 (n = 74, 54.4%). Nearly half (48.1% and 47.1%, respectively) of participants in the pre- and post-surveys held doctoral degrees, while the percentage for alumni respondents was slightly higher at 61.8%. Most respondents were researchers, followed by implementers, graduate students, and staff from donor agencies. Of the pre-survey participants, 22% indicated they had no experience in implementation science at the start of their training, and none self-reported having advanced experience. This situation shifted for the alumni respondents, as most reported having intermediate experience in implementation science, and 16.2% had advanced experience (Table 1). Amongst the non-participants, most were from earlier cohorts, independently of country of origin. Table 1Demographic characteristics of survey respondentsNo. (%) of participant responsesPre-survey 2024Post-survey 2024Alumni survey 2014–2024n = 54n = 51n = 136Response rate75%70%28%Age group 22–3418 (33.3%)15 (29.4%)15 (11.0%) 35–4420 (37.0%)20 (39.2%)74 (54.4%) 45–6416 (29.6%)16 (31.4%)47 (34.6%)Academic degree Bachelor’s3 (5.6%)2 (3.9%)4 (2.9%) Master’s25 (46.3%)25 (49.0%)48 (35.3%) Doctoral26 (48.1%)24 (47.1%)84 (61.8%)Role* Researcher32 (59.3%)24 (47%)81 (59.5%) Implementer14 (25.9%)19 (37.2%)30 (22.1%) Graduate student5 (9.3%)5 (9.8%)20 (14.7%) Donor3 (5.6%)3 (5.8%)5 (3.7%)World region East Asia and Pacific5 (9.3%)6 (11.8%)6 (4.4%) Europe and Central Asia2 (3.7%)4 (7.8%)9 (6.6%) Latin America and Caribbean6 (11.1%)4 (7.8%)15 (11.0%) North America28 (51.9%)21 (41.2%)54 (39.7%) South Asia1 (1.9%)1 (2.0%)11 (8.1%) Sub-Saharan Africa11 (20.4%)13 (25.5%)41 (30.1%) Middle East and North Africa1 (1.9%)1 (2.0%) Missing1 (2.0%)Implementation science experience** No experience12 (22.2%)6 (11.8%) Beginner24 (44.4%)26 (51.0%)36 (26.5%) Intermediate18 (33.3%)19 (37.3%)78 (57.4%) Advanced22 (16.2%) Course track Fundamentals43 (79.6%)40 (78.4%) Advanced11 (20.4%)11 (21.6%)Training year 20143 (2.2%) 20155 (3.7%) 20165 (3.7%) 201714 (10.3%) 201819 (14.0%) 201923 (16.9%) 202210 (7.4%) 202324 (17.6%) 202431 (22.8%) Missing2 (1.5%)*For alumni, the time period for the role was the year they attended training**For alumni, the time period for implementation science experience was at the time of the survey Demographic characteristics of survey respondents *For alumni, the time period for the role was the year they attended training **For alumni, the time period for implementation science experience was at the time of the survey Short-term outcomes Knowledge domain Figure 2 illustrates the distribution of responses for 11 questions in the pre- and post-surveys, for both groups in the training (fundamentals and advanced). In the pre-survey, participants indicating unfamiliarity with implementation science concepts (no knowledge) across 11 questions varied from 20 to 59%. However, in the post-survey, only 4% reported no knowledge in response to a question about different economic evaluation methods. When comparing the median responses between the two surveys using the Wilcoxon test, we observed a statistically significant difference in improvement (p-value < 2.2 × 10–16). Distribution of responses between pre- and post-surveys in the knowledge domain for both groups (fundamentals and advanced) Note Fig. 2: A question on identifying core components of an implementation strategy was restricted to trainees in the advanced track (n = 11). Abbreviations in Fig. 2: EBI, evidence-based intervention; IR, implementation research; TMFs, theories, models, and frameworks When we stratified the two training groups, we observed that in the fundamentals group, participants indicating no knowledge across 11 questions ranged from 26 to 70% in the pre-survey, with the majority expressing unfamiliarity with different types of economic evaluations in implementation research (70%) and key ingredients for writing and strategies for submitting grants (see additional file 3). In the post-survey for participants in the fundamentals group, only 2% reported not knowing a single question about different types of economic evaluations in implementation research. For participants in the advanced group, lack of knowledge in the pre-survey was noted in two questions: different types of economic evaluations in implementation research (18%) and key ingredients for writing and strategies for submitting grants (9%). In the post-survey, only 9% reported unfamiliarity with different types of economic evaluations. For both stratified analyses, the Wilcoxon test between pre- and post-responses was statistically significant. Skills domain In the pre-survey, 26% to 57% of the overall participants indicated a lack of skills in applying implementation science concepts and methods across eight questions. In the post-survey, this decreased to 2% and 6% for two questions on integrating qualitative and economic evaluation methods in research, respectively (Fig. 3). The difference was statistically significant (p-value < 2 × 10–16) when comparing the median responses from the two surveys. Distribution of responses between pre- and post-surveys in the skills domain for both groups (fundamentals and advanced) Abbreviations for Fig. 3: IS, implementation science; TMFs, theories, models, and frameworks In stratified analysis, within the fundamentals group, the range indicating no skills in the pre-survey was from 33%—integrating qualitative methods into research—to 70%—specifying an implementation strategy (see additional file 3). In the post-survey, only 5% self-reported not having the skills to integrate economic evaluation into research, and 2% reported not having the skills to conduct stakeholder analysis and to develop an engagement plan. For the advanced group, among pre-survey participants who reported no skills, the range was 9% (specifying an implementation strategy and selecting an appropriate approach to assess organizational readiness) to 18% (integrating economic evaluation methods into research and writing about implementation science methods in a grant). In the post-survey, the advanced group reported having no skill in only one question: integrating qualitative methods into research (9%). The differences between pre- and post-surveys in stratified groups were statistically significant in the Wilcoxon test. Intent to change domain When assessing the intention to change behavior related to the application of implementation science concepts and methods in their work by exposing 8 implementation science-related activities to participants via survey, we observed shifts between the pre- and post-test responses for those who did not indicate an intention to start a particular activity. For instance, 19% of respondents in the pre-survey reported not having plans to design an evaluation using quantitative methods, which decreased to 6% in the post-survey. A similar observation was noted for other questions in which changes in responses were observed between pre- and post-surveys, with the exception of one question related to designing an evaluation using economic evaluation methods (Fig. 4). Distribution of overall responses between pre- and post-surveys in the intent domain Abbreviation for Fig. 4: IS, implementation science In Fig. 4, we show the distribution of overall responses. The differences in the medians between the pre- and post-surveys were statistically significant (p-value = 0.005). In a stratified analysis within the fundamentals group, the percentage of participants expressing no plans to engage in any of the listed implementation science activities ranged from 21 to 33% in the pre-survey. In the post-survey, the number of participants reporting no plans to conduct implementation science-related activities decreased in most areas, except for one—designing an evaluation using economic evaluation methods—which increased from 33 to 38% (see additional file 3). For the advanced group, the range of responses indicating no intention to engage in implementation science activities in the pre-survey was from 9 to 36%. The areas where participants expressed no interest mainly were related to designing evaluations using economic assessments (36%) and training others in implementation science topics (27%). In the post-survey, there were significant shifts in these intentions, and we also observed an increase in participants responding that they do not plan to include a policy lens in their research (from 9 to 27%). Trainee’s satisfaction with the course Overall, participants reported being satisfied with various aspects of the course (Fig. 5). Higher satisfaction levels were linked to the quality of lectures and materials, relevance and usefulness of the course content, and overall quality and organization of the course. However, 8% of respondents expressed that the amount of content and the workload was only “fair.” Distribution of post-survey responses on satisfaction Thirty-one respondents provided feedback on the course quality in an open-ended format. In the content analysis, these comments were summarized and categorized into two groups: aspects that trainees appreciated the most and aspects they appreciated the least. The most appreciation was noted for course quality, as indicated by 12 respondents; 8 valued the organization and usefulness of the course, 5 highlighted the quality of the speakers, and 2 praised the positive environment of the course. Conversely, less appreciation was expressed regarding an overload of content, noted by 4 respondents, insufficient time to cover the material, reported by 3 respondents, and 2 expressed dissatisfaction about the physical environment (cold room temperature). When asked to provide suggestions for improvement, 29 respondents commented on aspects of organization and content. Regarding course organization, 9 participants recommended creating more opportunities to apply concepts, 5 suggested adding breaks and interactive activities between sessions, and 4 suggested the course duration should be longer, with shorter sessions, and to make materials available before the course starts. Three respondents proposed reducing the number of practical sessions and making lessons more adult-friendly. Regarding course content, 2 respondents suggested adding more examples about mental health, 1 mentioned emphasizing equity, and 1 suggested adding examples from the United States. When asked about their favorite topics, 29 trainees responded. The top four topics included implementation science theory, models, and frameworks (31%), followed by case studies (24%), an introduction to implementation science (21%), economic evaluation (14%), and grant writing (10%). Long-term outcomes Areas where implementation science is applied When assessing alumni trainees on the areas in which they applied implementation science concepts and methods after the training, three areas stand out: health systems strengthening (11.9%); maternal, newborn, and child health (11.4%); and HIV and STIs (11.2%). Seven respondents (1.7%) confirmed they have not yet applied implementation science methods in their work. Additional File 4 provides more information about areas where participants applied implementation science methods and tools. Post-training activities related to implementation science The alumni survey included questions about participants’ engagement in implementation science-related activities, and 84% confirmed they are now using implementation science theories, models, and frameworks in their work, 52% have designed an evaluation study since completing ISSI-UW, and 50% reported writing an implementation science-focused grant after the training program (Fig. 6). Distribution of alumni survey responses regarding post-training engagement in implementation science-related activities Abbreviation for Fig. 6: IS, implementation science When we review responses to post-training activities across cohorts from 2014 to 2024, we see few responses from the earliest cohorts and lower engagement among those in the most recent cohorts. Responses from the middle years tend to be similar; for example, about 60% of participants in cohorts from 2015 to 2019 confirmed that the training influenced their grant writing related to implementation science, decreasing to 44% in 2022 and rising again to 58% in 2023. Training others in implementation science topics We asked alumni respondents who reported training others in implementation science topics to describe the type of training they provided. Of the 51 responses we received, 26 indicated their roles in the training, 36 specified the audience, and 9 mentioned the duration of the training. Respondents identified their roles in these training sessions as teachers (n = 16, 61%), supervisors (n = 3, 11%), mentors (n = 2, 8%), material developers (n = 2, 8%), symposium organizers (n = 2, 8%), and consultant (n = 1, 4%). The audience types identified were students (n = 13, 36%), coworkers (n = 8, 22%), health workers (n = 6, 17%), ministry of health staff (n = 4,11%), faculty (n = 3, 8%), researchers (n = 1, 3%), and stakeholder staff (n = 1, 3%). All noted that the training durations were relatively short, varying from 1 h to 1 month. Two respondents described the training activity as follows: “Oriented my fellow staff on implementation science and refined the tool to implement projects based on what we’ve learned on the course.” (Alumni 31) “I trained students and teachers in IS [implementation science] with a 10-hour course. I supervise master’s and doctoral students who develop research using IS frameworks.” (Alumni 3) Other implementation science-related activities As shown in Fig. 6, 6% of alumni respondents reported being involved in “other activities.” These activities included recommending that their students take implementation science training, supervising implementation activities using the knowledge gained in training, and pursuing other implementation science courses. Influence of the ISSI-UW training on alumni work We asked alumni trainees how much their work has been influenced by their training in ISS-UW on a scale from “very large extent” to “not relevant.” Participants responded as follows: very large extent (n = 28, 21%), large extent (n = 44, 32%), moderate extent (n = 39, 29%), small extent (n = 19, 14%), and very small extent (n = 6, 4%). No respondents reported that the course was not relevant. In the content analysis of open-ended comments, some respondents described integrating what they learned in ISSI-UW at both the organizational and individual levels. For example, they reported using materials and resources from the institute in trainings they provided to others, adopting implementation science vocabulary to engage with other researchers, applying for implementation science-related grants, designing studies, and engaging stakeholders. “I lead a research center that aspires to become a center of excellence for implementation science. At the research center, we train young researchers in implementation science. In our training and workshops, we use the training materials and resources we received from the UW Implementation Science Institute.” (Alumni 6) “The mentorship and training provided by the Institute have equipped me with essential implementation science methodologies, enhancing my ability to design impactful studies. Additionally, the emphasis on community engagement has helped me understand the importance of involving stakeholders in the research process, ensuring that my work is relevant and actionable.” (Alumni 28) Influence of the ISSI-UW training on alumni networks We asked the alumni trainees whether they felt that the ISSI-UW helped expand their professional networks. Ninety respondents (66%) answered affirmatively, and 88 (65%) provided details on how the course enhanced their network. The following two categories emerged from our qualitative analysis of these responses: network activities and network subjects. Thirty-six respondents identified their networking subjects, which included other trainees (n = 12, 33%), other researchers (n = 10, 28%), University of Washington faculty (n = 7, 19%), other implementers (n = 5, 14%), and other institutions (n = 2, 6%). Network activities mentioned by 23 respondents included collaboration on research projects (n = 11, 48%), work opportunities (n = 9, 40%), establishing a regional network (n = 1, 4%), writing a grant (n = 1, 4%), and conducting stakeholder engagement (n = 1, 4%). “It helped to meet with researchers who are interested in implementation science. At the moment, I am collaborating with one of the alumni on a research project. Partly, this is because the training at UW has helped us to develop a similar understanding and interest about implementation science.” (Alumni 4) “The course has helped me to inform USAID [US Agency for International Development] investment in innovations and policy adoptions for TB [tuberculosis] especially with drug resistance TB new regimen introduction in the country.” (Alumni 49) Discussion In this article, we describe an evaluation of the ISSI-UW implementation science training program from 2014 to 2024. We used the Kirkpatrick model to guide the evaluation and conducted quantitative and complementary qualitative analyses to assess both short-term and long-term outcomes. Based on our analysis of the survey responses, the findings demonstrate improvements in knowledge, skills, and intent to change behavior, as well as high satisfaction among course trainees. We observed differences between the fundamentals and advanced training groups regarding their levels of knowledge, skills, and intent to change behavior. In the fundamentals training group, we observed significant shifts in knowledge, including those related to methods of economic evaluation and key components of grant writing. In contrast, in the advanced group, topics observing greater shifts in knowledge included using implementation outcomes in a study and key strategies for writing grants. These observations on the knowledge align with responses related to skills, where changes are more noticeable among those who reported lacking skills in integrating economic evaluations, specifying strategies, and writing grants in the fundamentals group. In the advanced training group, improvements are seen in selecting and applying theories, models, and frameworks, as well as integrating economic evaluations and writing grants. Given the time needed to observe and assess behavior change, as outlined in the Kirkpatrick model’s third level—behavior—we used “intent to change” as a proxy to assess trainees’ potential post-training behavior in applying implementation science methods and tools in their work. The use of intention as a proxy is theoretically grounded in the Theory of Planned Behavior, which posits that intentions precede behavior [20, 21]. In this area of intention, both the pre- and post-surveys revealed that 33% of trainees consistently indicated no plans to start designing evaluations using economic evaluation methods. This may be due to the methods not being applicable in their current careers or to their not being interested in economic evaluations. In contrast, the number of participants who reported lacking past experience but intended to start engaging in implementation science-related activities increased in the post-survey. This increase pertains to activities such as training others in implementation science topics, writing proposals to apply for funding, publishing manuscripts, incorporating a policy lens in research, and using qualitative methods when designing evaluations. When assessing the long-term outcomes of the training, we found that trainees are applying the knowledge and skills acquired in the course across multiple areas, with a particular focus on health systems strengthening, maternal and child health, and HIV and STI programs. They also reported engaging in using theory, models, and frameworks, designing evaluations, training others, writing grants, and involving stakeholders and other researchers with more appropriate language. Furthermore, trainees reported that the training facilitated the expansion of their professional networks, creating opportunities for research collaborations and job prospects. Linking these results to our evaluation model, we can summarize that high levels of satisfaction among trainees reflect a positive reaction to the training program. Participants’ favorable perception of the course structure’s relevance and quality of delivery suggests that the training successfully engaged learners and met their expectations – an essential precursor to deeper learning and behavioral change. Reported improvement in knowledge and skills related to implementation science methods and tools corresponds to outcomes at level 2 of the model—learning. This indicates that participants achieved the intended educational objectives and gained both conceptual understanding and methodological competence. Longer-term outcomes, such as participants applying concepts in teaching activities, training other people in the workplace, and developing a grant proposal, demonstrate behavioral change. These behaviors suggest that participants have transferred acquired knowledge and skills into their professional practice and institutional environments, which is one of the most critical tests of training effectiveness. The expansion of professional networks and increased institutional capacity (e.g., more individuals engaged in implementation research and training) represent results-level outcomes. The findings point to broader system-level effects of the training, including strengthened academic collaboration, the diffusion of implementation science expertise, and potentially enhanced research productivity. Implementation science training programs at other institutions have also been evaluated in the literature. For example, Carlfjord et al. [22] evaluated implementation training for doctoral students at Linköping University in Sweden using the Kirkpatrick model, in which they reported positive short- and long-term outcomes. In addition, Moore et al. [23] conducted a longitudinal evaluation of a course designed to build core competencies in implementation science. They outlined and described the Practicing Knowledge Translation course and evaluated participants’ use of theories, models, frameworks, knowledge, self-efficacy in knowledge translation, satisfaction, and feedback on the course. In this evaluation of 15 participants from four countries, comprising clinicians, researchers, healthcare managers, and policymakers [23], findings showed an increased application of their outcomes. In South Africa, a qualitative evaluation was conducted by Ojifinni et al. to assess the short- and long-term impacts of an implementation science training program [24]. The researchers gathered 10 students’ perspectives on their training experience at pre-training, post-training, and five years after graduation. Participants reported a favorable pre-training perception of the course and overall satisfaction. Most students continued to apply their training in their roles, while some pursued further education in implementation science-related programs five years after graduation [24]. More recently, the Institute for Implementation Science Scholars (IS-2) at Washington University in St. Louis used a pre/post design to evaluate skill building, mentoring, and networking, finding increases in skills, the establishment of a strong mentorship program, and a high number of network ties [25]. This program offered mentored training for early and mid-career investigators in implementation science methods, with an emphasis on equity. Compared with previous evaluations of implementation science capacity-building efforts, a strength of our study is its longer time period. We evaluated 11 years of training experience in implementation science in global health. By using the well-known Kirkpatrick model to guide the evaluation, we ensure clarity and comparability in similar studies and contribute to closing the existing gap of a few capacity-building programs in implementation science using established frameworks in evaluation efforts. Furthermore, using different types of assessments enabled us to triangulate our findings, which provided robustness to our analysis. By emphasizing a global health perspective and including participants from diverse contexts and backgrounds, the ISSI-UW provides an outstanding platform for exchanging experiences and knowledge, as well as for harmonizing understanding of implementation science concepts and methods worldwide. Our alumni include professionals from diverse contexts, including 33 LMICs, who were equipped with fundamental knowledge of implementation science methods, with the potential to accelerate the translation of evidence into practice. Furthermore, with the knowledge they acquire, they can improve local ownership of evidence-based interventions and support the development of contextually appropriate implementation strategies. Additionally, by training diverse groups of participants, the training program helps reduce inequalities in global knowledge production, promotes south-north and north–south collaborations, and increases the chances that implementation research reflects diverse health systems and sociocultural contexts. This study relies on self-reported survey data, which presents several limitations. Self-reports are subject to recall bias, as participants may not accurately remember past behaviors, and to social desirability bias—the tendency of individuals to portray themselves in a favorable light. Additionally, there is potential for response bias among alumni trainees, as individuals who chose to participate may differ systematically from non-respondents in ways that influence their perception or behavior. Furthermore, we examined the long-term outcomes of all cohorts of trainees at a specific time point, which contributed to a lower response rate in earlier cohorts because, for some participants, a substantial amount of time had passed since they participated in the course. Moreover, the Likert-scale responses are subjective, as we did not define which competencies distinguish beginners from intermediate or advanced learners. These factors limit the objectivity and generalizability of the findings. Also, we use intent to change as a proxy for behavioral change, which may not perfectly reflect actual behavior. Lastly, all pos-training engagement activities were self-reported, with no evidence that they actually occurred. Nonetheless, self-reported data provided valuable insight into participants’ perceptions and experiences – information that would be difficult to capture through other means – and thus remains an appropriate and informative data source for this study. Given the standardized model used in this evaluation, the study can be easily replicated in other similar institutions or initiatives. Several lessons emerged from our experience in conducting the training. A key insight is the need to develop a more structured and sustained mentorship platform to support alumni beyond the formal training period. While the training successfully enhanced participants’ knowledge and skills, many alumni expressed the value of continued mentorship to facilitate learning, collaboration, and application of implementation science methods in their institutional settings. Additionally, it is desirable to establish collaborative partnerships that enable the replication and adaptation of the training model across diverse contexts, particularly in LMICs, where capacity-building opportunities in implementation science remain limited. Empowering alumni to co-lead or localize training efforts promotes sustainability and equity in the development of global implementation science. Equally important is our recognition in this process that diversity of disciplines, backgrounds, and contexts of participants is a critical asset in implementation science training, as it enriches discussions, expands context-specific perspectives, and creates context-sensitive approaches to implementation challenges. Future evaluations on implementation science training should adopt comprehensive, multilevel, and theory-informed approaches that go beyond immediate post-training satisfaction to assess sustained learning and application. Finding ways to measure evidence of behavioral change more accurately is crucial for refining effectiveness analysis. Using mixed-methods designs to triangulate quantitative competency assessments with qualitative insights should also be considered, as this captures nuanced behavioral and contextual changes. We also recommend a thorough longitudinal follow-up of trainees to document real-world application of skills, professional network expansion, and organizational and policy impacts. Furthermore, embedding equity-focused and participatory evaluation strategies can ensure that programs benefit trainees from diverse disciplines and geographic regions fairly, addressing the persistent imbalances between high- and low-income settings. Finally, transparent reporting of evaluation tools, indicators, and outcomes will facilitate cross-program learning and contribute to a more cumulative evidence base on effective implementation science capacity building. Conclusions The evaluation of the intensive training in implementation science concepts and methods provided by ISSI-UW to a global audience of researchers, implementers, graduate students, and donors found positive short-term and long-term results in improving implementation science competencies. This highlights the importance of continuing to improve and fund initiatives like the ISSI-UW to spread essential knowledge and skills to a vast audience of worldwide participants and provide them with tools to enhance evidence generation in research and implementation efforts. Supplementary Information Additional file 1: Objectives and topics offered in Implementation Science Summer Institute at the University of Washington in 2024. We present a table outlining the objectives and topics for the implementation science fundamentals and advanced tracks. The table distinguishes between shared topics and specific topics for each track. Additional file 2: Survey questions for knowledge, skills, and intent domains. We present 11 questions to evaluate knowledge and 8 to assess skills and intent. Additional file 3: Figures of the stratified analysis for knowledge, skills, and intent. We present three figures showing the distribution of pre- and post-survey responses stratified by training groups (fundamentals and advanced). Additional file 4: Bar graphs with the names of areas participants indicate applying implementation science. One bar graph of the 2024 post-survey. One graph of the alumni (2014–2024) survey. Acknowledgements We thank the ISS-UW alumni trainees for participating in the surveys conducted. Abbreviations Implementation Science Summer Institute at the University of Washington Authors’ contributions KS, AM, EM, and AD conceived the study. RM and AD collected the data. AD prepared the data, conducted the analysis, and wrote the manuscript. All authors reviewed and approved the manuscript. Funding This work was supported by the University of Washington/Fred Hutch Center for AIDS Research, an NIH-funded program under award number AI027757 supported by the National Institute of Allergy and Infectious Diseases (NIAID), National Cancer Institute (NCI), National Institute of Mental Health (NIMH), National Institute on Drug Abuse (NIDA), Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), National Heart, Lung, and Blood Institute (NHLBI), National Institute on Aging (NIA), National Institute of General Medical Sciences (NIGMS), and National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK). Data availability All data generated or analyzed during this study are included in this published article, and its supplementary information files. Declarations Ethics approval and consent to participate Not applicable. Consent for publication Not applicable. Competing interests Kenneth Sherr is on the Editorial Board of Implementation Science Communications. The other authors declare that they have no competing interests. Footnotes Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Supplementary Materials Additional file 1: Objectives and topics offered in Implementation Science Summer Institute at the University of Washington in 2024. We present a table outlining the objectives and topics for the implementation science fundamentals and advanced tracks. The table distinguishes between shared topics and specific topics for each track. Additional file 2: Survey questions for knowledge, skills, and intent domains. We present 11 questions to evaluate knowledge and 8 to assess skills and intent. Additional file 3: Figures of the stratified analysis for knowledge, skills, and intent. We present three figures showing the distribution of pre- and post-survey responses stratified by training groups (fundamentals and advanced). Additional file 4: Bar graphs with the names of areas participants indicate applying implementation science. One bar graph of the 2024 post-survey. One graph of the alumni (2014–2024) survey. Data Availability Statement All data generated or analyzed during this study are included in this published article, and its supplementary information files.