Evaluation of healthcare simulation centers: development of the “STRIVE” tool using the Delphi method Authors: Yan Zhang, Yihong Lu, Huiyun Yuan, Chao Meng, Yifei Mu, Xiaolei Qian, Lars Konge, Zhengqian Bian Published: 2025 ABSTRACT AbstractBackgroundSimulation-based medical education (SBME) is a critical tool for teaching and assessing technical and non-technical medical skills. Healthcare simulation centers (HSCs) are multidisciplinary centers that are proliferating in number and provide simulation-based medical education to various groups of trainees, including medical students, residents, healthcare professionals, and nurses. Large investments in the physical space, simulation hardware, and expert human resources are required to set up HSCs. However, most published research on simulation training has focused on the content and evaluation of the training. The evaluation of HSCs has rarely been discussed. This study aimed to identify the most appropriate aspects for the evaluation of HSCs using the Delphi technique and to develop an evaluation model in China.MethodsThirty simulation centers were identified as potential participants. The first round of the Delphi process was a brainstorming exercise. The initial questionnaire contained one main question: What aspects do you think should be included in the evaluation of HSCs? The steering group reviewed the responses received from participants in the first round of the Delphi, removed duplicates, and organized the items into different categories. Responses from the second round of the Delphi method were analyzed and presented as frequencies and percentages. The consensus level was set at 70%. If this goal was not achieved, the participants were asked to participate in a third round of the Delphi exercise.ResultsTwenty-four experts (the heads of HSCs) completed the first round of Delphi. The response rate was 80%. In the first round, 106 items were suggested. After verifying the content and removing duplicates, 45 items were left. The model for the evaluation of the HSCs was named “STRIVE,” reflecting a combination of the initial of each of the six involved categories: simulation curricula, training of faculty, result of training, internal resources, variety of activities, effect of activities. The response rate was 100% in the second round. The STRIVE model identified the 24 most frequently mentioned items across six categories.ConclusionThe STRIVE model can help HSCs managers understand and evaluate their centers more comprehensively. It can also help them address their shortcomings.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12909-025-08332-y.Keywords: Simulation-based medical education, Healthcare simulation centers, Delphi method Background Simulation-based medical education (SBME) is valuable for teaching and assessing technical and nontechnical medical skills [1]. This is because SBME provides a controlled environment without the risk of patient harm, in which trainees can practice specific skills in a reproducible manner and receive timely performance-enhancing feedback geared towards a level of mastery before operating on patients [2, 3]. Healthcare simulation centers (HSCs) are multidisciplinary centers that provide simulation-based medical education to various groups of trainees, including medical students, residents, nurses, and other healthcare professionals [4]. The demand for healthcare simulation centers in medical education arises from the need to ensure safe, effective, and early acquisition of clinical skills in an increasingly complex healthcare environment. HSCs provide a risk-free setting for learners to practice essential procedures, adapt to rapidly evolving technologies, and build confidence before encountering real patients. They also supports broader goals such as improving patient safety and training quality [5, 6]. There has been a significant increase in the number of simulation centers in tandem with the popularity of SBME. The Society for Simulation in Healthcare (SSH) accredited more than 966 simulation centers worldwide [7]. The number of HSCs in mainland China is growing at an impressive rate, from 137 in 2017 to more than 900 in 2021 [8]. However, large investments in the physical space, simulation hardware, and expert human resources are required to set up HSCs [9]. This makes it essential for both newly built and existing centers to determine the optimal elements required to operate the center. In China, for example, hospitals with residency training programs are required to build an HSC of at least 600 m2; however, the qualitative measures are rarely defined [10]. Most published research on simulation training focuses on the content and evaluation of the training program, whereas evaluation of the entire HSC is rarely discussed, which is a prevalent challenge [11]. Evaluating simulation centers is therefore crucial to ensure they meet these core educational and healthcare objectives. Therefore, the aim of this study is to identify the most appropriate aspects for the evaluation of HSCs in mainland China using the Delphi technique and develop an operational evaluation model based on the achieved consensus. Methods Study design Most existing research on simulation-based training has focused on the design, delivery, and evaluation of specific training programs, while the comprehensive evaluation of healthcare simulation centers (HSCs) as institutional entities remains limited and underexplored. Moreover, there is currently no standardized or validated instrument available in Chinese that specifically addresses the multidimensional evaluation of HSCs within the mainland China. In light of this gap, the present study aimed to identify key evaluation domains for HSCs in China using the Delphi method and to develop a practical evaluation model based on expert consensus. We use the Delphi approach to reach a consensus among a panel of key opinion leaders (KOLs) in the simulation. The Delphi method is a technique used for gathering the opinions of anonymous experts and reaching a final consensus through iterative structured feedback and statistical analysis of the collected data [12–15]. Ethics approval for this study was not required as determined by the Shanghai Jiaotong University school of Medicine, Renji Hospital Ethics Committee. The reference number for this decision is EX-2024-004. Steering group The steering group consisted of experts from a variety of professions, including the director of a healthcare simulation center (ZQB), two senior clinical medical educators with more than 5 years of teaching experience in simulation (YZ and LCS), and a research nurse (YHL). Key opinion leaders Thirty simulation centers in Mainland China were identified as potential participants based on in-depth discussions with the Steering Group. To make the results more generally applicable, diverse HSCs from different geographical locations (East China, Central China, and West China), different types of HSCs (HSCs in specialist hospitals and HSCs in general hospitals), and different ranks of HSCs (HSCs in tertiary care hospitals and HSCs in primary care hospitals) were selected. We also included HSCs with different years of establishment (< 5 years, 5–10 years, and > 10 years) to cover both new and established centers. These centers deliver training programs to both medical students and a wide range of healthcare professionals. The study proposal and demographic information questionnaires were sent to the directors of HSCs. We obtained informed consent from all the individuals who took part in the study. Participants expressed their consent by filling out and returning the questionnaire, which contained a statement notifying them that their response indicated their agreement to take part in the study.They were also made aware that they could withdraw from the study at any time without incurring any consequences. Delphi round 1 The first round of the Delphi process was a brainstorming exercise. The initial questionnaire comprised a single question. “What aspects do you suggest should be considered when evaluating HSCs?” “What aspects do you suggest should be considered when evaluating HSCs?” WeChat (Tencent, Shenzhen, China), a Chinese mobile messaging application, was used to deliver the proposal, demographic questionnaire, and initial Delphi questionnaire (see supplementary file Questionnaire 1). The first round of the survey lasted approximately 1 month. After the initial questionnaire, a reminder was sent 2 weeks later. The steering group reviewed the responses received from the first round of the Delphi survey, removed duplicates, and organized the items into six categories. The processed information was used to develop a Delphi questionnaire in the second round. Delphi round 2 To ensure the conciseness and practicality of the evaluation criteria.,the Steering Group decided to include only four items in each of the six categories. The participants were asked to choose the four highest priorities from each category. Questionnaires were distributed through the Chinese online survey platform Wen Juan Xing (Ranxing Information Technology, Changsha, China). (see supplementary file Questionnaire 2) The analysis and presentation of responses from the second round of Delphi were conducted by calculating frequencies and percentages. The consensus level was set at 70%, indicating that at least 70% of the participants selected the same items as their top four priorities. Delphi round 3 During the third round, the participants were provided with the median percentage of the panel’s rating, as well as their own individual ratings for each item. The participants were given the option to maintain their previous ratings or make changes as necessary. Statistical analysis was performed using SPSS, V.25.0. Results Twenty-four of the thirty KOLs completed the first round of Delphi, with a response rate of 80%. Most respondents (67%) were from East China, 83% were located in general hospitals, and 71% were located in tertiary care hospitals. There was a good mix of newly established centers (Table 1). Table 1Demographic characteristics of the HSCsCharacteristicsNo. (%)LocationEast China1666%*Central China521%West China313%Type of the HSCsHSCs located in specialist hospital417%HSCs located in general hospital2083%Years of Establishment of HSCsLess than 5 years521%5–10 years1146%More than 10 years833%Rank of the HSCsHSCs based in tertiary care hospital1771%HSCs based in primary hospital729%*Due to rounding, the sum of the percentages exceeded 100% (e.g., 16/24 = 66.67%, 5/24 = 20.83%, 3/24 = 12.5%). To ensure that the total percentage equals exactly 100%, we adjusted the value of 66% without rounding Demographic characteristics of the HSCs *Due to rounding, the sum of the percentages exceeded 100% (e.g., 16/24 = 66.67%, 5/24 = 20.83%, 3/24 = 12.5%). To ensure that the total percentage equals exactly 100%, we adjusted the value of 66% without rounding Delphi round 1 In the initial round, the participants suggested 106 items. After the steering group reviewed the content and eliminated duplicate items, 45 items remained. The remaining items were categorized into six groups: simulation curricula (7), training of faculty (8), result of training (7), internal resources (9), variety of activities (6), and effect of activities (8). The evaluation model for HSCs is called the STRIVE model, and is an acronym for the six categories (Fig. 1). The STRIVE model for HSC evaluation Delphi round 2 and round 3 In the second round, all 24 participants completed the questionnaire, with a response rate of 100%. Across the four categories, four items reached 70% consensus and were included directly in the STRIVE model. In the other two categories, five items reached a consensus. However, the fifth-rated item was removed from the STRIVE model because of numerical limitations imposed by the steering group. The two items that were not included are as follows: In the Results of training category: The trainees rated their satisfaction with the Simulation Center above 80%. In the Internal resources category: A variety of training functions are available at the Simulation Center (e.g. skill training, OSCE exams, and scenario-based simulations).The consolidated list, along with the item ratings from the second round, was sent to all participating experts with an invitation to provide feedback should they have any objections. No differing opinions were received. The final list of 24 items for the STRIVE model is presented in Table 2. For comprehensive information regarding Round 1 and Round 2 of the Delphi process, please refer to Appendix Table 1 and Appendix Table 2. Table 2Final list of items in the STRIVE evaluation model for HSCsNo.ItemNo.%Simulation curricula1Skills training courses for both residents and specialists2396%2Skills training courses for medical students2292%3Team training courses2083%4Communication skills training courses1771% Training of faculty 1Annual simulation instructors training courses2396%2One or more faculty members involved in research in simulation-based education1875%3One or more faculty members involved in national and international simulation training courses every year1875%4Hospital support for incentive schemes for simulation faculty1771% Results of training 1Pass rate for the practical part of the annual National Practitioners Exam at or above 90%24100%21000 + trainees trained in the simulation center (per year)2083%3One or more skills competitions won by simulation center trainees every year1979%4One or more scientific papers published on simulation education every year1875% Internal resources 1One or more simulation modalities available in the Simulation Center (e.g. task trainers, virtual reality simulators, full-scale simulation)24100%2More than 3 full-time staff in the simulation center2396%31-million RMB investment in the operation of the simulation facility annually2188%41000 + m2 for simulation facilities at the center1979% Variety of the activities 1Training courses for the community (e.g. resuscitation courses or scientific knowledge popularization)2396%2Development of virtual reality simulation equipment2188%3Participation in multi-center simulation research initiated by the National Medical Examination Center (NMEC) or the China Medicine Education Association (CMEA)1771%4Patent applications1771% Effect of activities 1Improved patient safety through skills training and regular assessments of health care personnel2396%2Improved patient safety through in-situ simulation (ISS) and other full-scale simulation2188%3Dissemination of knowledge through presentations by faculty at local, provincial, and national educational conferences2083%4Involvement in development of a national simulation training guideline1875% Final list of items in the STRIVE evaluation model for HSCs Discussion Our STRIVE model developed the first national study using the Delphi method to create an evaluation model for healthcare simulation centers in China. The STRIVE model identified the 24 most frequently mentioned items across six categories. Each item can be scored, quantified, and compared among different HSCs. The STRIVE model offers several advantages. The model was created based on the consensus of simulation center directors across different parts of the country, different types of hospitals, and different ages of the simulation center. The STRIVE model is generally applicable in China. To achieve diversification, we included various types of HSCs, ranging from those in specialist hospitals to those in general hospitals, and from tertiary hospitals to primary hospitals. Additionally, we considered the influence of geographic location and regional economic conditions on the development of training centers. HSCs were intentionally selected from various regions in China, including both economically flourishing and less prosperous areas. This diverse sample selection enhances the generalizability of our evaluation model and allows its application to a wider range of HSC assessments. The conciseness and comparability of the STRIVE model are additional advantages. A key feature of the STRIVE model is its simplicity. In our model, the 24 finalized items are divided into six categories, with four items per category. To address differences in item importance within each domain, a weighted scoring system was adopted based on consensus from the steering group. Each category was assigned a total of 10 points, with two items weighted at 2 points and the other two weighted at 3 points. This approach allows for a more balanced reflection of the relative significance of each item. The total scores for each category are then plotted on a hexagonal radar scheme, facilitating visual interpretation and comparison. The hexagonal radar scheme was used to highlight and weigh data in a straightforward manner and is commonly used in other areas [16]. This allows users of the STRIVE evaluation model to easily identify their strengths and weaknesses. The STRIVE model provides a practical and context-specific approach to evaluating healthcare simulation centers (HSCs) in China. The model emphasizes clarity and usability, allowing for quick scoring and visual presentation through radar charts. This structure is designed to support self-assessment and institutional improvement. In contrast, international frameworks are generally more comprehensive and documentation-heavy. The Society for Simulation in Healthcare (SSH) accreditation includes seven domains—mission and governance, program management, resource management, human resources, program improvement, integrity, and expanding the field—with detailed requirements spanning 26 pages. Its focus is on programmatic accreditation and compliance processes [17]. The International Nursing Association for Clinical Simulation and Learning (INACSL) outlines ten standards in the Healthcare Simulation Standards of Best Practice, covering all stages of a simulation course: from professional development and prebriefing to simulation design, facilitation, debriefing, operations, outcomes, and evaluation. While detailed and widely adopted, these standards assess educational processes rather than overall institutional performance [18]. Similarly, the Society in Europe for Simulation Applied to Medicine (SESAM) groups its standards into two main areas: Core Values, which describe educational principles, and Scopes, which address course stages such as designing, teaching, debriefing, leadership, and research [19]. The Association for Simulated Practice in Healthcare (ASPiH) 2023 standards include both core values and domains for faculty, activity, and resource management, but again focus primarily on simulation activities rather than center-level assessment [20]. Specialty-based systems also exist, including the American College of Surgeons (ACS) Standards for Accredited Education Institutes and the American Society of Anesthesiologists (ASA) Simulation Program Endorsement, which provide criteria tailored to surgical and anesthesia training respectively [21, 22]. In China, a national expert consensus was issued in 2017, setting out nine domains and a total of 71 indicators (32 basic and 39 developmental) for simulation center development [23]. Compared to these models, STRIVE does not evaluate individual course phases in detail. Instead, it emphasizes a broader institutional perspective, including aspects such as community engagement and collaboration with industry for simulation device development. Its concise and structured format offers a time-efficient tool for benchmarking and continuous quality improvement. The first category was simulation curricula. The four highest-ranked items emphasize the vast variations in the training curriculum. HSCs are required to launch training programs for different levels of trainees, ranging from medical students to senior specialists. Regarding content, training curricula need to cover both technical and non-technical skills. The Italian Society for Simulation in Medicine has outlined the same requirement based on the ethical principle of “never first time on a patient.” Simulation curricula in Italy include both technical and non-technical training using simulation models with varying fidelity levels [24]. Mueller from the Steinberg Centre for Simulation and Interactive Learning also mentioned the diversity of simulation curriculum development at their center [25]. Lateef et al. concluded that the simulation center’s curricula, and not the simulators, define the center [26]. The STRIVE evaluation model thus reveals the need to cover all levels while designing the HSC curriculum, with good progression and inclusion training of both technical and non-technical skills. Another evaluation criterion in the STRIVE system is the significance of the simulation research. In the factor Training the faculty, a highly ranked item was about research. The importance was reiterated in the factor Results of training in terms of number of published articles and in the category of Effect of activities on faculty attendance at national, provincial, and local educational conferences. Medical education is becoming increasingly evidence-based, and simulation research is needed to determine the most efficient way to use simulations [27]. As the number of simulation centers increased, the number of simulation research publications increased dramatically [28]. To guide simulation research, the Society for Simulation in Healthcare Research presented the results of a Delphi survey and identified the priorities that must be addressed when implementing simulation programs [29]. Furthermore, publications in respected journals and presentations at national and international conferences can build the center’s reputation and attract further collaboration and investment [30]. The Variety of the activities of HSCs was another aspect mentioned by experts. The simulation program could include trainees from the community to improve the public’s resuscitation skills and increase the social impact of the center. For example, the Basic Life Support (BLS) program has been widely adopted by simulation centers and made available to the public [31]. There are other interesting attempts at functional diversity. Han et al. developed a simulation-based empathy enhancement program for nonmedical caregivers of older adults that resulted in lower levels of stress and burnout [32]. Using a simulation program, Martinelli et al. increased the understanding and support of anesthetists by their family and friends [33]. As mentioned in the STRIVE model, simulation educators are encouraged to collaborate with the medical device and simulation industries to develop innovative devices, such as virtual reality systems, and to apply for patents on inventions. As Munz noted, the change in simulation training must be led by simulation educators and not by the industry [34]. The versatility of the training center’s functions is also reflected in its ability to develop simulation training tools. Limitations The key opinion leaders who participated in this study were all from China, which limits the generalizability of the results. Some assessment items can only be used for HSCs in China. However, we plan to expand the study internationally and invite simulation center directors from other countries to participate in the development of a global version. Another limitation of our study is that we included only the four most highly rated items in each category. To ensure the tool remains practical and widely applicable, we intentionally prioritized brevity in the questionnaire design. Evidence from prior studies consistently demonstrates that shorter questionnaires are associated with higher response and completion rates without compromising data reliability [35, 36].For instance, in a comparison of three versions of the Research Participant Perception Survey containing 13 (Ultrashort), 25 (Short), and 72 (Long) items respectively, the shorter surveys achieved higher response and completion rates (Ultrashort: 64%/63%; Short: 63%/54%; Long: 51%/37%), while maintaining good reliability [37]. In addition to considerations of feasibility and response quality, we also referred to psychometric literature in determining the number of items per category. It is generally recommended that a minimum of three items per scale be used to ensure reliable convergence in confirmatory factor analysis.However, it is advisable to include an additional item to ensure the robustness of the construct. As such, using four items per category is a prudent and widely accepted approach where feasible [38]. Taken together, these considerations informed our decision to include the top four items in each category of the STRIVE model. This design balances psychometric rigor with practical efficiency and enhances the usability of the tool across diverse settings. Future perspectives Our team is currently conducting a follow-up validity study of the STRIVE model by collecting data from simulation centers across 40 hospitals in China. Using the weighted scoring system, we have completed preliminary assessments of center performance. The initial results are promising and support the tool’s practical applicability. These findings will help provide a more objective and comprehensive overview of the current status of HSCs in China and inform future refinements of the model. Conclusions In this study, the Delphi method was used to identify the most appropriate aspects for evaluating HSCs. A total of 24 items, divided into six categories and abbreviated as the STRIVE model, were included. The STRIVE model can help HSCs managers understand and evaluate their centers more comprehensively. Supplementary Information Supplementary Material 1. Supplementary Material 2. Supplementary Material 3. Supplementary Material 4. Acknowledgements We would like to thank all the teachers who participated in the questionnaire survey, and the staff at Renji Hospital Simulation Training Center who assisted with this research. Abbreviations Healthcare simulation center Key opinion leaders Simulation-based medical education Society for Simulation in Healthcare Simulation curricula, Training of faculty, Result of training, Internal resources, Variety of activities, Effect of activities Authors’ contributions ZY drafted the manuscript, LYH and MYF collected the data, QXL and MC conducted data analysis and interpretation, LK and YHY reviewed and revised the article, and BZQ designed the entire work. All authors approved the final version of the manuscript. Funding Not applicable. Data availability No datasets were generated or analysed during the current study. Declarations Ethics approval and consent to participate This study was based on questionnaires sent to the heads of the simulation centers; therefore, neither patients nor the public were involved in the design, or conduct, or reporting or dissemination plans of this research. Ethics approval for this study was not required as determined by the Shanghai Jiaotong University school of Medicine, Renji Hospital Ethics Committee. The reference number for this decision is EX-2024-004. We obtained informed consent from all the individuals who took part in the study. Participants expressed their consent by filling out and returning the questionnaire, which contained a statement notifying them that their response indicated their agreement to take part in the study.They were also made aware that they could withdraw from the study at any time without incurring any consequences. All contributors mentioned in the research paper remain unidentified. Consent for publication Not applicable. Competing interests The authors declare that they have no competing interests. Footnotes Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Supplementary Materials Supplementary Material 1. Supplementary Material 2. Supplementary Material 3. Supplementary Material 4. Data Availability Statement No datasets were generated or analysed during the current study.