Proposal of the Implementation Theory Selection Model and exemplar application in fall injury prevention Authors: Alexandra M. B. Korall, Helen Chong, Vicki Komisar, Dawn C. Mackey, Masood Khan, Femke Hoekstra, Susan G. Brown, Pauli Gardner, Christine Hames, Andrew C. Laing, Kathryn M. Sibley, Ivan Sarmiento, Ivan Sarmiento, Ivan Sarmiento, Ivan Sarmiento Published: 2024 ABSTRACT IntroductionThe use of theories, models and/or frameworks (TMFs) in implementation research and practice is essential for developing useful and testable implementation strategies. Recommendations and tools exist to aid implementation groups in selecting TMFs, but they do not explicitly outline a systematic method for identifying and selecting TMFs. This paper aimed to (1) propose a systematic consensus-based method to select TMFs to support implementation processes, and to (2) demonstrate the use of this novel method in the context of researching the implementation of hip protectors for fracture prevention in long-term care (LTC).Materials & methodsWe developed a systematic, consensus-based method for selecting TMFs, referred to as the Implementation Theory Selection Model (ITSM). The ITSM comprises five steps: (1) identify potentially relevant TMFs; (2) narrow the pool of TMFs; (3) appraise the relevance of eligible TMFs; (4) prioritize a short-list of TMFs for further, in-depth consideration; and (5) select TMFs through consensus with investigators and research user partners. We operationalized each step of the ITSM through a project investigating determinants of hip protector use and organizational readiness for implementation in a LTC organization in Ontario, Canada.ResultsUsing the ITSM in our case example, we identified 66 TMFs (Step 1). Of these, 23 met our eligibility criteria (Step 2) and were appraised twice, by five appraisers (Step 3). Six TMFs (Step 4) advanced to the consensus meeting, which was attended by nine investigators and three research users, including two organizational partners and one older adult. Three rounds of voting yielded a tie between the TMFs the group felt would be most appropriate. Research users from our partner LTC organization made the final selection preferring the combination of the Practical, Robust Implementation and Sustainability Model and Consolidated Framework for Implementation Research (Step 5).ConclusionsThe ITSM offers a step-by-step guide for implementation groups to adopt a rigorous, transparent and reproducible method for TMF selection. Although we have demonstrated the feasibility of operationalizing each step of the ITSM in our case example, continued research is needed to evaluate and refine the ITSM to ensure it is appropriate for a wide variety of implementation contexts. Introduction Implementation scientists have long advocated for the meaningful use of theories, models and/or frameworks (TMFs) in implementation research and practice [1, 2]. The use of TMFs is essential for developing useful and testable implementation strategies [2], promoting generalizability of findings across diverse settings through shared language and understanding [3], and facilitating knowledge synthesis and the accumulation of an evidence-base through standardized construct definitions and measurement [4]. Most implementation research, however, is uninformed, or ineffectively informed, by TMFs [5–7]. TMFs are often underused, misused or superficially used [8]. Implementation TMFs have varied aims. Some are useful for guiding the process of implementation, some for understanding and/or explaining what influences implementation outcomes, and others for evaluating implementation outcomes [9]. However, TMFs with a similar aim often contain different constructs, terminologies, and definitions [10]. The choice of TMFs can affect the process and outcomes of implementation. No single TMF, or combination of TMFs, is considered superior over others in its suitability to inform all implementation research, and the question of “Which TMF(s) should I use?” has persisted for decades [2]. The complexity of selecting implementation TMFs is amplified by the number of existing TMFs to choose from [8]. For example, a scoping review identified 159 TMFs that have been used, many only once (60%), in published implementation research for preventing and managing cancer and chronic diseases [11]. Implementation teams report that they consider a variety of criteria when selecting TMFs [8], and involving organizational and community partners in research decision-making, which includes TMF selection, has been proposed as a critical component of implementation [12]. However, there is little agreement as to which criteria are most important, and the ultimate selection of TMFs is often determined primarily by convenience or familiarity [8]. Recommendations and tools exist to help choose TMFs. Moulin et al. (2020) recommend considering the purpose, analytic level, degree of specificity and orientation of TMFs [4]. Scientists from Canada, the United Kingdom, and the United States developed the Theory Comparison and Selection Tool (T-CaST), which contains 16 criteria across four categories (applicability, usability, testability, and acceptability) that can be used to justify selection of a TMF for a given project [13]. While these resources offer valuable guidance, they do not operationalize a systematic method for identifying and selecting TMFs, which can be effectively replicated across implementation efforts and contexts. This paper aimed to (1) propose a systematic, consensus-based method to identify and select TMFs to support implementation processes, and to (2) demonstrate the use of this novel method in the context of researching the implementation of hip protectors for fracture prevention in long-term care (LTC). Materials and methods Implementation context Globally, approximately 1.6 million older people sustain a hip fracture each year [14]. Falls cause 95% of hip fractures in older people [15]. The consequences of hip fracture can be especially severe for LTC residents [16–19]. Older people who fracture their hip experience heightened morbidity and mortality [20–24], reduced quality of life [25, 26], and require greater health care use (e.g., emergency department visits, surgery, physician contact) [27]. Hip protectors are one strategy for managing falls and preventing hip fractures in LTC. They consist of hard shields or soft pads sewn or inserted into elasticized pockets of garments or undergarments, which cover the skin over the lateral aspects of the proximal femur. Hip protectors are designed to reduce the risk of hip fracture from falls by absorbing and/or shunting the impact energy away from the proximal femur [28]. Although the efficacy of commercial models of hip protectors varies [29], certain models likely reduce the risk for hip fractures in LTC when used as intended [30, 31]. Hip protectors have been recommended for older people in LTC, especially for those who are mobile and at high risk of falls [32, 33], but they only protect against hip fracture if worn during the fall. Research suggests hip protectors are not used consistently in LTC, which limits their clinical effectiveness [34]. There is a need for implementation research on how to enhance uptake and sustain the use of hip protectors in LTC [35]. Philosophical foundation and theoretical underpinnings Our philosophical approach to this work was rooted in pragmatism, a paradigm that aims to use research findings to solve practical real-world problems [36]. We aimed to develop and apply a systematic method to identify and select TMFs in the context of researching the implementation of hip protectors for fracture prevention in LTC. Consistent with an Integrated Knowledge Translation (IKT) model of collaborative research, our method considers the views of research users (those able to use research findings to make decisions) when reaching consensus on the final selection of TMFs [37, 38]. The goal is to receive and incorporate the feedback and advice of research users when deciding on the final choice of TMFs. Our systematic method for TMF identification and selection was guided by the James Lind Alliance (JLA) process (https://www.jla.nihr.ac.uk/jla-guidebook/). The JLA brings patients, clinicians, and care partners together in Priority Setting Partnerships to reach consensus on the most important unanswered questions for future research. Research ethics statement The individuals involved in the theory selection process were not human research participants, but instead were partners on the research team (level of ‘consult’ on the IAP2 Spectrum of Patient and Researcher Engagement in Health Research). Given the collaborative decision making role of these partners in the research process, research ethics board (REB) approval was not sought. This practice was consistent with the JLA guidance that views such decision making work as service evaluation and development (https://www.jla.nihr.ac.uk/jla-guidebook/). Overview of the Implementation Theory Selection Model (ITSM) We developed a systematic, consensus-based method for identifying and selecting TMFs, referred to as the Implementation Theory Selection Model (ITSM). The ITSM comprises five steps: (step 1) identify potentially relevant TMFs; (step 2) narrow the pool of TMFs to only those with an appropriate aim; (step 3) appraise the relevance of eligible TMFs; (step 4) prioritize a short-list of TMFs for in-depth consideration; and (step 5) select TMFs through consensus with investigators and research user partners (Fig 1). Table 1 summarizes each step of the ITSM. Illustration of the Implementation Theory Selection Model (ITSM). TMF = Theory, Model and/or Framework. 10.1371/journal.pone.0310117.t001Table 1Steps of the Implementation Theory Selection Model (ITSM).StepGoalDescription1Identify potentially relevant TMFs• Search multiple sources to identify a pool of potentially relevant TMFs, relying on existing works to expedite the process whenever possible• Example sources include: database of Dissemination & Implementation (D&I) Models in Health Research and Practice (http://www.dissemination-implementation.org); published reviews and academic articles of implementation TMFs, ideally including some conducted with similar target populations, interventions or settings2Narrow the pool of TMFs• At a minimum, narrow the pool of TMFs identified to only those that have an appropriate aim or purpose, depending on implementation context• Consider narrowing the pool further based on the socio-ecological level(s) included within the TMF, the inclusion and depth of analysis or operationalization of specific implementation constructs, and/or the orientation of the TMF based on the type of intervention and setting• Define eligibility criteria, based on aim, etc.• Retrieve at least one reference (article, website, etc.) per TMF• Screen the references, ideally by at least two reviewers, to determine whether each TMF meets the eligibility criteria3Appraise the relevance of eligible TMFs• Process spans the recruitment of appraisers, (random, if possible) assignment of TMFs to appraisers, compilation and review of references, appraisal of each TMF• Recommend using the Theory Comparison and Selection Tool (T-CaST) to guide the appraisal of TMFs• Ideally, each TMF appraised by at least two appraisers4Prioritize short-lists of the most relevant TMFs for the implementation objective(s)• Specify a manageable number of TMFs (we recommend no more than six) for further, in-depth consideration• Use the results of TMF appraisals to rank TMFs from most to least relevant for each implementation objective5Select TMFs through consensus with investigators and research user partners• Use formal methods of consensus, such as an adapted Nominal Group Technique (NGT) or Delphi Technique, with a group of investigators and research user partners to reach consensus on the final choice of TMF(s)TMF = Theory, Model and/or Framework. Steps of the Implementation Theory Selection Model (ITSM). TMF = Theory, Model and/or Framework. The ITSM was informed by the JLA method for priority setting (https://www.jla.nihr.ac.uk/jla-guidebook/). The JLA method includes steps for gathering data on potential research questions, evidence checking, interim priority setting, and the final prioritization of research questions through a workshop. AMBK and KMS proposed a first iteration of the ITSM’s five steps, which were revised through consultation with our academic and organizational team members. Once the five steps of the ITSM were finalized, AMBK and KMS proposed the sub-steps to carry out each step of the ITSM in our case example on hip protector implementation, which was revised iteratively through discussions with our research team as the process evolved. In the sections below, we describe and provide examples of how we operationalized each step of the ITSM in our case example on hip protector implementation to illustrate the feasibility and potential value of the ITSM. We encourage ITSM users to work collaboratively with research user partners to develop the specific sub-steps they will use to operationalize the ITSM’s five steps which are feasible given their access to available resources. Step 1: Identify a pool of potentially relevant TMFs The first step of the ITSM requires the use of multiple search strategies to identify a pool of potentially relevant TMFs, relying on existing works to expedite the process whenever possible. A recommended source is the database of Dissemination & Implementation (D&I) Models in Health Research and Practice (http://www.dissemination-implementation.org). This database can be filtered using pre-specified search criteria related to the purpose of the TMF (e.g., “Implementation”), socio-ecological level of influence (e.g., “Individual), field of origin (e.g., “Nursing”), and the presence of specific constructs (e.g., “Adoption”). Complementary sources may include published reviews and academic articles identifying TMFs used in implementation research and practice, ideally including some conducted with similar target populations, interventions or settings, as these characteristics are relatively easy to assess. In general, we advise against being overly prescriptive when developing a list of candidate TMFs, as the association between characteristics of a TMF and its ultimate relevance will be unclear at this step. Case example. In October 2019, we searched the database of Dissemination & Implementation (D&I) Models in Health Research and Practice (http://www.dissemination-implementation.org) using the pre-specified search criteria of “Implementation” (purpose of TMF), “Individual” (socio-ecological level) and “Organization” (socio-ecological level). To complement this search, we retrieved all TMFs identified in a scoping review of implementation research on cancer and chronic disease management and prevention, which were cited at least 10 times [11], along with the most used TMFs identified in a survey of implementation scientists [8]. Our decision to include TMFs with ten or more citations from the scoping review was pragmatic; our goal was to ensure we weren’t missing TMFs that are commonly used in a related field of interest. Finally, we retrieved all TMFs identified in a literature review of implementation research on the prevention and management of falls in older adults [39]. We elected to look beyond this review of implementation research on falls in older adults to compile a comprehensive list of candidate TMFs. Step 2: Narrow the pool of TMFs Implementation TMFs have been categorized into different categories based on their aim or purpose. These include process models, determinant frameworks, classic theories, implementation theories, and evaluation frameworks [9]. Process models describe or guide the implementation process [9]. Determinant frameworks, classic theories and implementation theories specify factors that impact implementation outcomes, describe mechanisms of change, and/or explain certain aspects of implementation [9]. Evaluation frameworks provide a structure for evaluating the success or failure of implementation [9]. The second step of the ITSM requires that the pool of TMFs identified in step 1 is narrowed, at a minimum, to only those that have an appropriate aim or purpose. The pool could be narrowed further based on the socio-ecological level(s) included within the TMF, the inclusion and depth of analysis or operationalization of specific implementation constructs, and/or the orientation of the TMF based on the type of intervention and setting [4]. Once the appropriate criteria have been identified, all TMFs should be screened using a systematic process to ensure they fit this criteria. This requires the retrieval of at least one reference for each TMF. As it can be somewhat challenging to make decisions regarding the eligibility of TMFs, ideally, at least two reviewers should independently screen each TMF, and discrepancies should be resolved through discussion or a third independent reviewer. When a complete dual review process is not feasible, single screening should be conducted only by an experienced reviewer [40]. Case example Our case example had two implementation objectives: to (1) investigate determinants of hip protector use; and (2) assess organizational readiness for implementation of hip protectors. We narrowed our search to determinant frameworks and implementation theories since their aims most closely aligned with our objectives. We excluded TMFs when we could not retrieve at least one reference for the TMF. Two reviewers (AT and AMBK) independently screened a sample of ten TMFs, and then met to compare decisions and discuss and resolve discrepancies. A single reviewer (AT) then screened all remaining TMFs for eligibility. To ensure TMFs were not missed by the first reviewer, the subset of TMFs deemed ineligible or unclear by AT were screened by an experienced reviewer (AMBK) to verify their eligibility. Step 3: Appraise the relevance of TMFs The third step of the ITSM appraises the relevance of each eligible TMF. This process spans the recruitment of appraisers, the (random, if possible) assignment of TMFs to appraisers, the compilation and review of references (e.g., academic articles, websites, questionnaires, etc.), and the appraisal of TMFs. We recommend using the T-CaST to assess the relevance of each TMF for the implementation objective(s) [13]. We adapted the original 16-item version of the T-CaST for Implementation Researchers [13] to create an Appraisal Form that can be used to appraise the relevance of a given TMF (SI File). A first iteration of the Appraisal Form was piloted by five team members, on two TMFs. After independently appraising both TMFs, the team met remotely via a two-hour online meeting to discuss and refine the Appraisal Form. Table 2 describes and provides rationales for modifications made to original T-CaST items. The final Appraisal Form contains 12 items probing the usability (5 items), applicability (4 items), and testability (3 items) of eligible TMFs. Appraisers indicate the extent to which they disagree or agree with each statement by responding on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). The Appraisal Form also contains items probing the overall perceived fit of the TMF to each implementation objective. Appraisers rate the fit of the TMF on a 3-point Likert scale, ranging from 0 (poor fit) to 2 (good fit). Ideally, each TMF should be appraised by at least two appraisers. 10.1371/journal.pone.0310117.t002Table 2Items contained in the Theory Comparison and Selection Tool (T-CaST) for implementation researchers, items included in our appraisal form, and reasons for modifying or excluding T-CaST items.Original T-CaST itemItem included in Appraisal FormRational for modification/exclusion Usability 1TMF includes relevant constructs (e.g., self-efficacy; climate)TMF includes relevant constructs1 (e.g., readiness; any other constructs you perceive are relevant). 1Note that the term ‘constructs’ is used broadly to refer to the types of variables (also known as classes or domains), as well as specific variables within a category, class or domain that are contained in the TMF.Modification: Appraisers were initially constrained by examples contained in original item and requested clarification as to what is meant by the term ’constructs’2Key stakeholders (e.g., researchers; clinicians; funders) are able to understand, apply, and operationalize TMF.N/AExclusion: Appraisers were not confident in their ability to anticipate whether research users would be able to understand, apply and operationalize TMF3TMF has a clear and useful figure depicting included constructs and relationships among them.TMF has a clear and useful figure or table depicting included constructs and relationships among them. Modification: Appraisers expressed that a table could be equally useful as a figure4TMF provides a step-by-step approach for applying it.TMF provides a step-by-step approach for applying it.N/A5TMF provides methods for promoting implementation in practice.TMF provides methods for promoting implementation in practice.N/A6TMF provides an explanation of how included constructs influence implementation and/or each other.TMF provides an explanation of how included constructs influence implementation and/or each other.N/A Testability 7TMF proposes testable hypotheses.TMF proposes testable hypotheses.N/A8TMF includes meaningful, face-valid explanations of proposed relationships.TMF includes meaningful, face-valid explanations of proposed relationships and constructs. Modification: Appraisers expressed that it was important for TMFs to also include meaningful, face-valid explanations of constructs contained in the TMF9TMF contributes to an evidence base and/or theory development because it has been used in empirical studies.TMF contributes to an evidence base and/or theory development because it has been used in empirical studies.N/A Applicability 10TMF focuses on a relevant implementation outcome (e.g., fidelity; acceptability).TMF focuses on a relevant implementation outcome (e.g., acceptability; adoption; appropriateness; feasibility; fidelity; implementation cost; penetration; sustainability). Modification: Appraisers were initially constrained by examples contained in original item11A particular method (e.g., interviews; surveys; focus groups; chart review) can be used with TMF.N/AExclusion: Appraisers expressed that investigators have expertise in mutliple research methods and compatability of TMF with a particular method was inconsequential12TMF addresses a relevant analytic level (e.g., individual; organizational; community).TMF addresses one or more relevant analytic levels (e.g., individual; organization). Modification: Appraisers expressed a desire for specific examples of relevant analytic levels and clarification that a single TMF need not address all relevant analytic levels13TMF has been used in a relevant population (e.g., children; adults with serious mental illness) and/or conditions (e.g., attention deficit hyperactivity disorder; cancer).TMF is generalizable to our context. Modification: Appraisers expressed that it was not feasible to assess what populations and/or conditions a TMF has been used in without doing a review of the literature14TMF is generalizable to other disciplines (e.g., education; health services; social work), settings (e.g., schools; hospitals; community-based organizations), and/or populations (e.g., children; adults with serious mental illness).TMF is generalizable to other disciplines and/or contexts. Modification: Appraisers were initially constrained by the examples contained in the original item and expressed a desire to simplify the statement Acceptability 15TMF is familiar to key stakeholders (e.g., researchers; scholars; clinicians; funders).N/AExclusion: Appraisers were not confident in their ability to anticipate whether stakeholders are familiar with a given TMF16TMF comes from a particular discipline (e.g., education; health services; social work).N/AExclusion: Appraisers expressed the discipline of origin of a TMF was inconsequentialTMF = Theory, Model and/or Framework. Items contained in the Theory Comparison and Selection Tool (T-CaST) for implementation researchers, items included in our appraisal form, and reasons for modifying or excluding T-CaST items. TMF = Theory, Model and/or Framework. Case example We invited six investigators and one research user from our partner LTC organization to participate in the appraisal of TMFs. We asked each appraiser to self-identify as either a novice or advanced appraiser. We defined a novice appraiser as having little to no familiarity with implementation TMFs, and an advanced appraiser as having some to substantial familiarity with implementation TMFs. We randomly assigned TMFs to appraisers using block randomization, ensuring that each TMF was appraised by one advanced and one novice appraiser. AMBK emailed appraisers a folder containing high-priority (recommended) and low-priority (optional) references for each TMF. We prioritized references that described the initial development and/or testing of the TMF, subsequent iterations of the TMF, along with example applications in the LTC setting. Based on feedback from pilot testing, no more than four references were categorized as high-priority for each TMF. We asked appraisers to read the high-priority references, at a minimum, before completing the Appraisal Form. Appraisers were also permitted to search for and retrieve additional references for a given TMF, as needed. Step 4: Prioritize short-lists of the most relevant TMFs for the implementation objective(s) Once all TMFs have been appraised, Step 4 of the ITSM uses data from Step 3 to prioritize short-lists of the most relevant TMFs for the implementation objective(s). The first step is to specify a manageable number of TMFs for further, in-depth consideration. This will vary across teams, but we advise shortlisting no more than six TMFs to advance to Step 5. Then, using the data collected during the appraisal process, TMFs should be ranked by a member of the research team from most to least relevant for each implementation objective. Case example. We short-listed three TMFs per implementation objective to advance to step 5. We calculated the mean fit of each TMF per implementation objective by averaging 3-point Likert-scale responses across appraisers. For each TMF, we also calculated a mean overall T-CaST score. This was done by: (1) calculating an individual appraiser’s mean usability, testability, and applicability scores; (2) summing an individual appraiser’s mean usability, applicability, and testability scores to derive an overall T-CaST score per appraiser (value range: 3–15); (4) averaging overall T-CaST scores across appraisers. Mean fit scores varied by implementation objective, whereas overall T-CaST scores did not. We ranked TMFs according to their mean fit score for a given implementation objective, and then by mean overall T-CaST score. Data were analyzed using JMP PRO version 14 (SAS Institute Inc, Cary, NC). Step 5: Select TMFs through consensus Step 5 of the ITSM reaches consensus on the final choice of TMF(s). Consensus should be reached jointly by a team of investigators and research user partners using formal methods of consensus, such as an adapted Nominal Group Technique (NGT) or Delphi Technique. The NGT uses face-to-face communication, and includes six stages: (1) formulation and presentation of a question; (2) silent idea generation; (3) round robin feedback and recording of ideas; (4) group discussion; (5) individual voting or polling; (6) tallying of votes, feedback of results, further group discussion (optional) and re-voting (optional) [41]. The Delphi Technique uses a series of questionnaires to reach consensus, often spanning weeks to months [42]. In general, the NGT is better suited when consensus is needed quickly (i.e., one day) and from a smaller group (less than 15) compared to the Delphi Technique [42]. Case example We invited all investigators and research user partners (n = 21) to attend a virtual, two-hour consensus meeting. We sent a formal invitation and reminder emails from the email account of our research manager, HC. Ten days before the consensus meeting, we distributed summaries that we created of the short-listed TMFs, along with examples of guiding questions (developed by AMBK and KMS) to facilitate participation in the consensus meeting. Summaries were two pages, and included a summary statement of the TMF, a graphical illustration of the TMF, an overview of key constructs contained in the TMF, the year the TMF was first published, the estimated number of times the TMF had been cited (abstracted from https://dissemination-implementation.org or Google Scholar), example applications of the TMF (applications in LTC setting and similar health conditions prioritized), and a summary of the appraisal scores, including mean fit per implementation objective, along with mean usability, applicability, and testability scores (see S2 File for an example). We also emailed references for each TMF under consideration, but these readings were optional. We encouraged attendees to send any questions via email and/or to arrange a time to chat virtually with AMBK prior to the consensus meeting. The day before the consensus meeting, we emailed attendees a copy of the meeting agenda, a summary of the consensus process, and guidelines for discussion. The consensus meeting was hosted online using virtual meeting software (Zoom) and was recorded. We hired an independent postdoctoral fellow (FH) to facilitate, who had experience with implementation TMFs (the Delbecq technique [41]). The facilitator began by reviewing the agenda (S3 File) and establishing ground rules for discussion. We used an adapted NGT [41] with three rounds of consensus. We used NGT as we wanted to reach consensus quickly, while still considering everyone’s opinions. Table 3 lists the questions asked in each round, which were shared with attendees before the meeting. 10.1371/journal.pone.0310117.t003Table 3Questions asked during each round of consensus in our case example.RoundQuestion1Which of the following TMFs is least relevant for guiding our initial investigation of barriers and facilitators to the use of hip protectors in [our partner] long-term care homes?2Which of the following TMFs is least relevant for guiding our initial exploration of organizational readiness to implement, monitor and evaluate strategies to increase use of hip protectors in [our partner] long-term care homes?3What is your preferred combination of TMFs to collectively underpin our objectives?TMF = Theory, Model and/or Framework. Questions asked during each round of consensus in our case example. TMF = Theory, Model and/or Framework. Round 1 and Round 2 were unconventional, parallel rounds, as the outcome of the first did not influence the outcome of the second. The goal of the first round was to eliminate one of the top three TMFs for our first implementation objective to investigate factors influencing use of hip protectors. The goal of the second round was to eliminate one of the top three TMFs for our second objective to assess organizational readiness for implementation. The facilitator began Round 1 and Round 2 by presenting a question to the group (Table 3). AMBK then presented a brief overview (1 minute per TMF) of the top three TMFs for each implementation objective, after which attendees could ask clarifying questions about the TMFs under consideration. As we asked attendees to come to the meeting prepared to share their views to the questions posed in the first two rounds, we did not allocate time for silent idea generation. Then, each participant was given an opportunity to contribute their views about which TMF was least relevant for either objective, depending on the round, and why. A note taker summarized each participant’s views in the chat window. A discussion followed exploring areas of agreement and divergence and the clarification of any uncertainties. Attendees indicated if they wanted to make a comment by clicking a virtual hand raise button and were provided one minute to make comments. After the discussions, attendees were asked to vote for the least relevant TMF for each implementation objective by clicking on links posted in the chat window to online questionnaires hosted by Survey Monkey. Fig 2 illustrates the voting process for Round 1 and Round 2. If the first poll did not yield a majority (more than 50%) of votes, then the two TMFs with the most votes to be eliminated advanced to a second poll to declare which TMF would be eliminated [43]. We broke ties using mean overall T-CaST scores derived from TMF appraisals. In the event of a three-way tie in the first poll, the two TMFs with the lowest mean overall T-CaST scores advanced to a second poll. In the event of a two-way tie in the second poll, the TMF with the lowest mean overall T-CaST score was eliminated. We tallied and presented the voting results at the end of each round. Elimination process for the first two parallel rounds of consensus in our case example. TMF = Theory, Model and/or Framework. The third round aimed to agree upon a final selection of TMFs to collectively underpin our implementation objectives. The facilitator instructed attendees that combinations of TMFs must include at least one of the remaining two TMFs from round 1, and at least one from round 2. We allocated five minutes for silent idea generation. Then, each participant was given an opportunity to contribute their views, and a summary of their comments was posted in the chat window. This was followed by a discussion period. We did not use live polling for voting. Votes could be submitted up to one week after the meeting via an online questionnaire administered via Survey Monkey, with links posted to the chat window and emailed to each attendee. We asked attendees to identify their first, second and third choice combinations of TMFs. We assigned three points for every first-choice vote, two points for every second-choice vote, and one point for every third-choice vote. We summed the total points assigned to each possible combination of TMFs (nine possible combinations) and ordered these from most to least points. The results were tallied and presented to the group in a follow-up meeting. The combination of TMFs with the most points was selected. In the event of a tie, we asked research users from our partner LTC organization (CH and SB) to decide which of the highest ranked combinations of TMFs they preferred. To inform their decision-making, we provided our partners with: (1) summaries of the TMFs still under consideration; (2) references for the TMFs still under consideration; (3) voting results and a summary of the group discussions from all three rounds of the consensus meeting. Results Case example The flow of TMFs through the ITSM in our case example is shown in Fig 3. Flow of TMFs through the Implementation Theory Selection Model (ITSM) in our case example. TMF = Theory, Model and/or Framework. Step 1: Identify a pool of potentially relevant TMFs After removing duplicates, we identified 66 potentially relevant TMFs. Step 2: Narrow the pool of TMFs Of the 66 potentially relevant TMFs, 23 TMFs met our eligibility criteria, including 9 determinant frameworks and 14 implementation theories. Reasons for exclusion were as follows: could not retrieve at least one seminal source (n = 1); classic theory (n = 15); process model (n = 19), evaluation framework (n = 3) or other aim of TMF (n = 5). The names and categorized aims of included and excluded TMFs are listed in S4 and S5 Files, respectively. Step 3: Appraise the relevance of TMFs Five investigators appraised the 23 eligible TMFs (S6 File). Two self-identified as advanced appraisers, while three self-identified as novice appraisers. Appraisers had similar qualifications in kinesiology and engineering. The highest degree earned was a Ph.D. for four appraisers, and a M.Sc. for one appraiser. The appraisal process spanned two months. Mean fit scores ranged from 0.5 to 2 (out of 2), and from 0 to 2 (out of 2) for objectives 1 and 2, respectively. Four TMFs had a mean fit score of 2 (“good fit”) for objective 1, and two TMFs had a mean fit score of 2 (“good fit”) for objective 2. Mean usability, testability, and applicability scores ranged from 1.8–4.4 (out of 5), 1.8–4.7 (out of 5), and 2.4–4.9 (out of 5) across TMFs, respectively. Step 4: Prioritize short-lists of the most relevant TMFs for the implementation objective(s) Table 4 shows the ranked order of TMFs by relevance, from most (rank = 1) to least (rank = 23) relevant, for each implementation objective. The Top 3 TMFs for objective 1 were the Normalisation Process Theory (NPT) [44], the Exploration, Preparation, Implementation, Sustainment (EPIS) Model [45, 46] and the Practical, Robust Implementation and Sustainability Model (PRISM) [47]. The Top 3 TMFs for objective 2 were the Organizational Readiness Theory [48], the Consolidated Framework for Implementation Research (CFIR) [10] and the Precede-Proceed Model [49]. 10.1371/journal.pone.0310117.t004Table 4Ranked order of the perceived relevance of theories, models and/or frameworks for the implementation objectives in our case example.Theory, Model or FrameworkRankObjective 1Objective 2Normalization Process Theory (NPT)1a4Exploration, Preparation, Implementation and Sustainability (EPIS) Model2a5Practical, Robust Implementation and Sustainability Model (PRISM)3a6Theoretical Domains Framework (TDF)419Precede-Proceed Model53aCritical Realism & the Arts Research Utilization Model (CRARUM)68Consolidated Framework for Implementation Research (CFIR)72aPromoting Action on Research Implementation in Health Services (PARiHS) framework87Dissemination of Evidence-based Interventions to Prevent Obesity917Organizational Readiness Theory (ORT)101aSticky Knowledge1114Push-Pull Capacity Model1215Implementation Drivers Framework1316Behaviour Change Wheel (BCW)1420Research Development Dissemination and Utilization Framework1511Implementation of Change in Health Care1623Active Implementation Framework179Implementation Effectiveness Model / Organizational Theory of Implementation of Innovations1810Conceptual Model of Implementation Research1921Interactive Systems Framework for Dissemination and Implementation2012Knowledge Transfer and Exchange2113Real World Dissemination2218Social Marketing Framework2322aTop 3 for each objective short-listed for consideration in the consensus meeting. Ranked order of the perceived relevance of theories, models and/or frameworks for the implementation objectives in our case example. aTop 3 for each objective short-listed for consideration in the consensus meeting. Step 5: Select TMFs through consensus Nine investigators and three research users, including two organizational partners and one older adult member of our community advisory board, attended the consensus meeting. Three investigators elected not to participate in group discussions or voting. Nine attendees voted in the first poll for Round 1. Of these, eight (89%) voted to eliminate the Normalization Process Theory, and one (11%) voted to eliminate PRISM. As the first poll of Round 1 yielded a majority of votes, NPT was eliminated. Eight attendees voted in the first poll for Round 2. Of these, seven (88%) voted to eliminate Precede-Proceed, and one (13%) voted to eliminate Organizational Readiness Theory. As the first poll of Round 2 yielded a majority of votes, Precede-Proceed was eliminated. The four TMFs that advanced to Round 3 were EPIS, PRISM, Organizational Readiness Theory, and CFIR. Six attendees voted in Round 3. Seven combinations of TMFs received at least one vote (Table 5). Combinations had to contain at least one of the remaining TMFs from Round 1 (EPIS and PRISM), and at least one of the remaining TMFs from Round 2 (ORT and CFIR). In general, combinations with two implementation TMFs received the most points. Two combinations each with two TMFs tied with the most points: (1) EPIS and Organizational Readiness Theory; (2) PRISM and CFIR. 10.1371/journal.pone.0310117.t005Table 5Number of first, second and third place votes received for each potential combination of theories, models and/or frameworks (TMFs) in the third round of consensus of our case example, and the resulting points assigned to each combination.CombinationNo. VotesPoints1st Place2nd Place3rd PlaceEPISa + ORTb0419PRISMc + CFIRd2119EPISa + CFIRd1014PRISMc + ORTb1025EPISa + PRISMc + ORTb1003EPISa + ORTb + CFIRd1105EPISa + PRISMc + ORTb + CFIRd0011aEPIS = Exploration, Preparation and Sustainability Model.bORT = Organizational Readiness Theory.cPRISM = Practical Robust Implementation and Sustainability Model.dCFIR = Consolidated Framework for Implementation Research. Number of first, second and third place votes received for each potential combination of theories, models and/or frameworks (TMFs) in the third round of consensus of our case example, and the resulting points assigned to each combination. aEPIS = Exploration, Preparation and Sustainability Model. bORT = Organizational Readiness Theory. cPRISM = Practical Robust Implementation and Sustainability Model. dCFIR = Consolidated Framework for Implementation Research. The tie between the top two TMF combinations was broken through decision by research users from our partner LTC organization. They selected the PRISM and CFIR to collectively underpin our implementation objectives. Research user partners explained that their primary reason for choosing this combination was they had experience with the CFIR and appreciated the patient-centered focus of the PRISM, which aligned with the philosophy of care in our partner LTC organization. Discussion Instead of relying on convenience or previous exposure, we developed a systematic, IKT and consensus-based method, referred to as the Implementation Theory Selection Model (ITSM), to guide the selection of TMFs for implementation processes. In a case example, we effectively applied the ITSM to select the PRISM and CFIR to collectively inform research on the implementation of hip protectors for fracture prevention in LTC. The value of the ITSM is best illustrated by our selection of the PRISM, which no investigators or research user partners were aware of before undertaking this process. In fact, team members had experience with only two of the six TMFs short-listed for the consensus meeting. It is unlikely we would have chosen the PRISM to inform our implementation objectives had we not undertaken this process. Before this process, several team members had experience with the CFIR. AMBK and KMS used the CFIR to inform their hypotheses and to analyze data in other research studies. Research user partners (SB, CH) used the CFIR to inform past implementation efforts in our partner LTC organization. It is worth mentioning that after a very systematic and rigorous process, CFIR was chosen in the end by research users partly because of previous exposure, despite our explicit attempt to de-emphasize this criterion. This is consistent with findings from Birken et al. (2017), who found that familiarity was an important criteria for selecting implementation TMFs [8]. Application of the ITSM in our case example helped investigators and research users to improve their understanding and awareness of implementation TMFs, and helped to connect members of the research team by acting as a community-building activity. In particular, this allowed investigators to engage research users early in the project, after conception but before protocol development. Aligned with an IKT model of research collaboration, we relied on research users to break the tie between the top two combinations of TMFs that surfaced in the final round of the consensus meeting. Research users expressed that this made them feel valued and provided them with an opportunity to contribute to research decision-making in a very tangible way. It is anticipated that early engagement may help facilitate meaningful engagement in later stages or phases of the research project. We are currently engaging with research users who attended the consensus meeting on a subsequent project assessing the readiness of LTC homes within our partner organization to implement hip protectors. Many questions about the acceptability, usability, appropriateness and ultimate impact of the ITSM remain unanswered. For example, does the ITSM meet the needs and expectations of implementation teams across a variety of implementation contexts? How practical is it for implementation teams to use the ITSM? What is the value-added of using the ITSM compared to traditional approaches to TMF selection? Does use of the ITSM result in the identification of different determinants of knowledge use, the selection of unique theory-based change methods and implementation strategies that more precisely address key determinants of change, and ultimately improved outcomes, when compared to “usual” methods for TMF selection? Such questions could be answered using methods adapted from ongoing studies (e.g. [50]) to examine the acceptability, usability, appropriateness and perceived impact of the ITSM. We acknowledge several drawbacks to the ITSM and in particular, the extensive time required to undertake this method while there are calls for rapid, responsive, and relevant health research [51–54]. The feasibility of widescale use the ITSM, in its present form, may be limited. For investigators wishing to adopt a simplified version of the ITSM, we recommend restricting the search strategy to identify a smaller number of potentially relevant TMFs. All six TMFs that were shortlisted and advanced to step 5 are contained in the inventory of Implementation and Dissemination Models in Health Research and Practice (https://dissemination-implementation.org). This was a key resource for identifying TMFs. We also recommend defining stricter eligibility criteria for screening (step 2), so fewer TMFs advance to appraisal (step 3). As we estimate it took a minimum of one hour for one appraiser to appraise one TMF, the entire appraisal process was very labor-intensive (23 TMFs x 2 appraisers/TMF x 1 hour/appraiser = 46 person hours). We narrowed the pool of TMFs to only those with an appropriate aim, but this could be narrowed further based on the socio-ecological level(s) included within the TMF, the inclusion and depth of analysis or operationalization of specific implementation constructs, and/or the orientation of the TMF based on the type of intervention and setting. We encountered challenges engaging research users in the selection of guiding frameworks. We found it difficult to engage the older adult members of our community advisory board in the consensus meeting. Only one older adult member attended, who voted only in the first round of the meeting, and who did not participate in all rounds of discussion. After the meeting, several attendees in the consensus meeting commented that they valued the insight of our older adult partner, and how it swayed their own votes. It would have been even more difficult to engage residents of LTC, even though their insight would be so important. The challenges encountered likely resulted from our use of more consultative and extractive approaches to patient and public engagement, as well as insufficient efforts to address power imbalances between academic researchers, organizational partners and patient partners. For those wishing to engage patients and members of the public more meaningfully in the selection of theory, a more democratic, equitable approach to research ‘co-production’ is needed that prioritizes the unique and vital perspectives of patients over more conventional research methods and understandings of what constitutes ‘academic’ research [55]. Despite these limitations, we believe the ITSM offers a practical, step-by-step guide for implementation groups to adopt a rigorous, transparent and reproducible method for TMF selection. It incorporates best practices for the selection of theory in implementation research and practice, by relying on existing tools (e.g., T-CaST, inventory of Implementation and Dissemination Models in Health Research and Practice), adhering to existing recommendations, and incorporating the feedback and advice of research users when reaching consensus on the final choice of TMFs. Although we have demonstrated the feasibility of operationalizing each step of the ITSM in our case example, continued research is needed to evaluate and refine the ITSM to ensure it is appropriate for a wide variety of implementation contexts. Conclusions We developed a systematic, consensus-based method for identifying and selecting TMFs, referred to as the Implementation Theory Selection Model (ITSM). The ITSM comprises five steps: (step 1) identify potentially relevant TMFs; (step 2) narrow the pool of TMFs; (step 3) appraise eligible TMFs; (step 4) prioritize a short-list of TMFs for in-depth consideration; and (step 5) select TMFs through consensus with investigators and research user partners. We then applied the ITSM to choose TMFs to inform research on the implementation of hip protectors for fracture prevention in LTC. This resulted in the selection of the PRISM and CFIR. The value of the ITSM is best reflected by our selection of the PRISM, which no investigators and researcher users were familiar with before undertaking this process. The ITSM offers a practical, step-by-step guide for implementation groups to adopt a rigorous, transparent and reproducible method for TMF selection, although continued research is needed to evaluate and refine this method to ensure it is appropriate for a wide variety of implementation contexts. Supporting information Theory, Model and/or Framework (TMF) appraisal form. (PDF) Example summary statement of the Practical, Robust Implementation and Sustainability Model (PRISM). (PDF) Agenda for the consensus meeting in our case example. (PDF) List of included and excluded theories, models and/or frameworks (TMFs) in our case example. (PDF) Categorized aims of included and excluded theories, models and/or frameworks (TMFs) in our case example. (XLSX) Appraisal scores of eligible theories, models and/or frameworks (TMFs) in our case example. (XLSX) 29 Apr 2024 PONE-D-24-05799Proposal of the Implementation Theory Selection Model and exemplar application in fall injury prevention: A systematic, consensus-based method to select implementation theories, models and/or frameworksPLOS ONE Dear Dr. Sibley, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Jun 13 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Ivan Sarmiento Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. In the online submission form, you indicated that "The data underlying the results presented in the study may be available from the corresponding author on request." All PLOS journals now require all data underlying the findings described in their manuscript to be freely available to other researchers, either 1. In a public repository, 2. Within the manuscript itself, or 3. Uploaded as supplementary information. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If your data cannot be made publicly available for ethical or legal reasons (e.g., public availability would compromise patient privacy), please explain your reasons on resubmission and your exemption request will be escalated for approval. 3. Please amend either the title on the online submission form (via Edit Submission) or the title in the manuscript so that they are identical. 4. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. 5. Please include a separate caption for each figure in your manuscript. 6. Please include your tables as part of your main manuscript and remove the individual files. Please note that supplementary tables (should remain/ be uploaded) as separate ""supporting information"" files Additional Editor Comments: ============================== I used the major revision option to give you more time to review, but I hope you will find it possible to address the comments in a shorter time.============================== [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ********** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: N/A Reviewer #2: N/A ********** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ********** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Dear co-authors of the manuscript "Proposal of the Implementation Theory Selection Model and exemplar application in fall injury prevention", Thank you for the opportunity to review the above manuscript and for your patience in receiving feedback. I enjoyed reviewing this submission, which I view as a helpful example for how to collaborate with research users in designing a study (we know far too little about this), and in particular selecting guiding frameworks. Below, I share a number of reflections, questions and comments that I think could help to further improve your submission, which I would be happy to review again. My thoughts are listed chronologically, following the manuscript page by page. (1) Page 1, line 19: an "r" is missing in "Schlegel-University" (2) Page 6, line 110: I appreciate that you mention the T-CaSt here and make it clear where your approach adds value. (3) Page 8/9, line 165/ 172: Here you highlight that it is important to find sources that fit the implementation context and/or stem from the same or a similar context as the implementation in focus. While this makes sense intuitively, I still wonder why this is important - a clearer argument here would be desirable. Or, said in a different way, why would a framework developed in education be less usable than one developed in the field of chronic diseases? I think it is worth making this more explicit. Additionally, I wondered how easy (or difficult) it was for you to make this assessment, and which criteria were used to conclude, "okay, this is close enough to our context". This may be hidden somewhere in the supplementary materials - and could be worth a remark or two in the text. (4) Page 8, line 180: A reader may be surprised about the fact that you sat with a review of TMFs used in prevention and management of falls in older adults - and nevertheless looked beyond this review for potentially usable TMFs that could fit your needs. Why did you feel this was worth going through? You describe a lot of complex, laborious legwork, and I think it is important to make it very clear why it was worth it for you. (5) Page 10, line 193: You describe how you could make decisions on which TMFs to in-/exclude based on different criteria. Is your experience that TMFs are sufficiently clear on these criteria such that these decisions are easy to make? This may be worth a remark here but could also be part of the discussion. (6) In this discussion, I am missing reflections on challenges to the process of selecting guiding frameworks in a collaboration between researchers and research users, including, for example, reflections on positionality, power imbalances and other factors that have been highlighted in the literature as the "dark sides" of co-production (see, e.g., Oliver et al., 2019 or Williams et al. 2020; Boaz et al., 2021). I am aware that you describe some - e.g., engagement - challenges but I see only scarce information on how you view your own role in the process described and which challenges such a process potentially presents for research processes. (7) Finally, I wondered if a simple figure of the ITSM could be added to the manuscript itself. I am aware that you have Table 1 in the supplementary materials, but nevertheless felt it could be helpful to have the five steps represented in a simple graphic that could go directly into the manuscript. Reviewer #2: Thank you for the opportunity to review PONE-D-24-05799, “Proposal of the Implementation Theory Selection Model and exemplar application in fall injury prevention: A systematic, consensus-based method to select implementation theories, models and/or frameworks.” I believe that the paper fills a critical gap in the TMF selection space by offering and demonstrating practical steps. The paper is well-written, concise, and instructive. I particularly like the tables and additional files; I imagine that they’ll be very useful to investigators looking to use ITSM. I have just three concerns that I hope the authors can address: First, although the authors’ application of the ITSM was highly rigorous, they do not explain how they developed the model. It is unclear how they arrived at the model’s five steps and sub-steps that articulate how to carry out each step. Related, some suggestions for sub-steps seem overly specific (e.g., in step 1, why only include papers that were cited at least 10 times?), and I was left wondering how the authors arrived at these criteria. A clear description of the methods by which they developed the model would be useful, maybe in the ‘overview of the model’ section on page 8. Second, I was glad that the discussion section acknowledged the large number and amount of resources required (e.g., at least two reviewers screen each TMF) to use the model as proposed, and I liked the suggestions that the authors offered for simplifying the steps on p 21-22 are useful, but I imagine that ITSM will be most useful to solo investigators who may not have the resources to carry out even the simplified approach. The paper would be strengthened with recommendations for an individual without the significant resources that the author had. I don’t think they need to endorse such a bare bones approach, but they could acknowledge the need for one, and propose modifications that would make it feasible for someone with few resources (including a deep bench of collaborators). Third, in the discussion section on page 21, the authors acknowledge that the influence of the model is unclear. This is a point worth fleshing out a bit more. There are a lot of questions – about usability of the model, the value-add beyond traditional approaches to TMF selection, etc. In addition, how the COAST-IS trial would inform the model’s development was not clear. More attention should be paid to what a future research agenda should look like in this respect. ********** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: Yes: Bianca Albers Reviewer #2: Yes: Sarah Abigail Birken ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. 15 Jul 2024 RESPONSE TO REVIEWERS MS. Ref. No.: PONE-D-24-05799 Title: Proposal of the Implementation Theory Selection Model and exemplar application in fall injury prevention Comments from the Editor: 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf Response: We have made the formatting of our manuscript compliant with PLOS ONE’s style requirements. 2. In the online submission form, you indicated that "The data underlying the results presented in the study may be available from the corresponding author on request." All PLOS journals now require all data underlying the findings described in their manuscript to be freely available to other researchers, either 1. In a public repository, 2. Within the manuscript itself, or 3. Uploaded as supplementary information. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If your data cannot be made publicly available for ethical or legal reasons (e.g., public availability would compromise patient privacy), please explain your reasons on resubmission and your exemption request will be escalated for approval. Response: We have uploaded the datasets supporting our findings in the supplementary information files S5 and S6. 3. Please amend either the title on the online submission form (via Edit Submission) or the title in the manuscript so that they are identical. Response: We have amended the title on the online submission form to match the title in the manuscript. 4. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. Response: Research users were not human participants in the research process, but instead were partners on the research team (level of ‘consult’ on the IAP2 Spectrum of Patient and Researcher Engagement in Health Research). Given the collaborative role of research users in the research process and as is common practice in collaborative methods of health research, research ethics board (REB) approval was not sought, and informed written or verbal consent to participate was not obtained for the involvement of research users in this capacity. To clarify, we have added the following statement to Page 6, Lines 132-138 of the Methods section (new text shown in green highlighter; line numbers correspond to the file labeled Revised Manuscript with Track Changes): “Research user partners were not human participants in the research process, but instead were partners on the research team (level of ‘consult’ on the IAP2 Spectrum of Patient and Researcher Engagement in Health Research). Given the collaborative role of research users in the research process and as is common practice in collaborative methods of health research, research ethics board (REB) approval was not sought, and informed written or verbal consent to participate was not obtained for the involvement of research users in this capacity.” 5. Please include a separate caption for each figure in your manuscript. Response: We have included a separate caption for each figure in the manuscript. 6. Please include your tables as part of your main manuscript and remove the individual files. Please note that supplementary tables (should remain/ be uploaded) as separate "supporting information" files. Response: We have included tables as part of the main manuscript and removed the individual files. We have uploaded separate “supporting information” files for supplementary files. Comments from Reviewer #1: Dear co-authors of the manuscript "Proposal of the Implementation Theory Selection Model and exemplar application in fall injury prevention", Thank you for the opportunity to review the above manuscript and for your patience in receiving feedback. I enjoyed reviewing this submission, which I view as a helpful example for how to collaborate with research users in designing a study (we know far too little about this), and in particular selecting guiding frameworks. Below, I share a number of reflections, questions and comments that I think could help to further improve your submission, which I would be happy to review again. My thoughts are listed chronologically, following the manuscript page by page. Response: Thank you for your positive appraisal of our manuscript. We hope the changes made to the manuscript have addressed your reflections, comments and questions. 1. Page 1, line 19: an "r" is missing in "Schlegel-University" Response: We have corrected this mistake. 2. Page 6, line 110: I appreciate that you mention the T-CaSt here and make it clear where your approach adds value. Response: Thank you for your positive comment. No changes were made. 3. Page 8/9, line 165/ 172: Here you highlight that it is important to find sources that fit the implementation context and/or stem from the same or a similar context as the implementation in focus. While this makes sense intuitively, I still wonder why this is important - a clearer argument here would be desirable. Or, said in a different way, why would a framework developed in education be less usable than one developed in the field of chronic diseases? I think it is worth making this more explicit. Additionally, I wondered how easy (or difficult) it was for you to make this assessment, and which criteria were used to conclude, "okay, this is close enough to our context". This may be hidden somewhere in the supplementary materials - and could be worth a remark or two in the text. Response: We included this statement in recognition of the fact that considering TMFs that have been used in similar contexts may serve as a useful starting point for considering. However, we also note the importance of not restricting consideration in this way (which is consistent with the Reviewer’s example above). We have removed this recommendation and now advocate against being too prescription when identifying a pool of potentially relevant TMFs. To address this comment, we removed the sentence on Page 9, Line 169 that read “The sources searched in this step should be adapted to fit the implementation context.” We also modified (new text shown in green highlighter) the following sentences on Pages 9-10, Lines 174-180 to read: Complementary sources may include published reviews and academic articles identifying TMFs used in implementation research and practice, ideally including some conducted with similar target populations, interventions or settings, as these characteristics are relatively easy to assess. In general, we advise against being overly prescriptive when developing a list of candidate TMFs, as the association between characteristics of a TMF and its ultimate relevance will be unclear at this step. 4. Page 8, line 180: A reader may be surprised about the fact that you sat with a review of TMFs used in prevention and management of falls in older adults - and nevertheless looked beyond this review for potentially usable TMFs that could fit your needs. Why did you feel this was worth going through? You describe a lot of complex, laborious legwork, and I think it is important to make it very clear why it was worth it for you. Response: In line with our response to the previous comment, and in acknowledging the Reviewer’s example, we do feel it is important to not restrict initial considerations just to previous applications in a specific area. To justify our approach, we added the following sentence to Page 10, Lines 192-194: “We elected to look beyond this review of implementation research on falls in older adults to compile a comprehensive list of candidate TMFs.” 5. Page 10, line 193: You describe how you could make decisions on which TMFs to in/exclude based on different criteria. Is your experience that TMFs are sufficiently clear on these criteria such that these decisions are easy to make? This may be worth a remark here but could also be part of the discussion. Response: Based on our experience, it is not always easy to make decisions regarding eligibility of TMFs. Hence our recommendation that each TMF be screened by at least two independent reviewers, with discussions resolved through discussion or a third independent reviewer. To emphasize our rationale for using this approach, we have made the following edits (changes shown in green highlighter) to the following text on Page 11 Lines 209-213: “As it can be somewhat challenging to make decisions regarding the eligibility of TMFs, ideally, at least two reviewers should independently screen each TMF, and discrepancies should be resolved through discussion or a third independent reviewer. When a complete dual review process is not feasible, single screening should be conducted only by an experienced reviewer (40).” 6. In this discussion, I am missing reflections on challenges to the process of selecting guiding frameworks in a collaboration between researchers and research users, including, for example, reflections on positionality, power imbalances and other factors that have been highlighted in the literature as the "dark sides" of co-production (see, e.g., Oliver et al., 2019 or Williams et al. 2020; Boaz et al., 2021). I am aware that you describe some - e.g., engagement - challenges but I see only scarce information on how you view your own role in the process described and which challenges such a process potentially presents for research processes. Response: Thank you for your insightful comment and for providing these useful references. We have expanded our discussion to reflect on positionality, power imbalances and our more consultative, extractive approach to engaging patients and members of the public. The discussion section now includes the following paragraph (new text highlighted in green) on Pages 28-29, Lines 511-524: “We encountered challenges engaging knowledge users in the selection of guiding frameworks. We found it difficult to engage the older adult members of our community advisory board in the consensus meeting. Only one older adult member attended, who voted only in the first round of the meeting, and who did not participate in all rounds of discussion. After the meeting, several attendees in the consensus meeting commented that they valued the insight of our older adult partner, and how it swayed their own votes. It would have been even more difficult to engage residents of LTC, even though their insight would be so important. The challenges encountered likely resulted from our use of more consultative and extractive approaches to patient and public engagement, as well as insufficient efforts to address power imbalances between academic researchers, organizational partners and patient partners. For those wishing to engage patients and members of the public more meaningfully in the selection of theory, a more democratic, equitable approach to research ‘co-production’ is needed that prioritizes the unique and vital perspectives of patients over more conventional research methods and understandings of what constitutes ‘academic’ research (55).” 7. Finally, I wondered if a simple figure of the ITSM could be added to the manuscript itself. I am aware that you have Table 1 in the supplementary materials, but nevertheless felt it could be helpful to have the five steps represented in a simple graphic that could go directly into the manuscript. Response: We have added a figure of the ITSM (Fig 1). Comments from Reviewer #2: Thank you for the opportunity to review PONE-D-24-05799, “Proposal of the Implementation Theory Selection Model and exemplar application in fall injury prevention: A systematic, consensus-based method to select implementation theories, models and/or frameworks.” I believe that the paper fills a critical gap in the TMF selection space by offering and demonstrating practical steps. The paper is well-written, concise, and instructive. I particularly like the tables and additional files; I imagine that they’ll be very useful to investigators looking to use ITSM. I have just three concerns that I hope the authors can address: Response: We thank the reviewer for their positive critique of our manuscript. 1. First, although the authors’ application of the ITSM was highly rigorous, they do not explain how they developed the model. It is unclear how they arrived at the model’s five steps and sub-steps that articulate how to carry out each step. Related, some suggestions for sub-steps seem overly specific (e.g., in step 1, why only include papers that were cited at least 10 times?), and I was left wondering how the authors arrived at these criteria. A clear description of the methods by which they developed the model would be useful, maybe in the ‘overview of the model’ section on page 8. Response: We agree with the reviewer that the manuscript was lacking details on how we developed the five steps of the ITSM and the sub-steps we used in our example to operationalize each step of the ITSM. However, we want to clarify that we are not advocating for others to use the same sub-steps as we did in our case example, but instead that each research team should work collaboratively with knowledge users to develop the specific sub-steps they will use to operationalize the five steps of the ITSM. We have added the following text (new text shown in green highlighter) to the section “Overview of the Implementation Theory Selection Model” on Page 7, Lines 147-159 to describe these details: “The ITSM was informed by the JLA method for priority setting (https://www.jla.nihr.ac.uk/jla-guidebook/). The JLA method includes steps for gathering data on potential research questions, evidence checking, interim priority setting, and the final prioritization of research questions through a workshop. AMBK and KMS proposed a first iteration of the ITSM’s five steps, which were revised through consultation with our academic and organizational team members. Once the five steps of the ITSM were finalized, AMBK and KMS proposed the sub-steps to carry out each step of the ITSM in our case example on hip protector implementation, which was revised iteratively through discussions with our research team as the process evolved. In the sections below, we describe and provide examples of how we operationalized each step of the ITSM in our case example on hip protector implementation to illustrate the feasibility and potential value of the ITSM. We encourage ITSM users to work collaboratively with research user partners to develop the specific sub-steps they will use to operationalize the ITSM’s five steps which are feasible given their access to available resources. We have also added the following sentences to Page 10, Lines 188-190 describing our specific rationale for only including TMFs with at least 10 citations from the scoping review on chronic disease management and prevention in Step 1 of our case example: “Our decision to include TMFs with ten or more citations from the scoping review was pragmatic; our goal was to ensure we weren’t missing TMFs that are commonly used in a related field of interest.” 2. Second, I was glad that the discussion section acknowledged the large number and amount of resources required (e.g., at least two reviewers screen each TMF) to use the model as proposed, and I liked the suggestions that the authors offered for simplifying the steps on p 21-22 are useful, but I imagine that ITSM will be most useful to solo investigators who may not have the resources to carry out even the simplified approach. The paper would be strengthened with recommendations for an individual without the significant resources that the author had. I don’t think they need to endorse such a bare bones approach, but they could acknowledge the need for one, and propose modifications that would make it feasible for someone with few r Submitted filename: Response to Reviewers.docx 15 Aug 2024 PONE-D-24-05799R1Proposal of the Implementation Theory Selection Model and exemplar application in fall injury preventionPLOS ONE Dear Dr. Sibley, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. ============================== Please address the comment from reviewer #2 regarding the relevance of the reference to COAST-IS. The reviewer has made a valid point that should be taken into consideration. The authors should provide a reference to support the statement on page 6, line 133, about the need for an ethics review. Is the collaborative nature of the research the reason for not needing a review, or is it the nature of the study and the source of data? This statement should be in line with relevant guidelines for the research context such as the TCP2. “Given the collaborative role of research users in the research process and as is common practice in collaborative methods of health research, research ethics board (REB) approval was not sought, and informed written or verbal consent to participate was not obtained for the involvement of research users in this capacity.” ============================== Please submit your revised manuscript by Sep 29 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Ivan Sarmiento Academic Editor PLOS ONE Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. Additional Editor Comments: Please address the comment from reviewer #2 regarding the relevance of the reference to COAST-IS. The reviewer has made a valid point that should be taken into consideration. The authors should provide a reference to support the statement on page 6, line 133, about the need for an ethics review. Is the collaborative nature of the research the reason for not needing a review, or is it the nature of the study and the source of data? This statement should be in line with relevant guidelines for the research context such as the TCP2. “Given the collaborative role of research users in the research process and as is common practice in collaborative methods of health research, research ethics board (REB) approval was not sought, and informed written or verbal consent to participate was not obtained for the involvement of research users in this capacity.” (p6, l133) [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: (No Response) ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: N/A Reviewer #2: N/A ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Thank you for having addressed al my comments - and for being responsive to suggestions made by another reviewer, which I also found helpful for revising the manuscript. I have nothing further to suggest and look forward to seeing this published. Reviewer #2: Thank you for the thorough responses to my initial review. I remain confused about the reference to COAST-IS. I think that the authors may be suggesting that the cited protocol for a pilot, which tees up a comparison of COAST-IS (an implementation strategy selection and tailoring approach) and usual care makes for a good template for comparison of ITSM (a TMF selection approach) to usual care. There are so many pilots intended to tee up a comparison of a new approach to usual care; I don't understand why COAST-IS, a protocol published more than 4 years ago, for which there are no subsequent publications and therefore has ostensibly been abandoned by the study team, is used to make this point. It feels like a red herring that could be made without reference to any other approach, just by saying that comparing ITSM to standard approaches of TMF selection are warranted. As currently written, I find the reference to COAST-IS to distract from what is a pretty straightforward point. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: Yes: Bianca Albers Reviewer #2: Yes: Sarah A. Birken ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. 20 Aug 2024 Author response: Thank you to the reviewers and editors for reviewing our manuscript. We have responded to the last two outstanding comments below and made relevant changes to the manuscript. We note the reviewers were otherwise satisfied with the revisions. Associate Editor/ Reviewer #2 comment: Please address the comment from reviewer #2 regarding the relevance of the reference to COAST-IS. The reviewer has made a valid point that should be taken into consideration. “Thank you for the thorough responses to my initial review. I remain confused about the reference to COAST-IS. I think that the authors may be suggesting that the cited protocol for a pilot, which tees up a comparison of COAST-IS (an implementation strategy selection and tailoring approach) and usual care makes for a good template for comparison of ITSM (a TMF selection approach) to usual care. There are so many pilots intended to tee up a comparison of a new approach to usual care; I don't understand why COAST-IS, a protocol published more than 4 years ago, for which there are no subsequent publications and therefore has ostensibly been abandoned by the study team, is used to make this point. It feels like a red herring that could be made without reference to any other approach, just by saying that comparing ITSM to standard approaches of TMF selection are warranted. As currently written, I find the reference to COAST-IS to distract from what is a pretty straightforward point.” Response: We have revised the statements in this section which reduce emphasis on the COAST-IS trial. We have made the statements more general and offer the reference to COAST-IS as an example. The pertinent section now refers to lines 460-472 and reads: “Such questions could be answered using methods adapted from ongoing studies (e.g. [50]) to examine the acceptability, usability, appropriateness and perceived impact of the ITSM.” Associate Editor comment: The authors should provide a reference to support the statement on page 6, line 133, about the need for an ethics review. Is the collaborative nature of the research the reason for not needing a review, or is it the nature of the study and the source of data? This statement should be in line with relevant guidelines for the research context such as the TCP2: “Given the collaborative role of research users in the research process and as is common practice in collaborative methods of health research, research ethics board (REB) approval was not sought, and informed written or verbal consent to participate was not obtained for the involvement of research users in this capacity.” Response: We have revised this text and offered a reference in support of the practice of not seeking research ethics approval for individuals involved in research decision making such as theory selection. This section (lines 131-137 now read: “The individuals involved in the theory selection process were not human research participants, but instead were partners on the research team (level of ‘consult’ on the IAP2 Spectrum of Patient and Researcher Engagement in Health Research). Given the collaborative decision making role of these partners in the research process, research ethics board (REB) approval was not sought. This practice was consistent with the JLA guidance that views such decision making work as service evaluation and development (https://www.jla.nihr.ac.uk/jla-guidebook/).” 26 Aug 2024 Proposal of the Implementation Theory Selection Model and exemplar application in fall injury prevention PONE-D-24-05799R2 Dear Dr. Sibley, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Ivan Sarmiento Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: 2 Sep 2024 PONE-D-24-05799R2 PLOS ONE Dear Dr. Sibley, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Ivan Sarmiento Academic Editor PLOS ONE