Efficacy of codesigned COVID-19 booster vaccine promotion materials for long-term care staff: a cluster-randomized trial
Authors: Clarissa Hsu, Annalisa V. Piccorelli, Beverly B. Green, Kimberly C. Arthur, Marla Becker, Breana Berry, Brianna Binion, Alphonse Derus, Margaret Gachuiri, Kelly Hansen, Ahoua Koné, Courtney McCracken, Bennett McDonald, Lauren Nisotel, Kirsten Senturia, Jaclyn Volney, Kanetha B. Wilson, Brian D Williamson
Published: 2026

ABSTRACT
BackgroundAmong health care workers, long-term care center (LTCC) staff are among the most hesitant to obtain COVID-19 vaccines and boosters. Our objective was to compare the effectiveness of enhanced usual care (EUC) to codesigned materials (Codesign) in increasing COVID-19 booster vaccination rates and willingness to promote the booster among long term care center (LTCC) staff.MethodsWe conducted a two-arm cluster-randomized trial in 40 LTCCs in Georgia (n = 21) and Washington (n = 19) from August 2021 to December 2023. EUC LTCCs distributed standard government COVID-19 booster promotion materials. LTCCs in the Codesign arm allowed staff to participate in virtual codesign teams. Over 10 weekly meetings, the teams developed tailored COVID-19 booster promotion materials. Materials were distributed to both arms in January 2023. Primary outcomes were LTCC-level COVID-19 booster vaccination and a Net Promoter Score (NPS) measuring booster promotion to coworkers. Secondary outcomes included staff-level vaccine hesitancy and confidence scores and booster promotion to friends and family.ResultsWe did not detect a significant difference between the EUC and Codesign arms in COVID-19 booster rates or primary NPS measure. Similarly, secondary and exploratory outcomes were not significantly different between either arm.ConclusionsThe null findings of this trial are consistent with the challenges of other clinical trials of COVID-19 vaccine-promotion interventions in improving COVID-19 vaccine uptake among LTCC staff. Lessons for future research include identifying better ways to: (1) engage leaders and staff, (2) design more effective targeting messages, and (3) develop ways to target and customize dissemination.Trial registrationThis study is registered at www.clinicaltrials.gov.Clinicaltrials.gov #NCT05449418. Date of registration: 6/22/2022.Supplementary InformationThe online version contains supplementary material available at https://doi.org/10.1186/s12889-026-28069-7.

Background

When COVID-19 vaccines became available, long-term care center (LTCC) staff had the lowest vaccination rates of frontline health workers, around 50% in April 2021. State and federal mandates increased rates to about 80% for the primary vaccination series in December 2021. However, with no mandates for booster vaccines, LTCC staff uptake lagged at 22% in 2022, declining to 15% by the end of 2023. Effective methods for increasing COVID-19 vaccine uptake and decreasing vaccine hesitancy (VH) among LTCC staff are crucial for protecting residents and staff, particularly with new virus variants. Evidence-based methods will benefit other vaccination efforts, particularly against emerging/novel viruses.

Efforts to increase COVID-19 vaccine uptake among healthcare workers apply educational and communication strategies similar to influenza vaccination [1]. For influenza vaccines, mandated vaccination, declination statements and masks for workers who refuse vaccination and financial incentives are effective [3]. However, these strategies can be perceived as coercive by low-wage workers [2].

Some evidence suggests that interventions that use community engagement for message design and dissemination increase vaccine uptake and decrease VH [3–9]. The National Academy of Sciences recommends community-engaged approaches to promote COVID-19 vaccines, leveraging social networks to tailor and communicate accurate information, improving trust and vaccination [3, 9]. These strategies are important for addressing VH, especially for immigrant/refugee groups and communities of color that experience racism and discrimination in research and healthcare [3, 4, 10–15]. Since disenfranchised communities often do trust personal social networks [16–18], we investigated previously untested community-engagement strategies for increasing uptake and promotion of COVID-19 booster vaccinations among LTCC staff, who have high risk for acquiring and transmitting COVID-19 [19].

ENSPIRE (Engaging Staff to Improve COVID-19 Vaccination Response at Long-Term Care Facilities) compared the effectiveness of enhanced usual care (EUC) COVID-19 booster-promotion materials from government sources such as the Centers for Disease Control and Prevention (CDC) to tailored materials from a virtual codesign process for increasing LTCC staff booster vaccination rates and willingness to promote boosters.

Methods

Trial design and staff recruitment

ENSPIRE was a two-arm cluster-randomized comparative-effectiveness trial. LTCCs (skilled nursing, residential rehabilitation, assisted living, and independent living organizations) were recruited from 10/12/2021 through 5/31/2022 [20], using an online eligibility screening survey [20]. Eligibility criteria were: (1) LTCC located in Georgia (GA) or Washington state (WA); (2) 40–200 employees (changed from 50 to 120 after recruitment challenges); (3) self-reported staff COVID-19 booster rate ≤ 60% as of December 1, 2021.20 The study was guided by the Diffusion of Innovation Theory that focuses on how new ideas are adopted and factors that affect adoption and/or diffusion [21, 22]. This study adheres to CONSORT guidelines and a completed CONSORT checklist is included as a supplementary file.

Randomization

We randomized 40 LTCCs (21 in GA, 19 in WA) to EUC or Codesign (20 per arm), stratified by state. Within states, we constrained randomization to balance four additional baseline characteristics: (1) urban/suburban vs. rural (inside vs. outside the largest metropolitan statistical area designations—Atlanta and Seattle); (2) baseline staff COVID-19 booster vaccination rates; (3) LTCC staff size; and (4) baseline staff survey response rates. For characteristics 2–4, continuous variables were categorized as above vs. below each state’s median [23, 24]. Study biostatisticians generating randomization were blinded to LTCC assignments until primary analyses were complete. The nature of the intervention meant that LTCCs knew their assignment.

Interventions

In the EUC arm, LTCCs distributed standard COVID-19 vaccine-promotion materials from the CDC and Association for Community Living, with 1–3 h of consultation to support dissemination. Codesign arm LTCCs allowed the study to recruit staff to participate in virtual codesign teams. Eligibility for codesign included: (1) currently working at a LTCC randomized to codesign; (2) Net Promotor Score (NPS) score (assessing willingness to promote boosters) of 7 or higher; and (3) no concerns identified from a screening interview about ability to collaborate. Codesign teams developed tailored COVID-19 vaccination-promotion materials to distribute at all LTCCs in the state’s Codesign arm over the course of 10 weekly online meetings (all interaction with codesign participants occurred via Zoom, email and/or text). Codesign participants were trained in nonconfrontational vaccine conversations to promote materials to their peers. Codesign workshops used Boot Camp Translation, Lean-inspired codesign, and user-centered design frameworks for engaging communities and product users, and the Immunity Community intervention for decreasing VH [25–30]. When products were ready for dissemination, all LTCCs (both arms) had a dissemination planning meeting with a study liaison and at least one LTCC leader. Sites chose how to distribute materials using a standardized dissemination checklist. After the meeting, LTCCs assumed responsibility for dissemination, with virtual support (phone calls, video chats) from study liaisons for 2 months. LTCC administrators were offered virtual support on an as-needed basis but no administrators requested this support. After the dissemination planning meeting, study liaisons conducted protocol-based check-ins at three timepoints: when materials were shipped (virtual), 1–3 weeks after shipping (virtual), and 4–8 weeks after receiving materials (in-person).

Data collection

At three time points (Fig. 1), LTCC-level data on staff vaccination rates were collected via brief, online surveys completed by a key contact at the sites, usually the LTCC director, administrator, or an infection control nurse. One LTCC was missing booster rates for all timepoints and was dropped from analyses. A second LTCC reported an incorrect booster rate at 12 months (confirmed by the LTCC), which we treated as missing. We used CMS-reported booster rates for 13 LTCCs that did not respond to initial requests for data at the 8-month timepoint [20]. LTCCs received an impact payment for participation. While follow up was at 8, 10 and 12 months post-baseline data collection, the 8 month follow up timepoint occurred 2 months after the start date of the dissemination period of the materials, which were posted and shared at varying times during the 2-month period at the discretion of LTCC administrators.

Fig. 1a Consort Diagram for Recruitment of LTCCs in GA & WA (10/21 − 6/22). b Consort Diagram for Recruitment of LTCC Staff for Staff Survey in GA and WA (6/22 − 7/22)

a Consort Diagram for Recruitment of LTCCs in GA & WA (10/21 − 6/22). b Consort Diagram for Recruitment of LTCC Staff for Staff Survey in GA and WA (6/22 − 7/22)

Individual-level staff data were collected via online surveys developed for this study (see supplementary file). LTCC staff were recruited via several methods, and all staff were eligible for surveys (in English and Spanish) through a REDCap database. During baseline survey administration, staff opted in to participate in “other study activities,” including additional follow-up email surveys. Staff received a monetary incentive after completing each survey.

Outcomes

Primary outcomes

LTCC level: COVID-19 booster vaccination rate, 10 months after baseline. The rate was based on all eligible staff working at the LTCC at that timepoint. Staff-level: COVID-19 booster NPS collected by staff survey, 8 months after randomization. This was collected earlier than the LTCC-level primary outcome because we expected attitudes to change more quickly than vaccination status. The NPS uses a 10-point Likert scale (1 “least likely,” 10 “most likely”). Our question was: “How likely are you to recommend the COVID-19 booster shot to a person you work with?” We defined a COVID-19 booster promoter as NPS 9 or 10; all others were designated not promoters.

Secondary outcomes

These were LTCC-level COVID-19 booster vaccination rates at 8 and 12 months after randomization and NPS at 12 months. Secondary outcomes from the staff survey at 8 and 12 months were: (1) VH based on the Vaccine Hesitancy Scale [31] modified for adult vaccinations, 10 (least hesitant) to 50 (most hesitant); (2) Vaccine Confidence based on a 3-question tool about confidence in the safety, importance, and effectiveness of the COVID-19 vaccine [32, 33]; and (3) 5 additional versions of the NPS, asking if respondents promoted the COVID-19 primary vaccine series and booster to coworkers, family, and friends.

Sample size

Our sample size calculations ensured at least 90% power to detect a minimum 10% increase for both primary outcomes comparing Codesign to EUC. Varying key assumptions achieved > 90% power for the LTCC-level outcome with 40 LTCCs with average EUC booster rate as high as 60%, number of staff per LTCC as low as 50, and intraclass correlation coefficient as high as 0.05. For staff-level calculations, we assumed 35% baseline survey participation and 10% dropout to achieve > 90% power with 40 LTCCs with average EUC NPS promoter rate as high as 50%.

Statistical methods

Our primary analyses used the intention-to-treat approach. We used generalized estimating equations (GEE) with an independence working correlation matrix for analysis of all outcomes to account for clustering between staff members at a given LTCC and over time. In all cases, we adjusted for constrained randomization variables (state, urban/suburban vs. rural, baseline staff booster vaccination rate, staff size, and survey response rate), outcome baseline value, and time, including an interaction between treatment assignment and time. We used a sandwich standard error estimator with small-sample correction factor of 40/35 [34]. Statistical significance used two-sided p-value < 0.05 and corresponding 95% CIs.

We used GEE linear regression to estimate differences in booster rates at each timepoint for the primary LTCC-level booster vaccination rate outcome, and the secondary outcomes of VH and COVID-19 vaccine confidence. We used GEE logistic regression to estimate differences at each timepoint for primary outcomes of staff-level COVID-19 booster promotion rate and secondary outcomes of booster promotion rate for all other versions of the NPS.

Sensitivity analysis used a per-protocol approach. LTCCs were classified as protocol-adherent if they did not change ownership before dissemination of vaccine promotion materials and if they disseminated the materials. Per-protocol analysis for both primary outcomes used the same methodology as the primary analyses but included an indicator of whether the LTCC was protocol-adherent rather than assignment to the codesign arm.

Other sensitivity analyses assessed the effect of using CMS-imputed booster rate data, the influence of outlying booster rate values or missing booster data on the results, and the effect on the booster rate at Codesign-arm centers with codesigners on staff (Supplementary Materials eMethods).

Results

Participating LTCCs and staff members

We assessed 260 LTCCs for eligibility: 98 did not meet inclusion criteria and 122 refused participation (95 actively, 27 passively) (Fig. 1a). Of the 40 randomized LTCCs, 23 were in urban/suburban locations. Average staff size at baseline was 106, average baseline survey response rate was 24.4% (range 4%-51%), and average center-reported baseline COVID-19 booster rate was 41.3% (range 11%-83%). Binary versions of these variables used in constrained randomization were balanced across study arms.

The final survey sample was 870 staff members, based on inclusion criteria (Fig. 1b; Table 1). A higher proportion of staff survey respondents randomized to the EUC arm lived in Washington (54.7%) and worked at rural LTCCs (53.1%); respondents at LTCCs randomized to the codesign intervention tended to live in Georgia (63.3%) and work in urban locations (58.6%). Distributions of age and gender identity were similar across arms. Proportions of self-reported racial, ethnic, or cultural groups differed across arms: 23.8% of EUC-arm respondents identified as African American or Black, compared to 38.8% in the Codesign arm; 42.9% of EUC-arm respondents identified as White or European descent compared to 32.4% in Codesign. Most other factors were similar across arms at baseline, including COVID-19 booster NPS (average 7.3, 55.9% promoters in EUC vs. 7.5, 58.8% promoters in Codesign) and self-reported booster status (60.7% EUC vs. 67.2% Codesign).

Table 1Baseline Center and Staff level variables overall and by intervention condition: GA & WA CombinedVariableOverall(Nc = 40)EnhancedUsual Care(Nc = 20)Co-design Intervention(Nc = 20)Center-level Variables at Baseline Georgia (vs. Washington)21 (52%)10 (50%)11 (55%) Booster rate41.3 (17.1)38.3 (16.9)44.2 (17.2) Booster rate above median in state*19 (48%)9 (45%)10 (50%) Number of staff members105.8 (36.7)111.9 (41.6)99.8 (31.0) Staff size above median in state*19 (48%)11 (55%)8 (40%) Urban* (vs. Rural)23 (57%)11 (55%)12 (60%) Staff survey response rate24.4 (10.6)27.2 (12.6)21.6 (7.6) Staff survey response above median in state*19 (48%)10 (50%)9 (45%) Reports data to CMS32 (80%)15 (75%)17 (85%)Overall(n = 870)EnhancedUsual Care(n = 508)Co-design Intervention (n = 362)Staff-level Variables at BaselineDemographics Georgia459 (52.8%)230 (45.3%)229 (63.3%) Urban450 (51.7%)238 (46.9%)212 (58.6%) Age (Years) (n = 868) 18—24121 (13.9%)76 (15.0%)45 (12.5%) 25 —34163 (18.8%)101 (19.9%)62 (17.2%) 35 —44196 (22.6%)99 (19.5%)97 (26.9%) 45 —54176 (20.3%)100 (19.7%)76 (21.1%) 55 —64167 (19.2%)105 (20.7%)62 (17.2%) 65+45 (5.2%)26 (5.1%)19 (5.3%) Gender Identity-Femalea705 (81.0%)420 (82.7%)285 (78.7%) Self-Reported Race and Ethnicity (n = 859) American Indian or Alaska Native12 (1.4%)< 10< 10 African (e.g., Nigerian, Kenyan, Ethiopian, Ghanaian)33 (3.8%)18 (3.6%)15 (4.2%) African American or Black258 (30.0%)119 (23.8%)139 (38.8%) Afro-Caribbean (e.g., Jamaican, Haitian, Trinidadian)20 (2.3%)< 1011 (3.1%) Middle Eastern or North African< 10< 10< 10 South Asian< 10< 10< 10 Asian31 (3.6%)23 (4.6%)< 10 Native Hawaiian or Pacific Islander< 10< 10< 10 Hispanic, Latino or LatinX37 (4.3%)23 (4.6%)14 (3.9%) White / European descent331 (38.5%)215 (42.9%)116 (32.4%) Multiracial/multicultural/multiethnic73 (8.5%)43 (8.6%)30 (8.4%) My race, ethnicity, or culture is not listed< 10< 10< 10 Prefer not to answer42 (4.9%)31 (6.2%)11 (3.1%)Employment Factors Employed by LTCCb (n = 856)767 (89.6%)456 (91.2%)311 (87.4%) Rolec (n = 857) CNA or Nursing Aide279 (32.6%)165 (32.9%)114 (32.0%) RN, BSN, LPN166 (19.4%)100 (20.0%)66 (18.5%) Administrative staff70 (8.2%)42 (8.4%)28 (7.9%) Management47 (5.5%)30 (6.0%)17 (4.8%) Food services85 (9.9%)45 (9.0%)40 (11.2%) Facilities management71 (8.3%)32 (6.4%)39 (11.0%) Transportation services12 (1.4%)< 10< 10 Sales and marketing10 (1.2%)< 10< 10 Social worker29 (4.4%)19 (3.8%)10 (2.8%) Life enrichment / recreational staff33 (3.9%)23 (4.6%)10 (2.8%) Other healthcare providers35 (4.1%)21 (4.2%)14 (3.9%) Otherd91 (10.6%)60 (12.0%)31 (8.7%)Duration of Employment at LTCC (n = 853) Less than 1 year308 (36.1%)174 (34.9%)134 (37.9%) 1—3 years231 (27.1%)142 (28.5%)89 (25.1%) 4 —6 years130 (15.2%)83 (16.6%)47 (13.3%) 7 —10 years80 (9.4%)42 (8.4%)38 (10.7%) > 10 years104 (12.2%)58 (11.6%)46 (13.0%)Baseline Staff Outcomes Self-reported COVID-19 Booster Rate546 (63.4%)306 (60.7%)240 (67.2%) COVID-19 Booster Shot to Co-worker NPS7.4 (3.4)7.3 (3.4)7.5 (3.4) COVID-19 Booster Shot Co-worker Net Promoter497 (57.1%)284 (55.9%)213 (58.8%) COVID-19 Vaccine Confident e (n = 868)615 (70.9%)358 (70.8%)257 (71.0%) Vaccine Hesitancy Score (n = 848)36.1 (5.6)36.2 (5.5)36.1 (5.7)For categorical variables, we report n (%). For continuous variables, we report mean (standard deviation)* Denotes variables used in randomizationNc = Number of clustersn = number of survey respondentsa- Remainder of sample consists of respondents that identify as either men, non-binary, gender non-conforming, or preferred not to answerb- Remainder of sample consists of respondents that were employed by an outside agency, through the state, self-employed, other, or did not knowc-Respondents could select multiple optionsd- “Other” includes roles that were not listed or endorsed by fewer than 10 individuals including: physician, physician’s assistant or nurse practitioner, personal care, security guard, and pharmaciste- Respondents were classified as confident or not confident based on the Covid Vaccine Confidence IndexCMS=Centers for Medicare and Medicaid Services, LTCC=long term care communities, CNA=Certified Nursing Assistant, RN=Registered Nurse, BSN=Bachelor’s of Science in Nursing, LPN=Licensed Practical Nurse NPS = Net Promoter Score

Table 1Baseline Center and Staff level variables overall and by intervention condition: GA & WA CombinedVariableOverall(Nc = 40)EnhancedUsual Care(Nc = 20)Co-design Intervention(Nc = 20)Center-level Variables at Baseline Georgia (vs. Washington)21 (52%)10 (50%)11 (55%) Booster rate41.3 (17.1)38.3 (16.9)44.2 (17.2) Booster rate above median in state*19 (48%)9 (45%)10 (50%) Number of staff members105.8 (36.7)111.9 (41.6)99.8 (31.0) Staff size above median in state*19 (48%)11 (55%)8 (40%) Urban* (vs. Rural)23 (57%)11 (55%)12 (60%) Staff survey response rate24.4 (10.6)27.2 (12.6)21.6 (7.6) Staff survey response above median in state*19 (48%)10 (50%)9 (45%) Reports data to CMS32 (80%)15 (75%)17 (85%)Overall(n = 870)EnhancedUsual Care(n = 508)Co-design Intervention (n = 362)Staff-level Variables at BaselineDemographics Georgia459 (52.8%)230 (45.3%)229 (63.3%) Urban450 (51.7%)238 (46.9%)212 (58.6%) Age (Years) (n = 868) 18—24121 (13.9%)76 (15.0%)45 (12.5%) 25 —34163 (18.8%)101 (19.9%)62 (17.2%) 35 —44196 (22.6%)99 (19.5%)97 (26.9%) 45 —54176 (20.3%)100 (19.7%)76 (21.1%) 55 —64167 (19.2%)105 (20.7%)62 (17.2%) 65+45 (5.2%)26 (5.1%)19 (5.3%) Gender Identity-Femalea705 (81.0%)420 (82.7%)285 (78.7%) Self-Reported Race and Ethnicity (n = 859) American Indian or Alaska Native12 (1.4%)< 10< 10 African (e.g., Nigerian, Kenyan, Ethiopian, Ghanaian)33 (3.8%)18 (3.6%)15 (4.2%) African American or Black258 (30.0%)119 (23.8%)139 (38.8%) Afro-Caribbean (e.g., Jamaican, Haitian, Trinidadian)20 (2.3%)< 1011 (3.1%) Middle Eastern or North African< 10< 10< 10 South Asian< 10< 10< 10 Asian31 (3.6%)23 (4.6%)< 10 Native Hawaiian or Pacific Islander< 10< 10< 10 Hispanic, Latino or LatinX37 (4.3%)23 (4.6%)14 (3.9%) White / European descent331 (38.5%)215 (42.9%)116 (32.4%) Multiracial/multicultural/multiethnic73 (8.5%)43 (8.6%)30 (8.4%) My race, ethnicity, or culture is not listed< 10< 10< 10 Prefer not to answer42 (4.9%)31 (6.2%)11 (3.1%)Employment Factors Employed by LTCCb (n = 856)767 (89.6%)456 (91.2%)311 (87.4%) Rolec (n = 857) CNA or Nursing Aide279 (32.6%)165 (32.9%)114 (32.0%) RN, BSN, LPN166 (19.4%)100 (20.0%)66 (18.5%) Administrative staff70 (8.2%)42 (8.4%)28 (7.9%) Management47 (5.5%)30 (6.0%)17 (4.8%) Food services85 (9.9%)45 (9.0%)40 (11.2%) Facilities management71 (8.3%)32 (6.4%)39 (11.0%) Transportation services12 (1.4%)< 10< 10 Sales and marketing10 (1.2%)< 10< 10 Social worker29 (4.4%)19 (3.8%)10 (2.8%) Life enrichment / recreational staff33 (3.9%)23 (4.6%)10 (2.8%) Other healthcare providers35 (4.1%)21 (4.2%)14 (3.9%) Otherd91 (10.6%)60 (12.0%)31 (8.7%)Duration of Employment at LTCC (n = 853) Less than 1 year308 (36.1%)174 (34.9%)134 (37.9%) 1—3 years231 (27.1%)142 (28.5%)89 (25.1%) 4 —6 years130 (15.2%)83 (16.6%)47 (13.3%) 7 —10 years80 (9.4%)42 (8.4%)38 (10.7%) > 10 years104 (12.2%)58 (11.6%)46 (13.0%)Baseline Staff Outcomes Self-reported COVID-19 Booster Rate546 (63.4%)306 (60.7%)240 (67.2%) COVID-19 Booster Shot to Co-worker NPS7.4 (3.4)7.3 (3.4)7.5 (3.4) COVID-19 Booster Shot Co-worker Net Promoter497 (57.1%)284 (55.9%)213 (58.8%) COVID-19 Vaccine Confident e (n = 868)615 (70.9%)358 (70.8%)257 (71.0%) Vaccine Hesitancy Score (n = 848)36.1 (5.6)36.2 (5.5)36.1 (5.7)For categorical variables, we report n (%). For continuous variables, we report mean (standard deviation)* Denotes variables used in randomizationNc = Number of clustersn = number of survey respondentsa- Remainder of sample consists of respondents that identify as either men, non-binary, gender non-conforming, or preferred not to answerb- Remainder of sample consists of respondents that were employed by an outside agency, through the state, self-employed, other, or did not knowc-Respondents could select multiple optionsd- “Other” includes roles that were not listed or endorsed by fewer than 10 individuals including: physician, physician’s assistant or nurse practitioner, personal care, security guard, and pharmaciste- Respondents were classified as confident or not confident based on the Covid Vaccine Confidence IndexCMS=Centers for Medicare and Medicaid Services, LTCC=long term care communities, CNA=Certified Nursing Assistant, RN=Registered Nurse, BSN=Bachelor’s of Science in Nursing, LPN=Licensed Practical Nurse NPS = Net Promoter Score

Baseline Center and Staff level variables overall and by intervention condition: GA & WA Combined

For categorical variables, we report n (%). For continuous variables, we report mean (standard deviation)

* Denotes variables used in randomization

Nc = Number of clusters

n = number of survey respondents

a- Remainder of sample consists of respondents that identify as either men, non-binary, gender non-conforming, or preferred not to answer

b- Remainder of sample consists of respondents that were employed by an outside agency, through the state, self-employed, other, or did not know

c-Respondents could select multiple options

d- “Other” includes roles that were not listed or endorsed by fewer than 10 individuals including: physician, physician’s assistant or nurse practitioner, personal care, security guard, and pharmacist

e- Respondents were classified as confident or not confident based on the Covid Vaccine Confidence Index

CMS=Centers for Medicare and Medicaid Services, LTCC=long term care communities, CNA=Certified Nursing Assistant, RN=Registered Nurse, BSN=Bachelor’s of Science in Nursing, LPN=Licensed Practical Nurse NPS = Net Promoter Score

Primary outcome: LTCC-level COVID-19 booster rate

Adjusted COVID-19 booster rate showed no statistically significant difference between Codesign and EUC arms at any timepoint (Table 2). The estimated adjusted COVID-19 booster rate at 10 months (primary timepoint for this outcome which occurred 4 months after LTCC receipt of the promotion materials) was 46.0% [95% CI = (37.9%, 54.1%)] in the EUC arm and 39.5% (30.4%, 48.6%) in the Codesign arm; adjusted difference in COVID-19 booster rates was − 6.5% (-19.4%, 6.4%). Adjusted differences in booster rates were smaller at 8 months [-1.1% (-15.8%, 13.6%)] and 12 months [-5.2% (-19.8%, 9.4%)]. Many centers had small changes from their baseline booster rate; any large increases in booster rates at one center were balanced by large decreases in booster rates at another center.

Table 2Adjusted Comparisons for Primary Outcomes (Center and Staff-level): GA & WA CombinedTime PointaEnhanced Usual Careb(Nc = 19)Co-Designb(Nc = 20)Adjusted Differenceb(95% CI)P-valueCenter-level COVID-19 booster rate8 Months43.3 (36.1, 50.5)42.2 (30.3, 54.1)-1.1 (-15.8, 13.6)0.89 10 Months 46.0 (37.9, 54.1) 39.5 (30.4, 48.6) -6.5 (-19.4, 6.4) 0.32 12 Months38.6 (29.7, 47.4)33.3 (22.7, 44.0)-5.2 (-19.8, 9.4)0.49Time PointaEnhanced Usual Cared(Nc = 20)Co-Designd(Nc = 20)Time PointaP-valueStaff-level proportion of Booster Net-promoters (to co-worker) 8 Months n EUC = 223n CD =178ICC=0.06 48.1 (41.7, 54.4) 47.9 (36.7, 59.1) -0.2 (-12.9, 12.6) 0.98 12 MonthsnEUC= 255nCD=194ICC=0.0445.4 (37.1, 53.7)46.6 (38.6, 54.7)1.2 (-9.2, 11.6)0.82Note: Bold data indicates it is a primary time pointa- Months post randomization, bolded items are the primary endpointsb- Least-squares Adjusted Mean was obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, and statec- The 95% confidence interval was calculated using a variance inflation factor adjusted standard error to account for the small number of clusters [37]d- Least-squares Adjusted Proportions were obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, state, and center level average baseline net promoter scoreNc = Number of ClustersnEUC = number of survey respondents enhanced usual carenCD = number of survey respondents codesign

Table 2Adjusted Comparisons for Primary Outcomes (Center and Staff-level): GA & WA CombinedTime PointaEnhanced Usual Careb(Nc = 19)Co-Designb(Nc = 20)Adjusted Differenceb(95% CI)P-valueCenter-level COVID-19 booster rate8 Months43.3 (36.1, 50.5)42.2 (30.3, 54.1)-1.1 (-15.8, 13.6)0.89 10 Months 46.0 (37.9, 54.1) 39.5 (30.4, 48.6) -6.5 (-19.4, 6.4) 0.32 12 Months38.6 (29.7, 47.4)33.3 (22.7, 44.0)-5.2 (-19.8, 9.4)0.49Time PointaEnhanced Usual Cared(Nc = 20)Co-Designd(Nc = 20)Time PointaP-valueStaff-level proportion of Booster Net-promoters (to co-worker) 8 Months n EUC = 223n CD =178ICC=0.06 48.1 (41.7, 54.4) 47.9 (36.7, 59.1) -0.2 (-12.9, 12.6) 0.98 12 MonthsnEUC= 255nCD=194ICC=0.0445.4 (37.1, 53.7)46.6 (38.6, 54.7)1.2 (-9.2, 11.6)0.82Note: Bold data indicates it is a primary time pointa- Months post randomization, bolded items are the primary endpointsb- Least-squares Adjusted Mean was obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, and statec- The 95% confidence interval was calculated using a variance inflation factor adjusted standard error to account for the small number of clusters [37]d- Least-squares Adjusted Proportions were obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, state, and center level average baseline net promoter scoreNc = Number of ClustersnEUC = number of survey respondents enhanced usual carenCD = number of survey respondents codesign

Adjusted Comparisons for Primary Outcomes (Center and Staff-level): GA & WA Combined

Note: Bold data indicates it is a primary time point

a- Months post randomization, bolded items are the primary endpoints

b- Least-squares Adjusted Mean was obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, and state

c- The 95% confidence interval was calculated using a variance inflation factor adjusted standard error to account for the small number of clusters [37]

d- Least-squares Adjusted Proportions were obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, state, and center level average baseline net promoter score

Nc = Number of Clusters

nEUC = number of survey respondents enhanced usual care

nCD = number of survey respondents codesign

Primary outcome: Staff-level COVID-19 booster promotion to coworkers

We observed no statistically significant differences at any timepoint in adjusted proportion of NPS scores (measuring COVID-19 booster promotion) for staff members in EUC vs. Codesign arms (Table 2). The adjusted proportion of those defined as promoting boosters to coworkers (NPS score 9 or 10) at 8 months (primary timepoint for this outcome) was 48.1% (41.7%, 54.4%) for EUC and 47.9% (36.7%, 59.1%) for Codesign. The adjusted difference between arms for proportion of COVID-19 booster promoters was − 0.2% (-12.9%, 12.6%). Adjusted difference in booster promoters at 12 months was 1.2% (-9.2%, 11.6%). Estimated ICC was 0.06.

Secondary outcomes: VH, confidence, promotion

No statistically significant difference between EUC and Codesign arms was found for VH or COVID-19 vaccine confidence (Table 3). The adjusted difference in the proportion of people classified as VH was 0.3 (-0.7, 1.2) at 8 months and 0.0 (-0.9, 0.8) at 12 months, comparing staff at Codesign vs. EUC LTCCs. The estimated adjusted difference in proportion of people classified as COVID-19 vaccine confident was 2.4 (-6.8, 11.6) at 8 months and − 0.1 (-8.5, 8.2) at 12 months.

Table 3Secondary Outcomes: GA & WA combinedTime Point aEnhanced Usual Careb(Nc = 19)Co-Designb(Nc = 20)Time Point aP-valueCOVID 19 Booster Confident8 MonthsnEUC=213nCD=172ICC=0.0679.7 (72.1, 87.4)82.1 (73.5, 90.8)2.4 (-6.8, 11.6)0.6112 MonthsnEUC=251nCD=190ICC=0.0382.7 (74.5, 90.9) 82.6 (76.0, 89.2)-0.1 (-8.5, 8.2)0.98Vaccine Hesitancy Scale 8 MonthsnEUC=219nCD=167ICC=0.0836.2(35.4, 37.0)36.5(36.0, 37.0)0.3(-0.7, 1.2)0.5612 MonthsnEUC=250nCD=188ICC=0.0536.0(35.3, 36.6)36.0(35.4, 36.5)0.0(-0.9, 0.8)0.95Staff-level Proportion of Net-PromotersCOVID-19 Booster… to a friend8 MonthsnEUC=222nCD=174ICC=0.0746.5(37.8, 55.2)47.2(36.8, 57.7)0.8 (-11.5, 13.0)0.9012 MonthsnEUC=254nCD=192ICC=0.0443.4 (34.2, 52.5)45.2(36.5, 53.9)1.8(-9.6, 13.2)0.76… to a family member8 MonthsnEUC=223nCD=175ICC=0.0748.5 (39.7, 57.4)47.7(38.8, 60.6)-0.8 (15.1, 13.6)0.9112 MonthsnEUC=254nCD=192ICC=0.0447.6 (39.3, 56.0)51.0 (43.7, 58.4)3.4(-6.9, 13.7)0.52COVID-19 Primary Vaccine Series… to a co-worker8 MonthsnEUC=223nCD=176ICC=0.0748.8 (40.8, 56.7)53.9 (40.9, 66.9)5.1 (-10.0, 20.2)0.5112 MonthsnEUC=254nCD=193ICC=0.0546.7 (36.7, 56.8)48.5 (39.0, 57.9)1.7(-11.0, 14.5)0.79… to a friend8 MonthsnEUC=222nCD=174ICC=0.0647.9 (38.9, 56.9)58.5 (37.4, 59.6)0.6 (-11.6, 12.8)0.92612 MonthsnEUC=254nCD=192ICC=0.0544.9 (34.5, 55.2)44.4(35.5, 53.4)-0.5(-12.0,11.1)0.94… to a family member8 MonthsnEUC=223nCD=175ICC=0.0650.7(40.9, 60.4)49.7(36.0, 63.4)-0.9(-16.4, 14.6)0.9112 MonthsnEUC=254nCD=192ICC=0.0646.7 (36.4, 57.0)48.4(39.2, 57.5)1.7(-9.4, 12.8)0.76a- Months post randomization, bolded items are the primary endpointsb- Least-squares Adjusted Means were obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, and state. Estimates correspond to the proportion classified as “confident” (for the COVID-19 booster confident outcome; ranges from 0 to 100%) or the combined vaccine hesitancy score (vaccine hesitancy scale outcome; ranges from 10 to 50)c- The confidence interval was calculated using a variance inflation factor adjusted standard error to account for the small number of clusters [37]d- Estimates were obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, state, and center level average baseline net-promoter score. Estimates correspond to theNc = Number of ClustersnEUC = number of survey respondents enhanced usual carenCD = number of survey respondents co-design

Table 3Secondary Outcomes: GA & WA combinedTime Point aEnhanced Usual Careb(Nc = 19)Co-Designb(Nc = 20)Time Point aP-valueCOVID 19 Booster Confident8 MonthsnEUC=213nCD=172ICC=0.0679.7 (72.1, 87.4)82.1 (73.5, 90.8)2.4 (-6.8, 11.6)0.6112 MonthsnEUC=251nCD=190ICC=0.0382.7 (74.5, 90.9) 82.6 (76.0, 89.2)-0.1 (-8.5, 8.2)0.98Vaccine Hesitancy Scale 8 MonthsnEUC=219nCD=167ICC=0.0836.2(35.4, 37.0)36.5(36.0, 37.0)0.3(-0.7, 1.2)0.5612 MonthsnEUC=250nCD=188ICC=0.0536.0(35.3, 36.6)36.0(35.4, 36.5)0.0(-0.9, 0.8)0.95Staff-level Proportion of Net-PromotersCOVID-19 Booster… to a friend8 MonthsnEUC=222nCD=174ICC=0.0746.5(37.8, 55.2)47.2(36.8, 57.7)0.8 (-11.5, 13.0)0.9012 MonthsnEUC=254nCD=192ICC=0.0443.4 (34.2, 52.5)45.2(36.5, 53.9)1.8(-9.6, 13.2)0.76… to a family member8 MonthsnEUC=223nCD=175ICC=0.0748.5 (39.7, 57.4)47.7(38.8, 60.6)-0.8 (15.1, 13.6)0.9112 MonthsnEUC=254nCD=192ICC=0.0447.6 (39.3, 56.0)51.0 (43.7, 58.4)3.4(-6.9, 13.7)0.52COVID-19 Primary Vaccine Series… to a co-worker8 MonthsnEUC=223nCD=176ICC=0.0748.8 (40.8, 56.7)53.9 (40.9, 66.9)5.1 (-10.0, 20.2)0.5112 MonthsnEUC=254nCD=193ICC=0.0546.7 (36.7, 56.8)48.5 (39.0, 57.9)1.7(-11.0, 14.5)0.79… to a friend8 MonthsnEUC=222nCD=174ICC=0.0647.9 (38.9, 56.9)58.5 (37.4, 59.6)0.6 (-11.6, 12.8)0.92612 MonthsnEUC=254nCD=192ICC=0.0544.9 (34.5, 55.2)44.4(35.5, 53.4)-0.5(-12.0,11.1)0.94… to a family member8 MonthsnEUC=223nCD=175ICC=0.0650.7(40.9, 60.4)49.7(36.0, 63.4)-0.9(-16.4, 14.6)0.9112 MonthsnEUC=254nCD=192ICC=0.0646.7 (36.4, 57.0)48.4(39.2, 57.5)1.7(-9.4, 12.8)0.76a- Months post randomization, bolded items are the primary endpointsb- Least-squares Adjusted Means were obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, and state. Estimates correspond to the proportion classified as “confident” (for the COVID-19 booster confident outcome; ranges from 0 to 100%) or the combined vaccine hesitancy score (vaccine hesitancy scale outcome; ranges from 10 to 50)c- The confidence interval was calculated using a variance inflation factor adjusted standard error to account for the small number of clusters [37]d- Estimates were obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, state, and center level average baseline net-promoter score. Estimates correspond to theNc = Number of ClustersnEUC = number of survey respondents enhanced usual carenCD = number of survey respondents co-design

Secondary Outcomes: GA & WA combined

a- Months post randomization, bolded items are the primary endpoints

b- Least-squares Adjusted Means were obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, and state. Estimates correspond to the proportion classified as “confident” (for the COVID-19 booster confident outcome; ranges from 0 to 100%) or the combined vaccine hesitancy score (vaccine hesitancy scale outcome; ranges from 10 to 50)

c- The confidence interval was calculated using a variance inflation factor adjusted standard error to account for the small number of clusters [37]

d- Estimates were obtained from generalized estimating equation adjusted for baseline center-level booster rate, number of center staff, proportion of staff completing baseline survey, rural/urban, state, and center level average baseline net-promoter score. Estimates correspond to the

Nc = Number of Clusters

nEUC = number of survey respondents enhanced usual care

nCD = number of survey respondents co-design

For promoting the COVID-19 primary vaccine series and booster to family, friends, and coworkers (Table 3), we saw no statistically significant difference in any promotion outcome between arms. Estimated differences ranged from − 0.9% (-16.4%, 14.6%) for promoting the COVID-19 primary vaccine series to family at 8 months to 5.1% (-10.0%, 20.2%) for promoting the series to a coworker at 8 months.

Per-protocol sensitivity analysis for primary outcomes

Of 20 LTCCs in the Codesign arm, 15 met the per-protocol criteria. Comparing per-protocol intervention LTCCs to remaining LTCCs, estimated adjusted differences between Codesign and EUC in COVID-19 booster rates were 8.5% (-6.2%, 23.1%) at 8 months, 3.9% (-8%, 15.9%) at 10 months, and 6.0% (-7.9%, 19.9%) at 12 months. Per-protocol staff-level analyses estimated adjusted differences in the proportion of booster promoters (to coworkers) were 3.0% (-10.2%, 16.2%) at 8 months and 1.2% (-8.5%, 10.8%) at 12 months. These per-protocol findings showed no significant demonstration of differences in EUC and Codesign effects on LTCC-level COVID-19 booster rates or staff-level proportions of COVID-19 booster promotion (to coworkers).

Sensitivity analyses: Missing data, influential values, co-designer effects

Removing values imputed using CMS data did not change results for LTCC-level COVID-19 booster rates. Of 870 staff members who completed baseline surveys, 498 completed at least one follow-up survey; nonresponse was balanced across arms. Using a GEE-based imputation method for NPS yielded adjusted differences close to zero, with CIs overlapping zero, similar to primary analyses. Three facilities had booster rates identified as influential based on sensitivity analyses; however, the models fit to datasets without each of these sites supported the findings of no statistically significant difference in booster rates between EUC and Codesign. Considering the NPS as continuous and those classified as passives (NPS 6–8) or detractors (NPS ≤ 5) did not yield differences between arms. Having a codesigner on staff at a LTCC in the Codesign arm had no effect.

Discussion

In this cluster-randomized trial with LTCCs, we saw no difference between the EUC and Codesign arms in COVID-19 booster vaccine uptake (booster rate) or booster promotion (NPS). Trends in adjusted COVID-19 booster rate and the proportion of booster promoters decreased slightly in both arms. Secondary and exploratory outcome proportions decreased slightly (e.g., vaccine confidence) or were stable (e.g., VH) over time, with no significant difference between arms.

These findings are consistent with the only other published trial of COVID-19 vaccine-promotion intervention among LTCC staff [35]. Berry et al. randomized 133 skilled nursing facilities to receive usual care or a multicomponent intervention and found no differences between arms for receipt of COVID-19 primary vaccine or booster or for any secondary outcomes. We are unaware of other published trials of interventions to increase COVID-19 vaccination among LTCC staff, highlighting the need for additional research, especially since COVID 19-booster rates continue to be much lower than flu vaccination among LTCC staff [36]. Together with the findings from our study, this continued vaccine hesitancy among LTCC staff reinforces the need for innovative strategies to increase COVID-19 vaccine uptake among LTCC staff, as well as for new vaccines in future pandemics.

Our findings align with the Diffusion of Innovation model [21, 22], which posits that failure of diffusion is often linked to three main factors: (1) pros and cons of the innovation, (2) characteristics of adopters including their reactions and social influence, and (3) the larger social context. For COVID-19 vaccines, many questions were raised about their safety and efficacy. The social and political context was rife with misinformation, and public health messaging about the vaccines changed as scientific knowledge evolved. Adopters may have had limited interest in promoting the vaccine, because it did not always prevent COVID-19 infection [37]. Finally, the LTC context during our study included challenges which were further exacerbated by the pandemic such as staffing shortages, implementation of difficult protocols [38] and mandates, and industry volatility leading to high turnover of both staff and ownership [39, 40]. During our study reporting of vaccination rates was mandatory for all LTCCs receiving federal funds; however, booster rates remained low, necessitating additional interventions to improve these rates. Within this difficult diffusion environment, tailored materials for specific racial groups did not prove to be better than generic materials.

We experienced intervention design and implementation challenges, including difficulty recruiting codesign participants and challenges with the dissemination of the video-based materials. Posters and giveaways were distributed as planned by almost all participating LTCCs and we do not believe dissemination challenges were the result of insufficient support to LTCC administrators. Other manuscripts in progress, including an implementation evaluation [41], explore in more depth possible reasons for why the interventions did not change vaccination rates and describe successes and potential improvements.

Finally, we encountered several methodological challenges. First, recruiting staff for the survey was difficult. Despite multiple recruitment strategies, we fell short of our 35% response goal and our 90% follow-up response rate goal. Further, high staffing turnover industry-wide may have meant that those who completed the follow-up survey may not have been exposed to the intervention given its longitudinal design. We also saw significant variability among LTCCs in staff survey responses rates. Second, we encountered unexpected data quality issues in LTCC booster rates. We thought that mirroring CMS reporting requirements would ensure high quality data and minimize burden. However, rapidly changing CMS data collection parameters and variation in LTCCs’ interpretation of our data request meant that our data did not align with CMS data as closely as expected. We did not discover this issue until post-intervention data collection had begun. We had trouble correcting anomalies with administrators, in part due to administrator turnover. Our booster data had more within-site variation than expected and differences between what was reported to the study versus to CMS.

Our study also had strengths and lessons. Our study benefitted from rigorous biostatistical methods with sensitivity and exploratory analyses, for confidence in null results. Our forthcoming robust evaluation with interviews, focus groups, and intervention fidelity assessment will provide additional insights into vaccine promotion. Other lessons included: (1) engage several administrative leaders (e.g., nursing and human resources directors) instead of only administrators, (2) be cautious when using compliance/quality data in a rapidly evolving situation, and (3) rely more on personal communication and relationships than products such as posters and videos to promote behavior change when the topic is confusing or polarized.

Conclusion

Despite the null findings, this trial surfaced important lessons and areas for future research. Additional work is needed to understand how best to engage LTCCs in research given their importance in disease prevention and mitigation and ensuring health and well-being for a growing number of older adults. Further exploration of policy interventions such as required public reporting of staff vaccination rates, individual vaccine mandates, and new policy innovations, as well as the ways these system level interventions can be combined with individual and organization interventions, may also be useful to explore. Finally, future research should cover when and how tailoring materials is effective, how codesign can be as efficient as possible and still produce impactful messages and interventions, and how best to communicate effectively when knowledge is changing rapidly.

Supplementary Information

Supplementary Material 1.

Supplementary Material 1.

Supplementary Material 2.

Supplementary Material 2.