How has co-design been used to address vaccine hesitancy globally? A systematic review Authors: Filipa Alpeza, Helena Avermark, Elisa Gobbo, Sibylle Herzig van Wees Published: 2024 ABSTRACT ABSTRACTImproving vaccine confidence is a topic of major public health importance. Reasons for vaccine hesitancy are multifactorial, making it challenging to find strategies to address them. This systematic review aimed to synthesize the literature on how co-design has been used to reduce vaccine hesitancy. We searched six databases in March and October 2024. Eligible studies described the co-design process used to develop interventions for addressing vaccine hesitancy and increasing vaccine confidence. We assessed the quality of included studies, extracted and descriptively summarized the key data. Twenty-seven articles were included, 20 of which were based in a high-income setting. Most studies centered on the COVID-19 (n = 9) and HPV (n = 9) vaccines. Co-design yielded diverse interventions, with videos being the most common intervention format (n = 11). We observed substantial variations in the reporting style and terminology used within the studies and limited attempts to assess intervention effectiveness.KEYWORDS: Co-design, co-development, vaccine hesitancy, vaccine confidence, vaccination, intervention, systematic review Introduction Vaccines are a major public health success responsible for preventing morbidity and mortality related to various diseases.1 Despite this success, vaccine hesitancy remains a global challenge,2 and has been described as one of the top ten threats to public health.3 Vaccine hesitancy is a complex phenomenon,4 often described as the “delay in acceptance or refusal of vaccination despite availability of vaccination services”.5 However, this definition has been challenged by authors who argue that the decision-making process should be emphasized, proposing that vaccine hesitancy is a psychological state of indecisiveness experienced when making vaccine-related decisions.6,7 Nevertheless, it is widely recognized that vaccine hesitancy is not static or equal to complete refusal of all vaccines,5,6,8,9 but also includes choosing to delay the uptake of a specific vaccine or deciding to vaccinate despite concerns.10,11 In this review, the term ‘vaccine hesitancy’ is therefore used to reference various degrees of apprehension toward vaccination. Another important concept is vaccine confidence, defined as the trust people have in the safety and efficacy of vaccines, as well as the institutions that deliver them.12 A lack of confidence strongly contributes to vaccine hesitancy,13 therefore, changes in vaccine confidence are likely to affect the extent to which individuals are hesitant toward vaccines.14 We acknowledge the interconnectedness of these terms and use them both in the remainder of this paper. The Vaccine Hesitancy Determinants Matrix5 demonstrates the multifactorial nature of vaccine hesitancy, emphasizing the interplay between historical, political, and socio-cultural factors, influences from individual experience and social environment, and specific vaccine-related concerns. As a result, it is often emphasized that there is no “one size fits all” approach to addressing vaccine hesitancy.9,15 Instead, interventions are recommended to be tailored to the specific contextual settings and developed in collaboration with the community.9,16–19 Specifically, interventions and programs ought to be designed utilizing bottom-up strategies, elevating the expertise of the community members.19 Parallel to the increasing academic interest in vaccine hesitancy and confidence,15 a vast body of the literature has emerged describing the experience and process of engaging end-users in the development of health services.20,21 It has been noted that various ‘co-terms’ (e.g., co-design, co-development, co-production) are often used interchangeably to refer to a participatory process.22 Others suggest that co-production is distinct from co-design,23 or that co-design denotes a phase of the co-production process.21 For the purpose of this paper, we will use the term co-design to refer to the meaningful involvement of the end-user in the research process.24 Previous literature has reviewed the use of co-design in various parts of the health system, including clinical care,25–28 health promotion and disease prevention22,29,30 and health education.31,32 Despite the growing interest in vaccine hesitancy, there is a notable gap in the literature reviewing the use of co-design to reduce hesitancy or improve vaccine confidence. Thus, the aim of this article is to review and synthesize the existing literature on the use of co-design for developing interventions to address vaccine hesitancy and increase vaccine confidence. Methods We conducted a systematic literature review based on the following research question: How is co-design being used to address vaccine hesitancy? The review was guided by the Joanna Briggs Institute (JBI) framework for systematic reviews33,34 and reported per the PRISMA 2020 reporting guidelines.35 The checklist is available in Appendix Table A1. The study protocol was registered in PROSPERO in March 2024 (CRD42024520229) Search strategy A literature search was performed on the following databases: MEDLINE, Web of Science Core Collection, PsycINFO, CINAHL, sociological abstracts, and publicly available content database. A complementary search was done on Google Scholar and the first 200 hits were screened. After the first search was performed on March 8, 2024, the search was last updated on October 11, 2024 by rerunning the searches and deduplicating against the previous results using Covidence.36 In Google Scholar, the re-run search was limited to results from 2024 and the first 200 hits were used. The search strategy was developed by MEDLINE (Ovid) in collaboration with librarians at the Karolinska Institutet University Library. For each search concept Medical Subject Headings (MeSH-terms) and free-text terms were identified. The search was tested against already identified key articles as a validation check of the search strategy. The search was then translated, in part using Polyglot Search Translator,37 into the other databases. No language restriction was applied, and databases were searched from inception. The strategies were proof-read by another librarian prior to execution. Some of the key search terms included ‘vaccines,’ ‘immunization,’ ‘trust,’ ‘community participation,’ and ‘co design.’ The full search strategies for all databases are available in Appendix Table A2. De-duplication of the original search was done using the method described by Bramer et al.38 and for the update using Covidence.36 One final, extra step was added to compare DOIs. The database searches were supplemented with a hand search and citation searching of included articles to contribute to a more comprehensive coverage of relevant studies. Study selection All identified records were imported to the software Rayyan,39 where titles and abstracts were screened according to the eligibility criteria. Two authors (FA and HA) independently screened each record. After unblinding, conflicts were resolved through discussion with a third author (EG). These steps were then repeated for a second, full-text screening. Eligibility criteria Inclusion criteria were all empirical articles published in peer-reviewed journals describing the co-design process for developing an intervention to promote vaccine confidence or reduce vaccine hesitancy. Co-design here is defined as meaningful collaboration with the intended end users,24 which meant that collaborations with only experts, policy makers, and individuals working in healthcare did not qualify as co-design. For inclusion, all articles needed to i) tackle vaccine hesitancy specifically, ii) have data collected during intervention development, iii) include active engagement of the end users, and (iv) have end users involved in the intervention development and not merely the implementation. Articles were excluded if i) they only looked at overall vaccine coverage and not vaccine hesitancy specifically, ii) the interventions were developed without active involvement of the end user but using existing or prior knowledge about vaccine-related beliefs and behaviors, iii) they had community-based interventions without evidence of community involvement in the intervention design, and iv) they did not describe the process of co-design. We also excluded protocols, editorials, and articles unavailable in full text. Languages were restricted to articles published in English and Swedish, and the databases were searched from inception. Restricting the languages to those spoken by the authors was done to preserve the integrity of data interpretation and synthesis. The eligibility criteria we used ensured that studies were directly relevant to our research question of how co-design of interventions is being used to address vaccine hesitancy. Data extraction and quality assessment Two researchers (FA and HA) independently conducted the data extraction and quality assessment. For data extraction, the team developed a data extraction table guided by the research question and the JBI framework.34 The table, available in Appendix Table A3, was structured to cover study characteristics (authors, publication year, title, and country), descriptive information about the intervention, the co-design process (co-design methodology, frameworks utilized) and collaborators, and lessons learned and experience with the co-design process (including the barriers and facilitators of the process). While we did not aim to perform a comprehensive evaluation of intervention effectiveness, we noted the intended impact of interventions and cases where preliminary effectiveness testing was conducted. We considered the effectiveness was tested if there was an attempt to evaluate whether the intervention achieved its intended outcomes. The data from each study was then summarized in the extraction table by both extractors. Co-design studies are diverse in terms of methods and reporting style, making it challenging to find a universal quality assessment tool. Therefore, there is a risk of excluding eligible studies due to misalignment with existing quality assessment tools rather than due to true quality concerns.40 We acknowledged this risk and aimed to minimize it by following the approach by Brett et al.40 This approach involves using a generic quality assessment tool and excluding studies only if they failed to report two core components: the aim and the methods. In this review, the quality assessment tools we used were the CASP (Critical Appraisal Skills Programme) randomized controlled trial41 and an adapted version of the CASP Qualitative Study42 checklists, depending on the study design. The templates for both tools are available in Appendix Table A4. If the study passed the first two questions (referred to as ‘the screening questions’), we proceeded with the other questions in the checklist and the study was included in the review. The screening questions were as follows: ‘Was there a clear statement of the aims of the research?’ and ‘Was the methodology appropriate?.’ The articles that passed the quality assessment were given a quality score out of seven or ten, depending on the tool (see Table 2). The quality assessment was performed independently by HA and FA. Table 2.Quality assessment of included studies, per tool. Quality assessment of included studies, per tool. Data synthesis and analysis Guided by our data extraction table, we conducted a narrative synthesis of findings for each extraction element, aiming to map out the major themes. In accordance with the PRISMA guidelines, we summarized the study characteristics using descriptive statistics and summarized the principal components of co-design, overall experiences of the co-design process, including the reported barriers and facilitators, and the key features of final interventions. Two researchers (FA and HA) synthesized the data separately to increase trustworthiness. We then compared our two syntheses to ensure consensus and alignment in interpretation, after which these were combined into a summary synthesis, discussed and agreed upon by all authors. The multi-step approach to synthesis was taken to minimize the risk of interpretive and reporting bias. Throughout the process, all authors met regularly for discussions, feedback, and to reach consensus. Results Twenty-seven articles were included in this review (Figure 1). The initial search yielded 6,700 articles, 3,703 of which remained after de-duplication. Two articles were found through citation searching. The updated search resulted in 670 articles. The most common reason for article exclusion during full-text screening was the absence of co-design , followed by an insufficient description of the co-design process . We excluded six studies for not addressing vaccine hesitancy nor vaccine confidence building. Prisma flowchart. Adapted from page et al.32 Table 1 provides an overview of the included studies, summarizing the study location, vaccine of interest, aim, co-design collaborators, and the type of co-designed intervention. The studies spanned 14 countries, including low-, middle- and high-income countries, representing all WHO regions (Figure 2). The largest number of studies originated from the United States,43–54 while five out of six European studies were conducted in the United Kingdom.55–59 Only one study included transnational collaboration, involving collaborators and researchers from Uganda, Kenya, and Rwanda.60 Included studies were predominantly published in 2023,43,49,50,55,60–66 and only four studies were published before 2020.47,48,51,52 Table 1.Summary of included studies.ArticleLocationVaccineStudy aimCo-design collaboratorsInterventionAya Pastrana et al.66ColombiaCOVID-19To describe the processes used to co-design messages to promote the voluntary uptake of COVID-19 vaccines by inhabitants of Colombia.General population representing the country’s regions.14 evidence-based COVID-19 vaccination audio messages, lasting around one minute each.Blake et al.57UKCOVID-19To rapidly develop and test an internationally relevant, multimedia e-learning package providing education about the COVID-19 vaccine for health and care workers (and trainees), in order to facilitate global promotion and uptake of the COVID-19 vaccines.Nurses and healthcare students.A reusable learning object (RLO) targeting HCW.Bocquier et al.67FranceHPVTo describe the development process of the PrevHPV intervention using the GUIDance for rEporting of intervention Development framework as a guide.Adolescents, their parents, school staff (e.g., teachers, school nurses) and health professionals (GPs and health students).A three-component intervention, including adolescents’ and parents’ education and motivation, GPs’ training and easier access to vaccination.Brewer et al.53USAHPVTo apply the Boot Camp Translations process to develop effective community-responsive messages, materials, and dissemination plans to promote HPV vaccination in two Colorado communities.Adolescents, parents, health care providers, and diverse range of human service and public health employees.Messages suitable for delivery via posters, social media, and educational materials for paediatric primary care settings.Chia-Chen Chen et al.47USAHPVTo co-develop two digital, personal stories about HPV and HPV vaccination with VN mothers who had vaccinated adolescent children, and examined HPV-related knowledge, attitudes, beliefs and vaccination intent among a community group of VN mothers who have unvaccinated adolescents.Vietnamese-American mothers with at least one vaccinated child aged 11–17.Digital stories in Vietnamese and English.Cook et al.70GhanaNot specifiedTo document the co-design process of Cranky Uncle Vaccine digital game in Ghana, describe the challenges that have emerged from its piloting in Ghana and report results from a pilot study testing the game’s effectiveness.Community health workers and young people aged 17–32.An adaptation of a digital game intended to counteract vaccine misinformation.Crawshaw et al.55UKCOVID-19Co‐design a culturally tailored behaviour change intervention with Congolese migrants (non‐UK born) to strengthen their COVID‐19 vaccine uptake.Congolese migrants.A three-component intervention comprising community-led workshops, short plays and posters and flyers.Cunningham-Erves et al.45USACOVID-19Describe the development of a theory-based, culturally appropriate library of motivational messages for a social marketing campaign to promote COVID-19 vaccination among African Americans who are vaccine hesitant.African Americans in the Nashville, Tennessee area.A message library with message subsets presented using 3 modes: content, image, and video.Cunningham-Erves et al.43USAHPVDevelop an individually tailored educational intervention for HPV vaccine–hesitant parents that providers could deliver before clinic visits to increase HPV vaccine confidence and uptake.Community members from diverse racial and ethnic backgrounds who are cancer survivors, caregivers, representatives of cancer-related organizations, and other community members with an interest in cancer prevention and control, as well as an advisory panel comprising 5 hesitant parents participating in interviews.A mobile phone–based intervention (SMS text message) that provides individually tailored education to vaccine-hesitant parents.Dhaliwal et al.64IndiaChildhood vaccinationsLeverage CHWs’ local expertise and strengthen their role in vaccine uptake, ultimately achieving a sustainable community-level intervention to facilitate long-term improvements in vaccine acceptance and uptake.Community health workers and community leaders.Five-component intervention to strengthen vaccine uptake, involving engagement of religious leaders, pictorial pamphlets, videos of respected community members, CHW training and revised visit schedule for vaccine delivery in Mewat.Fisher et al.56UKHPVTo co-produce an educational package tailored at increasing vaccine uptake in areas and populations with lower HPV vaccination coverage.Young people (12–15 y.o.) and professionals involved in delivery of the HPV vaccination programme.The ‘EDUCATE’ package, comprising a training manual and PowerPoint presentation, interspersed with five short films.Fisher et al.59UKHPVTo document the intervention planning and development processes used to co-design the COMMUNICATE films.Ethnically diverse, vaccine hesitant parents.Films intended to support vaccine-hesitant, ethnically diverse parents’ decision-making about the HPV vaccination programme.Lee et al.52USAHPVTo develop a novel, theory-guided narrative intervention modifying variables influencing HPV vaccination behaviours, targeting Cambodian American mothers with adolescent daughters.Four Khmer mothers who have adolescent daughters (both vaccinated and unvaccinated).A DVD containing a collection of narratives to enhance HPV vaccinations.Hopkins et al.60Uganda, Kenya and RwandaNot specifiedDocument the development of the vaccination-related Cranky Uncle version and its co-design process for the East African setting.Key community member groups, including youth aged 16 to 24 years, parents and child caregivers, medical students, and health workers.A new version of a pre-existing mobile-based app (digital game).Kaduru et al.63NigeriaChildhood vaccinationsTo educate caregivers on vaccines and vaccine-preventable diseases and empower them to seek immunisation services as a right by engaging them through a human-cantered design of trust building using community theatre as a tool.Community members, in particular community leaders and women of childbearing age with at least one child between 0–24 months.Theatre performances open to all community members.Katz & Paskett51USAHPVTo develop multilevel interventions to improve HPV vaccination rates among vaccine-eligible adolescents.Caregivers (parents and one grandparent) from Appalachian Ohio and Columbus, Ohio.Multi-level intervention (incl. PowerPoint-presentation, posters for clinics, reminder magnets, newsletters and articles).Kerrigan et al.62AustraliaCOVID-19To unpack the processes employed to create COVID-19 vaccine videos for First Nations communities and to present the evaluation of the videos.First Nations leaders and Elders.16 co-designed, locally relevant videos lasting on average 5 minutes.Lee et al.69Hong KongInfluenzaTo develop and evaluate the effectiveness of a voluntary flu vaccination programme in Hong Kong.Students and people who are involved in on-campus flu vaccination campaigns.A multiphase intervention programme, involving vaccine promotion (posters, SMS, email…) and administration.Long et al.50USAInfluenzaTo develop preventive health videos using a patient-centred approach and test their impact on preventive health uptake.Patients treated for inflammatory bowel disease.Five animated educational videos, lasting 1–2 minutes each.Lum et al.49USACOVID-19To describe HHA’s and administrators’ perspectives related to COVID-19 vaccination messaging, and to co-design a Communication Toolkit to address COVID-19 vaccination misinformation and mistrust among direct care workers.Home health aides working in selected home health and hospice institutions.A Toolkit prototype with messages tailored to the context of home care.Maertens et al.48USAHPVTo gather an in-depth understanding of how to modify the HPV component of the Teen VaxScene intervention to improve its use in general, as well as for the Latino population specifically.Latina parents of female adolescents and young adult Latinas.CHICOS - a web based educational intervention aiming to increase HPV literacy.Osman et al.54USACOVID-19To describe the process for engaging a cross-sector team centring equity and youth voice, outline intervention strategies to enhance COVID-19 vaccine confidence and access, and discuss initial results and lessons learned through the community-engaged process.Youth-serving agencies, public health professionals, healthcare institutions, and youth experiencing homelessness.A four-component intervention comprising youth-friendly messaging about COVID-19 vaccination; health events at youth-serving agencies; vaccine aftercare kits for youth; and training for staff at youth-serving agencies to help them navigate vaccine conversations with youth.Parsons et al.58UKInfluenzaTo co-develop an intervention (in the form of an animation) that could be used as one part of a wider annual flu vaccination campaign to increase flu vaccination uptake locally amongst this at-risk population.Pregnant women (as well as midwives, clinicians and public health-professionals).A video animation conveying facts about flu and the vaccination.Qasim et al.61PakistanCOVID-19To utilise community-led co-design methodology to evaluate misinformation regarding COVID-19 and develop contextual interventions to address misinformation in a marginalised peri urban slum communities of Landhi town Karachi, Pakistan.Different community beneficiaries and influencers (pregnant women, unmarried women, elderly women, female teachers, Aalima (female religious teacher), female healthcare provider, housewife, male teacher, male healthcare provider, male EPI vaccinator, local informal healthcare workers, religious leaders/masjid imams, shopkeepers, street vendors, youth, community leaders, and union council chairmen (local government representative).A single intervention package for dissemination of information involving four different strategies (school-based, mosque-based, healthcare-provider based and youth-based approach).Reñosa et al.65PhilippinesChildhood vaccinationsDrawing on local narratives to design, refine and ultimately test a story-based intervention that connects vaccine hesitant caregivers, policymakers, healthcare workers(HCWs) and other community actors, thus laying the groundwork for a meaningful campaign that restores trust in vaccines.Vaccine hesitant caregivers (e.g. parents, other family members,legal guardians), policymakers, healthcare workers(HCWs) and other community actors.A video including a 5-minute animated cartoon.Saunders et al.44USACOVID-19To develop a curated social media web application for LTCWs with low confidence in COVID-19 vaccines.LTCW partners from diverse backgrounds.Social media web application (interactive website).Streuli et al.46USAChildhood vaccinationsUse CBPR models and community co-design to develop a customised, culturally and linguistically appropriate VR educational programme specifically focused on paediatric vaccinations targeting a vaccine hesitant community.Members of the Somali community of San Diego (either Somali refugee, Somali immigrants, US-born Somali Americans).VR-environment providing culturally and linguistically relevant education on vaccines and health. Summary of included studies. Number of articles per country. The most common vaccines studied were the COVID-19 44,45,49,54,55,57,61,62,66 and the HPV vaccine43,47,48,51–53,56,59,67 (Figure 3). The remaining studies focused on childhood vaccinations46,63,64,68 and the flu vaccine.50,58,69 Two studies did not specify the type of vaccine it was targeting.60,70 Type and frequency of vaccine targeted. The terminology used within the included studies varied substantially. Albeit different terms used; all studies discussed some key elements related to co-design. For clarity purposes, we refer to these elements as co-design collaborators (i.e. those who participated and gave input in intervention design), data collection methods, and description of the final intervention. Groups repeatedly mentioned as co-design collaborators were children and youth,52–54,56,59,61,67 parents, and guardians,47,48,51–53,59,63,65,67 healthcare workers44,48,52,54,56,57,61,64,65,67,70 and community leaders.61–64 Many studies focused on working with underserved populations, including residents of informal settlements,61 youth experiencing homelessness,54 migrants and refugees,46,55 indigenous communities,62 and ethnic minority groups.45,47,48,51,52,54,56,57 The articles did not clearly and consistently report how many collaborators were involved in the co-design process, with numbers ranging from fewer than 1047,50,52 to more than 100.60,63 With regard to data collection, all articles reported using qualitative methods to gather collaborators’ input and feedback. There was a roughly equal split between in-person and remote data collection. The most common methods were interviews,43–45,49,50,52,55,58,59,61–67,69 workshops,46,47,53,55–58,60,61,64,66,69,70 focus group discussions,46,48,50,51,59,61,63,65,67,69 and questionnaires.44,46,54,61,63,64,66,67,69 Nine studies43,45,51,54–56,59,60,62 mentioned offering the collaborators reimbursement for their time and contributions. Among those, the most common form of reimbursement was gift cards or vouchers,43,45,51,54–56,59,62 ranging from US $20 to US $50 in value. In addition to providing gift cards, Osman et al.54 hired two collaborators as advisors to the research team and paid them on hourly bases. Two studies listed the collaborators as coauthors on the published paper.59,62 The included studies utilized a variety of frameworks to guide the co-design process and select the data collection methods. Eight studies46,47,50,51,54,55,62,64 drew on principles from community-based participatory research (CBPR) and participatory action research (PAR), while four studies adopted the design thinking approach.46,49,61,69 Apart from that, the Theory of Reasoned Action,43,45 Health Belief Model,43,45 Behavior Change Wheel55,56,59 and Theory of Planned Behavior65,67 were used. Additionally, two studies used inoculation theory,60,70 which exposes people to weakened forms of misinformation in order to build resilience against it. Prior to the co-design process, researchers tended to familiarize themselves with the study context by performing literature searches or interviewing other stakeholders such as policymakers and healthcare managers.43,54,56,59,60,65,67 In line with respective frameworks, co-design was typically described as a process occurring in several phases, and it was common for different collaborators to be involved at different stages. For example, Qasim et al.61 first conducted key informant interviews with community influencers and later engaged with the community more broadly by organizing co-design workshops. Similarly, Kaduru et al.63 began by establishing contact with community leaders and subsequently collected data from other community members. Regardless of the framework used, co-design collaborators were typically consulted more than once during the co-design process, allowing them to engage with several iterations of the intervention. Noteworthily, the co-design methods varied based on the intervention development phase. Researchers were more involved in conceptualization and problem definition, sometimes guided by surveys or interviews.61,65 Methods such as workshops were often used when the intervention concept was already established, or refinement was needed. The workshops then involved storyboarding, roleplay, drawing, trialing, and giving feedback. For example, Crawshaw et al.55 interviewed Congolese migrants in the UK to gain insights into their vaccination practices, which were the basis for generating potential intervention components. They then selected three components and brought them to workshops for adaptation. A similar approach was used in studies by Hopkins et al.60 and Cook et al.70 whereby collaborators participated in co-design workshops to adapt an existent digital game. These examples show how co-design can be used to build on an existing initiative and make it applicable to a specific setting or issue. In some studies, the collaborators changed, or the number of collaborators grew, nearing the final stages of intervention development, such as evaluation and pre-testing.44,45,57,65,70 A possible advantage of this was getting a different perspective due to individuals being ‘naïve’ to the intervention. Six studies created a Community Advisory Board (i.e., a core group of collaborators), which was meant to oversee the entire co-design process, and the board members remained unchanged throughout.43–46,48,64 The co-design processes described produced a diverse range of interventions, with the majority being either entirely digital or including a digital component.43–54,56–62,64–67,69,70 Formats ranged from videos45,47,50,52,56,58,59,62,64,65 to apps,60,70 text,43,45,49,51–53 or audio54,66 messages, websites,44 and digital toolkits.48,49,56 Videos were the most common , and were either educational or narrative, featuring relatable stories of influential community members.52 The two studies with entirely non-digital interventions developed theater performances63 and community-led workshops and posters.55 It was common for interventions to have multiple components and be referred to as ‘intervention packages.51,54–56,59,61,64,67,69 Even though all studies dealt with vaccine confidence or hesitancy, the intended outcomes varied. For example, for some interventions, success was defined as improvements in vaccination-related knowledge, and attitudes,54,57,59,64,65 while for others it was increase in immunization rates,55,63,69 greater resilience to misinformation60,70 or impressions and engagement with the intervention.62 Evaluation of effectiveness of co-designed interventions was beyond the scope of this review. Nonetheless, we noted that only seven studies reported intervention effectiveness,47,50,57,63,65,69,70 while some mentioned that effectiveness will be tested in future studies.43,44,52,66 The effectiveness was predominately tested through pre–post design47,57,63,69,70 using various outcome measures. For example, some studies linked the co-designed interventions to an increase in vaccination rates,63,69 greater intention to get vaccinated47 and improved skills to promote vaccine confidence.57 A pilot test of the digital game from the study by Cook et al. revealed significant improvements in vaccine attitudes, vaccine importance, and intent to get vaccinated immediately after playing the game.70 However, the strength of these associations is questionable, and it is arguable whether these outcomes are synonymous with ‘effectiveness.’ Two included studies tested their interventions through randomized controlled trials (RCTs). One of the RCTs reported that vaccine hesitant care givers had significantly improved attitudes toward childhood vaccinations after watching a co-designed animation compared with controls.65 The second RCT found no sustained difference in receipt of, or intention to receive, the flu vaccine between the group who watched co-designed animated videos and the control group.50 It is important to note that some effectiveness data might have been shared in standalone publications which did not meet the eligibility criteria for this review. When reflecting on the co-design process, the authors repeatedly mentioned that establishing mutual trust between researchers and co-design collaborators was vital. This was achieved by, for example, delaying data collection and first dedicating some time to observation, building rapport with the target community or engaging with community leaders or other influential community members.61–65 Accommodating collaborators’ needsfor example, when it comes to arranging co-design sessions,54,59 and showing empathy toward collaborators’ concerns before trying to address them was also recommended.45,65 Osman et al.54 advocated for having a tangible proof of accountability toward co-design collaborators, especially if they came from marginalized groups. They created an equity and positionality statement which they revisited throughout the study. Some noticed that co-design had a positive impact on community ties and empowerment, as well as people’s general interest in taking care of their health, thereby providing benefits beyond the scope of the designed intervention.47,54,62,64 Several challenges and limitations of co-design were also noted, the most prominent one being a lack of generalizability of findings beyond the study context.43–45,48,56,60,63,64,66 The generalizability issues stemmed from uncertainty that co-design collaborators were truly representative of the population for whom the intervention was being designed.43 For example, people who fall on more extreme parts of the vaccine hesitancy spectrum and would thus be a target group for an intervention, would also likely be hesitant to engage in a co-design study on this topic.44 Furthermore, those participating in co-design could be subject to social desirability bias or confirmation bias, which might prevent them from sharing their honest feedback.63 In a focus group or workshop environment, there might be a tendency for people to build on and reinforce points made by others, or to change opinions when exposed to other people’s arguments. For example, Blake et al.57 reported that concerns shared by the most hesitant individuals negatively impacted the vaccine-related attitudes of other co-design collaborators. Another potential source of friction in co-design was power imbalances. These might occur between researchers and the co-design collaborators or among the collaborators themselves. Therefore, it was important to ensure that a handful of individuals were not dominating the discussions,57,61 and that the researchers avoided a paternalistic approach55,66 and remained open to feedback.51,58 Another challenge some authors reflected on was gender imbalances during recruitment. For example, researchers in one study said that it was more demanding to engage female stakeholders, as they refused to be audio or video recorded.61 Contrary to that, three studies involved female collaborators only,48,49,63 pointing to a lack of male perspective. Fisher et al.59 reported that, although their collaborators were male and female parents, they did not manage to recruit fathers willing to participate in the intervention itself (i.e., the short films). Most authors did not justify the number of collaborators they recruited. Still, Katz and Paskett51 advised that the sample size should be large enough to capture a range of views but small enough to make reaching a consensus possible. Some authors reflected on the differences between in-person and remote co-design, noting that digital data collection might help reach a larger number of potential collaborators, but that some people might be uncomfortable with or uninterested in digital participation.54,61,66 Moreover, establishing a meaningful connection with collaborators might be more challenging to achieve remotely.66 Regarding cross-regional co-design, Hopkins et al.60 acknowledged language barriers, cultural variations, poor internet connectivity, and differences between urban and rural communities as potential challenges. Finally, several articles commented on the co-design being a time- and resource-intensive process.44,51,54,56,58,59,61 Quality assessment We excluded four studies during the quality assessment phase because they did not pass the two screening questions.71–74 This meant the studies lacked a clear aim and did not outline the methods used, making them incompatible with systematic data extraction and synthesis. Among the studies that passed the quality assessment, the most common reasons for imperfect quality scores were limited information on how authors critically examined their influence on and relation with the collaborators , ethical considerations , and unclear descriptions of the recruitment strategy (Table 2). This constrained our insights into the researchers’ engagement with the co-design collaborators outside the data collection activities. A strength of the studies was that they all provided a rationale for the various co-design methodologies that were used, which positively contributed to the quality scores. Although the methods were well-reported and justified, details on the data collection tools, such as survey instruments or interview schedules, were usually not provided. This makes the validity of those tools questionable, especially if the authors did not use pre-validated measures but developed their own. Discussion The aim of this review was to synthesize studies on the development of co-designed interventions to address vaccine hesitancy and build vaccine confidence. Based on the eligibility criteria, 27 articles were included in this review. Most studies were published after the onset of the COVID-19 pandemic, possibly due to a growing concern around the proliferation of vaccine-related misinformation.75,76 The format of co-designed interventions varied substantially, but most were at least partly digital and had multiple components. The observed breadth of intervention types might be explained by sources of vaccine hesitancy often being vaccine- and context-specific.77,78 The co-design collaborators were often members of underserved or minority populations, which have previously been found to be more prone to vaccine hesitancy.79,80 Co-design might be useful for amplifying the voices of marginalized groups, but researchers should remain mindful of power imbalances.81–84 Regardless of the kind of co-design approach that was used, the most prominent drawback seemed to be a lack of generalizability. Although generalizability arguably contradicts the highly iterative and personalized nature of co-design, there is an underlying intention to transfer the co-designed interventions from the co-design collaborators to, at the very least, the wider population of end-users they are intended to represent. Groger et al.85 and van Velsen et al.86 provide insightful reflections on the role of generalizability in qualitative and human-centered research, questioning whether it is achievable to recruit a representative, unbiased cohort. In our review, there were concerns about the collaborators not embodying the sentiments of the target audience, as reluctance to receive a vaccine might extend to reluctance to participate in vaccine-related research. Therefore, there is a risk of co-designed interventions being meaningful only for those who engaged in their development, resulting in limited external validity. This issue might be particularly prominent for sensitive and controversial public health topics such as vaccine hesitancy, which could have implications for the appropriateness of co-design in this context. Most studies included in this review were published in the last 4 years, indicating that co-design is a relatively new strategy for building vaccine confidence. Therefore, it is unsurprising that many co-designed interventions have not yet been tested and that effectiveness data seem to be pending. However, the increased utilization of co-design in public health necessitates a deeper investigation of its effectiveness.23 Although all included studies endeavored to tackle vaccine hesitancy, the intended impact of interventions and performance indicators varied substantially. This might hinder systematic evaluation of effectiveness of co-design in future research.87,88 Moreover, using vaccine-related attitudes as indicators of effectiveness might be misleading, as they may not translate into vaccine uptake.89,90 In this review, we briefly summarized the effectiveness data if such data were available in the included studies. This was done to illustrate the heterogeneity in effectiveness testing; therefore, the presented findings should not be used to derive any conclusions on the effectiveness of co-design for addressing vaccine hesitancy. Dedicated reviews looking into effectiveness would be valuable, especially as new studies emerge. However, such reviews might benefit from focusing their research question on a specific vaccine type, co-design framework, or effectiveness proxy to make meaningful comparisons. Authors should be wary of the distinction between a successful co-design study and an intervention that successfully targets vaccine hesitancy. For example, a study in which an intervention is created based on collaborators’ input might be deemed ‘successful’ without the intervention itself having a proven effect on vaccine hesitancy. We detected a lack of information on the ethical considerations, recruitment strategies, participation rates and costs of co-designed interventions, which was reflected in our quality assessment scores. Although the authors typically noted that the collaborators consented to being involved in the co-design study, it was unclear how they ensured that consent was informed and how it was recorded. Additionally, there was a lack of discussion around the collaborators’ right to withdraw and how withdrawal would be handled, especially if the final intervention included photographs or videos of those withdrawing. Another common issue was ambiguity regarding the recruitment process and absence of justification for the number of participants involved. The reason for this might be that there is currently no guide on the sample size needed to reach data saturation in co-design studies. Comparable shortcomings were identified by Slattery et al.91 who investigated research co-design in health and found it was commonly used but rarely reported and evaluated in a detailed manner. Moreover, McGill et al.22 who studied co-design in chronic disease prevention, pointed to issues such as unclear terminology, anecdotal accounts of acceptability and feasibility, and scarcity of formal evaluation. Similar drawbacks of co-design studies have been described in systematic reviews by Eyles et al.,92 Green et al.,93 Kiss et al.,25 and King et al.94 Strengths and limitations To the best of our knowledge, this is the first study to systematically describe the use of co-design to address vaccine hesitancy. A major strength of the study is that two authors independently performed screening, quality assessment, extraction, and data synthesis for all articles, thus contributing to the rigor of our findings. However, there are limitations to this work. Some studies might have been overlooked due to using different terminology to denote co-design, being published in databases we did not search or in gray literature, being conducted in languages other than English and Swedish and being reported in a way that is incompatible with our quality assessment framework. Furthermore, there was substantial variation in the amount of detail reported in the studies, which could have biased our synthesis, and the results presented here. The majority of included studies were conducted in high-income countries, which limited our insights into co-design strategies in low- and middle-income settings. Considering the differences in resource availability and societal norms, evidence from low- and middle-income countries would contribute to a more well-rounded understanding of co-design approaches to vaccine hesitancy. Recommendations Although the use of co-design for addressing vaccine hesitancy appears to be prominent, our study points to a lack of standardization when it comes to impact measures, definitions of key terms, and reporting style. Therefore, it might be beneficial to perform a scoping review to identify any consensus in co-design studies as well as areas in which agreement is lacking. Additionally, the field could benefit from more comprehensive and transparent descriptions of co-design processes. Although co-design studies might vary across different settings, we devised several recommendations that are context independent. To begin with, researchers should clearly define the common terms they are using, such as ‘co-design’ and ‘vaccine hesitancy.’ They should also acknowledge the biases that might arise from their relationship with collaborators, and outline how they intend to minimize the risk of bias. If the study involves data collection using measures such as questionnaires, their validity should be considered. More information should be provided on the strategies employed for recruitment of hesitant individuals and for obtaining their informed consent. Moreover, researchers should endeavor to test the effectiveness of co-designed interventions. Further research compiling evidence across co-design studies, such as evidence on effectiveness, is necessary for deriving more generalizable insights about the use of co-design for vaccine hesitancy. Considerations regarding the cost-effectiveness and sustainability of co-design seem to be scarce and should be acknowledged in future studies. Insights into co-design practices for reducing vaccine hesitancy in low- and middle-income countries are currently lacking but are vital for a more comprehensive understanding of this important global health topic. Conclusion Co-design is a prominent strategy to support the development of interventions to address vaccine hesitancy. Considering that co-design is an iterative process guided by user input, considerable heterogeneity among co-design studies is to be expected. Nevertheless, we found that there is an opportunity to standardize the reporting of these studies, particularly when it comes to descriptions of the recruitment strategy, ethical considerations, intended outcomes, and definitions of key terms. Most authors emphasized that building trust with end-users and engaging them in intervention development were worthwhile undertakings. Although co-design might empower research participants and help address the specific concerns of hesitant individuals, and thus make interventions for reducing hesitancy more effective, the data to corroborate this is still emerging. Supplementary Material Supplemental Material Acknowledgments We would like to thank the Karolinska Institute librarians Emma-Lotta Säätelä and AnnaMia Eborn Martinovic for their invaluable contribution during the search process. Biography Filipa Alpeza is a final-year student in MSc Health Economics, Policy and Management at Karolinska Institutet in Sweden. She obtained her bachelor’s degree in health and medical sciences from the University of Warwick, United Kingdom, in 2023. Funding Statement Open access funding provided by Karolinska Institutet. Disclosure statement No potential conflict of interest was reported by the author(s). Credit authorship contribution statement Filipa Alpeza: Conceptualization, methodology, formal analysis, data curation, writing – original draft, writing – review, and editing. Helena Avermark: formal analysis, data curation, writing – original draft, writing – review, and editing. Elisa Gobbo: methodology, writing – review, and editing. Sibylle Herzig van Wees: conceptualization, methodology, writing – review, and editing. All authors approved the final version. Supplementary material Supplemental data for this article can be accessed on the publisher’s website at https://doi.org/10.1080/21645515.2024.2431380 References Associated Data Supplementary Materials Supplemental Material