Leveraging digital health systems maturity assessments to guide strategic priorities Authors: Phiona Vumbugwa, Nancy Puttkammer, Moira Majaha, Andrew Likaka, Sonora Stampfly, Paul Biondich, Jennifer E. Shivers, Kendi Mburu, Olusegun O. Soge, Chris Longenecker, Jan Flowers, Caryl Feldacker Published: 2024 ABSTRACT BackgroundMany low- and middle-income countries (LMICs) face the daunting task of digitising, maturing and deciding where to invest in digital health systems.AimDescribing the facilitators and barriers to conducting digital health maturity assessments and how health leaders can prioritise the assessments.SettingeHealth leaders from 10 African countries, working or supporting Ministries of Health’s digital health and participating in the eHealth Leaders’ Forum from July 2023 to September 2023.MethodsThis qualitative, descriptive study utilised key informant interviews conducted via Zoom with 14 conveniently selected leaders. We used Dedoose Version 9.0 to develop themes based on the health system’s building blocks.ResultsParticipants identified maturity assessments as a critical first step to digital health strengthening, showing the system’s performance and building a baseline response to systematic data quality challenges. Barriers to conducting digital health maturity assessment include lacking collaborators’ buy-in, fragmented vision, overdependence on donor priorities, non-supportive policies and an inadequately skilled workforce. Facilitators include multi-stakeholder engagement, understanding the country’s digital health ecosystem and appropriately integrating maturity assessment objectives. Recommendations include capacity building in data use and conducting maturity assessments at all health system levels to grow the demand and value of digital health strengthening.ConclusionPromoting digital health maturity assessments can help leaders to make appropriate decisions to prioritise areas of improvement and steward maturity advancement as a pathway to strengthening the health system.ContributionWe spotlight the perspectives of African eHealth leaders, centering voices on the barriers, facilitators to planning and recommendations for implementing digital health systems maturity assessments. Introduction Health information systems (HIS) are critical for health systems strengthening (HSS). Global health organisations endeavour to create digitised and integrated HIS with the capacity to collect, collate and analyse vast amounts of information for rapid response to public health needs.1,2 A digitised health system obtains, effectively uses, and securely exchanges information to improve public health practice and population health outcomes.3 Digital health systems provide fast, reliable and efficient ways for governments to track public health interventions but establishing and maintaining them can be costly. The coronavirus disease 2019 (COVID-19) pandemic exposed gaps in how digital health supports information sharing for decision-making.4,5 Many low- and middle-income countries (LMICs) have developed national digital health strategies utilising the World Health Organization (WHO)-International Telecommunications Union (ITU) National eHealth Strategy Toolkit, with strategic objectives for creating a more governed and managed health system.6,7 The WHO-ITU National eHealth Strategy toolkit is a framework and method for developing a national eHealth vision, action plan and monitoring framework.8 Post-COVID-19, several digital systems implemented for early detection, reporting and response across government systems and partners are yet to be evaluated in the light of guidance to integrate the systems.9 The lack of interoperable information systems and limited implementation of automatic data exchange threatens health systems’ functioning and performance. Strengthening a digitised health system requires understanding strengths, gaps and maturity. This points to the need for an appropriate framework to track and assess a system’s core functions and capabilities.10,11 Digital health systems maturity assessments (DHSMA) are often conducted as part of a governance function to learn and provide evidence for strategic investment in HIS. We define maturity as the degree to which a digitised HIS is interoperable, scalable, offers security and privacy, complies with healthcare standard regulations, and makes health information readily available.4,5 In contrast to health systems assessments that may focus across health system building blocks, DHSMA focuses on reviewing an organisation’s processes, capabilities and performance against either internal or external standards, with specific reference to the essential components that make up a well-functioning HIS.12 Digital health systems maturity assessments allow an organisation to benchmark itself over time and lay out a pathway for strategic improvement over time. Assessing digital health systems’ maturity is important to know what has been tried and/or done to scale or where strategic objectives still fall short. Health leaders need proven tools to assess the maturity of their digital health systems. Conducting maturity assessments supports the implementation of the WHO-ITU National eHealth Strategy Toolkit. A few countries in Africa, such as Kenya, Ethiopia, Ghana and Zambia, have conducted maturity assessments; however, these have not been conducted consistently.13,14 Repeating HIS maturity assessments over time allows for longitudinal monitoring of HIS maturity and provides the follow-up to verify that recommendations were implemented.14 There is little published literature on the utility of HIS maturity assessments to health system leaders and planners and the facilitators and barriers to conducting HIS maturity assessments in LMICs are not described. Our study explores (1) health leaders’ perceptions of the value and importance of digital health maturity assessments as part of HIS governance and strategic planning, (2) barriers and facilitators to planning a maturity assessment and (3) health leaders’ recommendations for overcoming barriers to conducting digital health maturity assessments. The study provides information relevant for the institutionalisation of HIS maturity assessments as a critical ingredient for health system governance and holistic HSS as represented in Figure 1. Conceptual framework for digital health systems maturity assessments and the contribution to health systems strengthening. Research methods and design Study design The project used a qualitative descriptive design to assess health leaders’ perceptions of barriers, facilitators and recommended strategies for conducting maturity assessments. Key informant interviews were conducted using a structured key informant guide. Study setting and participants The study was conducted with African health directors from LMICs participating or supporting a global health informatics leaders’ network. In 2023, I-TECH Digital Initiatives Group (DIGI), in partnership with Regenstrief Institute, launched the eHealth Leaders Forum community of practice (eHLF CoP) for national health information leaders in the Ministry of Health (MoH). The eHLF provides peer learning, networking and a place to share best practices. Health leaders discuss digital health implementation, share challenges faced and opportunities for resources or research and offer peer support in digital/HISs assessments, planning and improvement. Through monthly forum meetings, health leaders expressed the need for support in analysing and selecting interventions that strengthen digital health. As champions of digital health in respective countries, the setting provided a platform to interact with skilled and knowledgeable participants on the subject matter. eHealth Leaders Forum is one of several initiatives for digital health capacity building supported by the United States (US) Centers for Disease Control and Prevention’s (CDC) Technical Assistance Platform (TAP). eHealth Leaders Forum is part of the overall TAP capacity development strategies, including digital health training for senior or mid-level leaders and using informatics-savvy health organisation (ISHO) assessment tools at national and sub-national levels.15 All MoH respondents participated in eHLF and some but not all the respondents were exposed to other TAP capacity development interventions. Participant selection, recruitment and eligibility Health leaders were selected using a convenience sample from 10 countries. Participants either had a leadership position in the MoH at the director level (n = 10) or represented partner organisations funding or supporting digital health innovations in the countries (n = 4). The leaders participated in or supported the eHLF. To manage selection bias and generalisability, participants were stratified to ensure representation from different regions of Africa. To enhance the validity of results, participants were asked to share examples to support their responses. Although the representation sample size was small (10 African leaders), the key results were validated during the eHLF meeting where more than 20 other health leaders were present. All participants had at least 2 years of experience in their roles and were expected to be conversant with health informatics systems or the digital health ecosystem. Participants with less than a year of experience in their roles and those who did not respond to a second follow-up email were excluded. An introductory message was developed and initial contact with countries’ MoH digital health leaders was made through email and share the study information. The study team used the email to follow-up and to recruit participants. All participants recruited for the study had been engaged in eHLF for at least 6 months. At least one health leader per organisation and country was purposively selected to participate. Data collection The key informant guide questions were structured and aligned with the six building blocks: governance and leadership, health information, health workforce, financing, medicines and technologies and service delivery16 (see Appendix 1). All participants received an initial email message seeking consent to participate in the study. A discussion format was used in a 30 min virtual interview (via Zoom) scheduled at the participant’s preferred time to solicit and probe more on responses, with participants providing supporting documents where applicable. The interviews were recorded and in cases where the discussion exceeded 30 min, permission to proceed was sought from the participant. Participants answered questions based on their knowledge and shared strategic documents supporting their responses. Interview saturation was reached once participants responded to questions with the same ideas as in earlier responses. Data analysis The interview transcripts were analysed using Dedoose Version 9.0. We developed a codebook and coding linked to the interview questions. Inductive and deductive themes emerged as we analysed the codes for each transcript. Inter-coder reliability was performed with primary and secondary coders by defining the codes, testing coding together, independent coding and discussion after coding. Reliability was tested using Cohen’s kappa formula and coders’ agreement 0.81 of the coding decision. Qualitative thematic analysis was conducted to identify themes relevant to each specific pillar, while content analysis was used to summarise evidence of best practices to support narratives.17,18 Themes for contribution to digital health maturity assessments matched best practices drawn from supporting documents. Recommendations on prioritising and conducting digital health maturity assessment were assigned priority levels from highest to lowest based on the frequency with which the theme was mentioned across respondents. Ethical considerations Ethical clearance to conduct this study was obtained from the University of Washington (UW) internal review board (IRB) Human Subjects Division (HSD) reviewed and approved the research under the UW IRB STUDY00018156. All participants provided formal verbal consent prior to an interview and consent to record the discussion, which was manually transcribed. We confirm that all necessary participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient, participant, or sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. To ensure anonymity, participants were allocated numbers and recordings were started after the demographic data were collected. No study documentation or information was shared via WhatsApp to maintain confidentiality. All personally identifiable information was anonymised during transcription, and WhatsApp was encrypted to ensure data privacy; transcripts were stored on SharePoint in a password-protected computer. Results A total of 14 interviews were conducted; 12 were males, while 2 were females (Table 1). All participants occupied the deputy director level or above. Of the 14 participants, 10 were national level representatives from MoH, universities, and implementing partners (labelled as ‘MoH’) and 4 were representatives from funding and global organisations (labelled as ‘PO’). Among all participants, six had more than 8 years; four had 5 to 7 years; and 4 had 2 years to 4 years of working experience leading digital health in their respective organisations. All five regions of the African continent were represented. We provide the participant number, age, and organization (01/44/MoH) for each participant quote. TABLE 1Participants’ profile.InstitutionTotalNational levelInternational levelMinistry of Health770Universities220Implementing partners110Funding partner202Global organisation202 Total 14 10 4 Participants’ profile. In the following sections, we report findings under the three objectives. Objective 1: Digital health maturity assessment contribution to strengthening health systems Digital health systems maturity assessments were identified as a critical beginning step to strengthening health systems. Several countries had not conducted DHSMA. Health leaders cited maturity assessments contributions including systems performance review, prioritisation of resources, needs and strategies. All participants agreed that there is a huge gap in conducting DHSMAs in their countries as a national approach, with only 3 out of 10 countries attesting to having conducted a digital health maturity assessment in the past 3–5 years. Similarly, one participant stated that strategic planning for DHSMAs has neither been performed nor prioritised at the continent level. Some participants confirmed being involved in implementing DHSMAs as a once-off, research-based or nationwide activity. Respondents cited that most of these initiatives were donor-driven; hence, they lacked follow-up to recommendations and ownership of results and did not prioritise countries’ specific needs. Participants indicated that a DHSMA contributes to knowing the system’s performance and understanding the gaps and strengths to build a baseline to strengthen digital health. Other participants acknowledged that maturity assessments provide a platform to respond to systematic challenges and recommendations on data quality seen through data quality assurance (DQAs) and monitoring and evaluation (M&Es), which most organisations support. Summary of objective 1 Table 2 summarises the contribution of digital health maturity assessments to strengthening HIS and provides examples of corresponding best practices from evidence cited by participants. TABLE 2Summary of digital health maturity assessments contribution to health information systems strengthening.Summary pointQuoteBest practice cited by key informantsIdentifies focus areas that need to be addressed in a digital health system, its strengths and prioritise action.‘If you don’t know the existing systems’ capacity and maturity, how can you make decisions and allocate resources to support this? So, the maturity assessment is critical in providing leaders with the information they need to plan and support efficient and optimal digital health strengthening.’ (Participant 05, 39 years, MOH)Zambia conducted the ISHO maturity assessments to strengthen the performance of the SmartCare digital health system.Provides data for evidence-based decisions, including policies and guidelines that prioritise interventions to strengthen the digital health system.‘After we did the assessment, where we found the most weaknesses is where we focused. But I’m not necessarily saying that the other areas were not focused on. Still, we were able to focus on that area and have a number of initiatives listed in this area of workforce get to the level that we would want to be as a country.’ (Participant 08, 36 years, MOH)Kenya developed an online human resource capacitation platform, training on digital health systems as part of yearly accreditation.Provides better advocacy opportunities for political will and prioritisation of resource allocation towards interventions to strengthen a digital health system.‘… [W]e have that’s what we call ANICiiS, which is the entity, but before that had to do all kinds of assessment in the health sector to see what the gaps are, what needs to be done, which resulted in what we call the digital plan for the country.’ (Participant 12, 42 years, MOH)The Democratic Republic of Congo’s national Digital Health System budget is 80% funded by the government’s post-digital health maturity assessments.Tracks longitudinal evolution and progress of digital health system performance and in providing accurate data‘… [A]fter the previous assessment, we then realized there are a number of issues that we needed to add looking at infrastructure, governance, the establishment of a health informatics and data analytics department, which is one critical area and addresses some issues pertaining to governance.’ (Participant 03, 46 years, MOH)Ethiopia conducted an ISHO assessment in 2020 and followed up with another assessment in 2023 to track the progress of its digital health system roadmap.Digital health maturity assessments provide an opportunity to work on systematic challenges seen in DQAs and M&E, which most programme budget allocations prioritise‘Currently, we are planning another level of assessment because there’s some level of inconsistency with HIV data in Nigeria. So, we didn’t want to determine the exact number of people who are currently on treatment because our spectrum data was showing that Nigeria has just reached maturity level on treatment, yet we are still having new patients on treatment.’ (Participant 01, 44 years, MOH)Nigeria is planning an ISHO assessment to validate the performance of the digital health systems.DQA, data quality assurance; M&E, monitoring and evaluation; ISHO, informatics-savvy health organisation; MoH, Ministry of Health; KIIs, key informants. Summary of digital health maturity assessments contribution to health information systems strengthening. DQA, data quality assurance; M&E, monitoring and evaluation; ISHO, informatics-savvy health organisation; MoH, Ministry of Health; KIIs, key informants. Objective 2: Facilitators and barriers to digital health maturity assessments Governance and leadership Government support and political will were identified as the main facilitators for promoting digital health maturity assessment. Policies and best practices backed by the MoH’s digital health system priorities were considered as effective motivators for implementation. Policies reportedly empower health leaders by creating an enabling environment and allowing resource allocation to build a strong base for assessing digital health: ‘At the national level, a policy could create a national health information exchange between agencies, public and private, and would help to improve the coordination of tracking implementation through digital health maturity assessment.’ (Participant 06, 45 years, PO) Participants cited the lack of collaborators’ buy-in and shared vision as barriers to digital health maturity assessments. Strategic plans are not enough to encourage engagement in digital health assessments without partner support. One respondent stated that: ‘Once we have that five-year strategic plan, we pick out key flagship activities, and we got this digitalization and the strengthening of the health information systems assessments. Once we do that, we engage with the member states ministries to get their buy-in, but you will not probably get 100% of the support, which is like one weakness or challenge because we don’t impose ourselves.’ (Participant 11, 43 years, MoH) Another respondent echoed the lack of support from the government and policymakers and its impact on digital health assessments and strengthening: ‘In some countries, there is a lack of shared vision to invest in digital health. This can make it difficult to get the resources needed to plan maturity assessment and sustain digital health systems strengthening.’ (Participant 02, 45 years, MoH) Health financing The availability of dedicated funding for digital health systems was seen as a conduit to facilitate digital health maturity assessments. Participants acknowledged the advocacy surrounding the importance of digital health and its benefits, with some governments beginning to fund digital health systems interventions and infrastructure. Three participants from national and international levels gave an example of best practice in the Democratic Republic of Congo, where the government finances 80% of digital HISs, with 1.5% of the national health budget invested towards strengthening digital health systems. All participants firmly acknowledged donor support, including the development of district health initiatives and electronic health records (EHR) systems, as important to start conversations around maturity assessment and assessing scalability. One participant stated: ‘The openness by the government to receive financing from donors for the assessments, that’s an opportunity …’ (Participant 01, 44 years, MoH) All participants acknowledged that financing for digital health remains the most significant challenge impacting efforts to prioritise planning of digital health systems assessments and overall system strengthening. It was commonly reported that a specific national HIS allocated budget was lacking. One key informant stated: ‘Looking at advocacy around M&E has been shown partly by having these critical positions in place but is not complimented by a budget to say if we have at least 8%–10% of the ministry budget committed to M&E and informatics, then we know that the oil of the system is guaranteed. So, we find that advocacy is not there in terms of its translation into tangible activities due to poor funding. So, I would advocate that 8%–10% of the ministry budget always support the HIS’s M&E, including assessments, health informatics, and systems strengthening.’ (Participant 09, 52 years, PO) Most participants agreed that over-reliance on donors and funders prevents flexibility in planning for digital health assessments, which are generally not planned as part of the restricted funding. The same is seen to have placed over-dependency on donor priorities with less prioritisation or room to negotiate for financing other critical competing priorities such as maturity assessments. In support, a few participants indicated that their country’s budget to roll out the District Health Information System (DHIS2) is funded by the Global Fund and the President’s Emergency Plan for AIDS Relief (PEPFAR). One respondent stated: ‘In terms of specific budget to support, like DHIS2 hardware and infrastructure, that has been donor funded. Much of the support for DHIS2, if not mistaken, about 80% comes from the Global Fund, 20% is from PEPFAR, and 0% is from the government. Right now, we are rolling out EHR, electronic medical record; much of the support is coming from PEPFAR followed by Global Fund, and assessing their maturity is not part of the grant.’ (Participant 04, 51 years, MoH) Several participants indicated that having outside funders as the leading financial supporters severely impacts the ability of a country to prioritise and promote interventions that support digital health maturity assessments, as donors dictate the priority of the funding. One participant agreed, saying: ‘… [W]hat is not going well is that the financing structure has been too donor oriented, so the priorities have been donor orientated.’ (Participant 12, 42 years, MoH) Workforce Participants noticed the lack of training and skillset in maturity assessments for health workers who are essential health data collectors and users as a barrier to collecting valuable and credible data for prioritising digital health assessment planning. One participant supported saying: ‘… [S]kills and training of health workers in maturity assessment are limited, and that is a barrier, as it means that they may not be able to provide the most up-to-date information to show the performance of the system; there is a delay in getting this data to make decisions, delaying planning on appropriate digital health interventions.’ (Participant 11, 43 years, MoH) Skilled informatics and maturity assessment workforce were cited as vital in successfully implementing digital health assessments, yet countries lack personnel who can lead or conduct digital health assessments. Most health directors cited a lack of recognition for health informatics roles within the health workforce and a lack of power to negotiate priorities to focus on digital health being a barrier to digital health systems assessments. The lack of motivation and poor working conditions, including long hours, low pay and inadequate resources, contributed to governments’ inability to retain key informatics skilled staff. It was commonly observed that countries lack specific training structures for health informatics or digital health personnel, as digitisation of HIS has only recently become part of the health system structure. Two participants from national universities echoed that training has often been ad hoc and developing standardised training programmes is challenging and happening slowly: ‘At the moment, the highest level of training that we have in the country for health information people is a diploma. We don’t yet have a degree that particularly looks at Health Information System, so that’s the first one.’ (Participant 10, 46 years, PO) Respondents believed that improving these conditions could motivate leaders to invest in the digital health workforce, which is key in digital health assessments and overall healthcare improvement. Most participants, 11 of 14, echoed that the current workforce structure facilitates the capacity of available health workers with knowledge of the importance of conducting digital health maturity assessments. Participants cited methods such as including digital health systems mentorship or training in license renewal platforms, departmental mentorships to review reports, data use, and tailored training on digital health for specific needs. One participant stated: ‘… [D]igital health systems training should start from where the health providers like the nurses, pharmaceutical, laboratory technologists are trained at the colleges, universities, and so on, They can be introduced to health information systems, management of health information systems so they can start having a good understanding about information systems, the benefits, uses, challenges so that when they go out there, the workforce can be already empowered.’ (Participant 05, 51 years, MoH) All the above were seen as facilitators to appropriately integrate maturity assessment objectives as the workforce understands the country’s digital health ecosystem and priorities better, thus making planning easy. Across the board, all respondents noted that the workforce’s skill set is essential in ensuring a well-functioning digital health, with the ability to plan and conduct assessments effectively and implement any required changes noted. Infrastructure and medical supplies Most participants believed the diversity of systems, the heterogeneity in investments in information communication technology (ICT) infrastructure, and digital health supplies to be significant barriers for many LMICs to plan for digital health maturity assessments due to cost and availability. Also, the expensive technology has led to having several fragmented digital health systems, either disease-specific or programme-specific and non-interoperable; as such, integrating digital health maturity assessment plans becomes a challenge. Many cited this as why digital health is not prioritised over other health system pillars. The digital health infrastructure should be defined to plan for a maturity assessment, as explained by one participant: ‘I think the infrastructure is a real challenge because we need to ensure that we have connectivity, which is one of the challenges, apart from the equipment like servers, firewalls, and switches that are too expensive to buy and maintain, thus makes digital health systems assessment less priority, we only replace the piece not functional.’ (Participant 08, 36 years, MoH) A few participants brought up an important point that for the few countries that have made steady progress in planning and implementing digital health assessments, the biggest challenge has been having standard and/or user-friendly tools, standardising and having interoperable systems as countries use different electronic medical records (EMR) systems, such as open-source medical records system (OpenMRS), laboratory and management information systems (LAMIS) and others. Many participants stated that each implementing partner would have its own unique system, resulting in a disintegrated digital health landscape, making it challenging to prioritise their maturity assessment planning as systems need to be separated because their maturity and implementation may not be comparable: ‘So, there are multiple systems that are in use and coming, and they are not interoperable. As a result, there is duplication and redundancy in that aspect; we find it hard to pick which one to strengthen.’ (Participant 13, 51 years, PO) Uniquely, one participant pointed out that infrastructure for HISs falls under different departments and is regulated by ministries, like the Ministry of ICT or Finance. This makes it challenging for the MoH to prioritise digital health systems funding and planning digital health maturity assessments as the infrastructure belongs to a different ministry as cited by one participant: ‘Secondly, in many ministries of health, infrastructure is not necessarily the Ministry of Health’s concern, but it is maintained by another ministry, for example the Ministry of ICT, which also get funds from the Ministry of Finance. The fact that the infrastructure is domiciled in another ministry is also a challenge for the health ministry.’ (Participant 02, 45 years, MoH) Stating something almost similar, several respondents cited technology evolving quickly and infrastructure becoming outdated or incompatible faster than strengthening processes can catch up, posing a challenge for keeping digital health assessment planning and implementation up to speed. Summary of objective 2 A summary of barriers and facilitators to planning digital health maturity assessments is provided in Table 3. TABLE 3Summary of barriers and facilitators to planning digital health maturity assessments.HSS pillarBarriersFacilitatorsExamples of digital health strategic documentsLeadership and governanceLack of collaborators’ buy-in, shared vision and supporting policies.Coordinated implementation of digital health policies through a national HIS ministryMinistry of Public Health, DRC. National Development Plan Health Informatics 2020–2024.19FinancingThere is no specific national digital health system budget to support digital health maturity assessments.Allocate a specific national digital/HIS strengthening budgetDemocratic Republic of The Congo Ministry of Public Health Quantified Roadmap of Digital Health Investments.20Information systemsFragmented systems that rely on power and connectivity.Open-Source Systems that are interoperable and standardised building from external collaborationsMinistry of Public Health, DRC. National Development Plan Health Informatics 2020–2024.19WorkforceInadequate technically capacitated staff to properly understand, use, maintain and manage digital health systems infrastructure and technology.Curriculum for digital health systems workforce and health workers orientationKenya’s Virtual Academy online training for health workers.21Zimbabwe and Ethiopia informatics training within the University of Zimbabwe and the University of GondarService deliveryCompeting prioritiesSiloed and fragmented digital health systems divorced from the national HIS structure make choosing or integrating systems to focus maturity assessment difficult.Collaborating with other organisations and agencies, leveraging partnerships to provide resources, expertise and support for digital health strengthening effortsDRC’s One Health digital health systemZambia’s SmartCare SystemMedical suppliesA lack of standard/user-friendly tools costly equipment and infrastructure, including maintaining server rooms, providing reliable connectivity at all levels.Flexible resource allocation systemsIntegration of mobile technology to supplement HISHIS, health information systems; HSS, health systems strengthening. Summary of barriers and facilitators to planning digital health maturity assessments. HIS, health information systems; HSS, health systems strengthening. Objective 3: Ways health leaders can better plan for digital health maturity assessments Participants provided various ways health leaders can better plan digital health maturity assessments in their countries. All participants recommended multi-stakeholder collaborative engagement when planning maturity assessments, from the idea’s conception to completion. Several participants attested that engagement is crucial to ensure that all parties understand and agree on the objectives and methodology of the assessment, leverage existing M&E systems and piggyback on already established systems, such as the DHIS2 monitoring, to effectively plan for a digital health maturity assessment. One participant emphasised that: ‘First of all, the methodology must be understood in a consultative process supported by widespread stakeholder engagement early on in the process including both public and private sector within the health information area. The second part will be after that widespread consultative process, I would want the objectives for the National Health Information System maturity assessment to be outlined and linked to the strategic plan of the digital health system for all implementers’ involvement.’ (Participant 10, 46 years, PO) In support of the aforesaid, several participants agreed that for digital health maturity assessment planning to be effective, there needs to be strong collaborative engagement of all stakeholders to advocate for government prioritisation and political will to support the initiative. Several participants stated that collaborators should be involved in planning a maturity assessment based on their context. Another participant opined that stakeholders hold different powers and expertise, which is key when planning a maturity assessment for the country. Participants outlined the need to bring a sense of recognition, ownership and support to plan a participatory action-based maturity assessment. More than half of the participants supported planning for action by prioritising using context-based assessment tools, minimising duplication of activities, and letting countries decide the processes. In support, one participant echoed that: ‘Stakeholders should not be restricted only to MoH’s HIS and digital health department but include all partners, implementing, funding, Ministry of Information and the regulatory bodies, telecommunications [public and private], power supply organizations, community representatives and advocates. Including such key stakeholders necessitates them to understand the need for prioritizing digital health assessments and their role in setting goals and ownership of recommendations.’ (Participant 14, 41 years, MoH) Almost all (n = 11/14) participants highly prioritised increasing the use of data when planning for digital health maturity assessment. Respondents indicated that the available systems’ data should be used to show the system’s weaknesses or strengths for the leaders to focus the assessment. A few also cited that stakeholders should have access to and the ability to analyse or report the data, which would prepare them to understand assessment findings and take ownership of improving the system. Summary of objective 3 Table 4 summarises recurring recommendations from participants. TABLE 4Summary of recommendations of ways health leaders can better plan for digital health systems maturity assessments.Recommendations when planning a maturity assessmentKey quotePriorityCollaborative engagement through stakeholder mapping‘A multi-stakeholder, a multi-organization, multidisciplinary approach to the assessment, so ensuring that there’s a lot of good participation from a key set of diverse stakeholders …’ (Participant 06, 45 years, PO)HighIdentify the country’s priorities and choose appropriate implementation tools, implementers, time and process.‘And finally, relevance to the local context, so, for the assessment to work well, it should be relevant to that local context so that findings apply to those specific needs of each country or organization. The data collection tool itself should be flexible to meet those needs.’ (Participant 07, 45 years, PO)HighIncrease utilisation of available health data to grow the demand and value of digital health and have its assessment prioritised.‘I think we need to be very specific in the way that we solve the problems by using tools, for instance, these same digital health maturity tools, break them down even to the lowest level and keep ourselves alive all the time, like the way that people report monthly, or weekly, to keep health information systems alive or used through reporting and assessments …’ (Participant 13, 51 years, PO)HighCapacity building at all levels to ensure scalability and continuity of the assessment process‘One is that country leaders need capacity building and more sensitization. Some of these health leaders don’t understand the digital health terminology, and the leaders need to be sensitized to the health information systems used to give more support.’ (Participant 03, 46 years, MoH)HighIdentify DQA and M&E planned and include a digital health assessment to provide a cost-effective approach to conducting assessments that can be implemented at routine intervals.‘… [A]lso, being flexible in the data collection and in the assessment tool itself and applying scalability to already existing plans like M&E and DQA conducted by partners can be cost-effective and promote continuity of digital health maturity assessment, to address different types of systems sizes and complexities and having a cost-effective approach to conducting assessments …’ (Participant 11, 43 years, MoH)MediumAdvocate for increased political will, government ownership, inclusion in strategic plans and investment into digital health maturity assessments‘I feel like the best recommendation I would have is having the political will for and understanding the need for accurate data, then it is easy to mobilize for resources to ensure that maturity assessments are planned and happening because there is some form of accountability.’ (Participant 12, 42 years, MoH)LowDQA, data quality assurance; M&E, monitoring and evaluation. Summary of recommendations of ways health leaders can better plan for digital health systems maturity assessments. DQA, data quality assurance; M&E, monitoring and evaluation. Discussion Health leaders recognised the value of having evidence from DHSMAs to guide them in planning for HIS strengthening. They recognised that DHSMAs provide a critical first step to enhance a system’s performance status, highlighting areas to integrate, expand and scale up towards the long-term goal of a functional, optimised, sustained and strengthened health system. Barriers to implementing maturity assessments include a lack of skilled workforce knowledgeable in digital health maturity assessments, fragmented digital health systems using expensive infrastructure and a lack of financing. Key facilitators to implementing digital health maturity assessment included coordination, collaborating with existing M&E programmes and knowledgeable health workers to conduct digital health assessments at all health facilities. Addressing these barriers and facilitators is crucial for effective digital health strengthening and data-driven decision-making in healthcare systems. Governance is critical to HSS; maturity assessments, especially participatory assessments, can help strengthen that. Political will in LMICs can be a stepping stone to developing digital health policies that strengthen DHSMA to not only rely on international aid. Weak health systems governance in LMICs has resulted in fragmented or ad hoc health policy formulation, poising challenges in implementing DHSMA and impacting efforts to strengthen overall health systems.22,23 Leadership and governance for digital health include having multisectoral digital health steering committees such as an eHealth Technical Working group that oversees the implementation of digital health, interoperability activities and financial resourcing to aid the implementation of recommendations from the assessment.14,24,25 Information derived from maturity assessments can benefit digital health governance in: (1) lobby for political will to finance digital health systems, (2) providing guidance for improvement in health systems’ policies and (3) improving efficiency, effectiveness, performance and productivity in the whole health system.26 Through participatory planning, health systems governance leadership in Ghana and Rwanda effectively prioritised areas to improve digital health, supported by strong political will and governance structures.27,28 The LMICs governments should start growing focus on making the DHSMAs sustainable for a health system to function optimally. Growing knowledge about the value of a digital health maturity assessment is critical to their sustainable implementation. The over-reliance on donor funding has led to digital systems being heavily supported by international organisations in many LMICs, affecting long-term implementation and sustainability of digital health systems, as objectives shift to those of funders. Firstly, making DHSMAs routine and operationalised as part of a strategic vision could increase demand for sustainable digital health assessments. Secondly, efforts should focus on building the culture of conducting regular DHSMAs using integrated and decentralised approaches. Integration supports participatory planning to co-develop the goals/objectives of the assessment based on country needs, priorities and collaborators’ implementation efforts.29 Participatory planning addresses not only technical aspects but also cultural, structural and governance-related factors that contribute to an effective maturity assessment. Thirdly, adapting existing digital health assessment tools such as ISHO and Stages of Continuous Improvement (SOCI) to the country context could aid in routinising DHSMAs without high costs.16,30,31 Fourthly, strengthening the capacity of health leadership in planning and conducting systems performance monitoring at all health facilities promotes accountability to health data, increasing data use. When the health team understands their responsibility, accountability and teamwork are cultivated, essential to improving data use, quality for informed decision-making, policy change and planning.5,25 All these factors could potentially sustain implementation and maturity assessments using available resources without relying much on donor funds. Although there is limited literature on barriers and facilitators to implementing DHSMAs, our findings resonate to some recommendations where maturity assessments have been conducted outside Africa. Provider experiences, usability of data collection tools, and systems interoperability were challenges cited in conducting maturity assessment in Australia, resonating with our findings.32 In the US and Asia, context-based maturity assessments coupled with provider knowledge were key drivers to successful and sustainable assessments for their patient monitoring systems.33,34 Our study’s themes about routinising DHSMAs align with the US Public Health Informatics Institute’s concept of using assessments for advancing US county-level public health departments as ‘informatics-savvy organisations’.12 Countries realise the benefits and value of conducting digital health maturity assessments when governments implement the recommendations into binding policies and HSS activities. Digital health maturity assessments are critical to establishing an evidence base and process for systematically prioritising objectives in the health sector. Digital health systems maturity assessments have the potential to focus resources, de-duplicate work, reduce staff workload and strengthening health systems. Because the digital health landscape and context will evolve over time, assessments should not be conducted as a one-time marker but as part of a routine iterative cycle. Also, longitudinal countrywide maturity assessments should be conducted for understanding digital health progress, updating the strategic vision, strategic objectives and country policies. Health leaders must have a shared vision and skilled champions to plan and/or implement the activities, financing and coordination for sustainable solutions that do not rely on international aid alone.35 This approach can be a pathway to ensuring the DHSMA results will be relevant and useful to all critical partners supporting digital health systems beyond donor-driven investments and projects. Limitations Firstly, most countries had not conducted a maturity assessment at the time of interviews, so participant knowledge was based on M&E or demographic health surveys, which did not focus on digital health systems. Secondly, most (80%) key informants were health leaders from Africa who held positions by appointment and were engaged through eHLF, so their responses may have been biased and not have represented all health system leaders in LMICs. However, we expect the barriers, facilitators and recommendations they named would resonate with other LMIC regions as their responses were based on their area of expertise. Thirdly, the research did not ask about the drawbacks of conducting maturity assessments or why they might not bring value to HIS, which may have introduced bias by conveying an assumption that digital health maturity assessments are important. Lastly, the structure of questions resulted in confounding responses, with some participants treating HIS strengthening and digital health assessments interchangeably. All responses were clarified to ensure the correct meaning in interpretation. Conclusion Strengthening HISs is vital in improving healthcare for all in LMICs. With the growing access to technology and increasing demand for digital health solutions, assessing their maturity to aid in identifying digital health priorities plays a vital role in improving HSS. Countries still face challenges in conducting digital health maturity assessments and operationalising results to strengthen their HIS. The challenges include lack of prioritisation of digital health because of low political will, a lack of shared vision as a result the donor-dependent funding of digital health system and a lack of essential skills in the health workforce to conduct maturity assessments. Addressing these barriers is crucial for planning for and executing digital health maturity assessments, potentially achieving effective HIS strengthening through data-driven decision-making in healthcare systems. Key to planning an effective digital health maturity assessment includes multi-collaborative engagements, contextualising to country needs and priorities, using existing resources and structures or M&E plans, advocating for government prioritisation and gaining political will. Institutionalising digital health maturity assessments as part of digital health governance offers a promise to adopt and build a foundation for having interoperable, integrated and sustainable HIS integral to a well-functioning and strengthened health system. There is a need for further longitudinal country-led DHSMAs, including a systematic study of their impact in improving the planning of digital health strategies and coordinating of investments. Government ownership of DHSMAs and related digital health initiatives will strengthen the policy environment for successful digital information systems in LMICs.