Development and psychometric validation of digital tool maturity scale for primary healthcare supply chain system in Ethiopia Authors: Tesfahun Hailemariam, Tadesse Awoke Ayele, Araya Abrha Medhanyie, Adane Kebede, Tadesse Alemu Bekele, Sami Tewfik, Alem Desta, Shishay Welay, Tariku Nigatu Published: 2026 ABSTRACT AbstractReliable measurement tools are essential for assessing facility performance and guiding digital tool implementation maturity. Despite Ethiopia’s progress in digital tool implementation and maturity, the lack of a standardized instrument to assess digital supply chain maturity remains a significant challenge. The lower-level pharmaceutical supply system (Dagu) is a locally developed open-source platform in the supply chain system designed to provide systematic record-keeping and manage health commodities in Ethiopia. This study aimed to develop and validate Dagu maturity assessment tool at primary healthcare units in Ethiopia. A cross-sectional study was conducted in all Dagu system implementing facilities in Ethiopia. Item identification, domain generation, and establishment of face and content validity were conducted. Data were collected electronically using the Open Data Kit and analyzed using R version 3.4.6. Factor analysis assumptions such as KMO and Bartlett’s test of Sphericity, were checked, and confirmatory factor analysis was carried out to determine item loading across measurement domains. Model fitness was assessed using Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI), Tucker–Lewis Index (TLI) and Standardized Root Mean Square Residual (SRMR). Item-level explanatory power was evaluated using square multiple correlation (R2). A total of 365 facilities were included for psychometric development and validation. Dagu implementation maturity tool was developed, comprising five domains, twenty-four sub-domains and ninety-four stages of progression with a strong content validity index. The largest factor loading was found in issue data quality, while the lowest was found in stock according to plan. Similarly, the data quality domain demonstrates the highest reliability among the domains identified. The model fit indices were found to be acceptable: RMSEA = 0.079, SRMR = 0.062, TLI = 0.904, and CFI = 0.921. The highest R2 was reported for issue completeness, followed by receive completeness while the lowest R2 was found in stocked according to plan. This study provides the first validated tool to measure supply chain digital tool implementation maturity in Ethiopia’s primary healthcare. The tool reveals strong psychometric performance and highlights the importance of domains, sub-domains and stages of progression identified for supply chain digital tool maturity assessment. This tool can be applied in similar settings to track facilities digital tool maturity progress.Supplementary InformationThe online version contains supplementary material available at 10.1038/s41598-026-57726-5.Keywords: Maturity assessment tool, Digital health, Validation, Development, Confirmatory factor analysis, EthiopiaSubject terms: Health care, Medical research Introduction Effective pharmaceutical supply chain management is vital for ensuring the continuous availability of essential commodities in low-resource health systems1. In 2007, the Ethiopian Pharmaceutical Supply Service (EPSS) was established to streamline the supply of pharmaceuticals2. To digitize the process, Ethiopia implemented the Integrated Pharmaceutical Logistics System (IPLS), providing a unified framework for inventory control and distribution across public health facilities2. While the IPLS significantly improved commodity flow, the increased complexity of forecasting, procurement, and warehousing has made it difficult for the EPSS to maintain an efficient logistics system3. Specifically, the challenges that emerge from manual data management systems often compromise the quality of the health system and act as a bottleneck evidenced by paper-based logistics management information systems (LMIS) lead to poor data quality and delayed evidence-based decision-making4,5. Technology plays a pivotal role in maintaining adherence and enhancing visibility in a complex service-delivery environment, where manual systems limit performance6. To address these challenges, Ethiopia introduced Dagu, an electronic Logistics Management Information System (eLMIS)-a term from Afar culture referring to a traditional system of timely and reliable information sharing-to automate the supply chain process at service delivery points2. The Dagu system was designed to automate the supply chain process at service delivery points and enhance the overall performance of the health system in line with the national digital revolution agenda6,7. While a Dagu implementation progress assessment tool was introduced to monitor Key Performance Areas (KPAs) such as transaction management and data quality, data use for decision making, sustainability, ownership, and product availability, a significant research gap remains and the tool lacks psychometric validations to measure the progression from basic digital tool adoption to full maturity8. Digitalization and scale up of priority health information system is one of the priority areas of Ethiopia information revolution initiatives. However, there is limited evidences for maturity assessment tool for digital health interventions including supply chain management system. The current tool developed and validated is the first of its kind and help to monitor and follow country’s digital health implementation maturity particularly for pharmaceutical supply chain system implementation at primary healthcare units as it enables facilities to track progress through incremental stages toward full maturity. In addition, the tool will help to bridge the gaps between adoption and maturity of digital health solutions in low resource settings. Thus, by reviewing the existing documents in the area and engaging experts through consultative workshops for insights and expertise opinions, this study aimed to develop and conduct the psychometric validation of a digital tool maturity instrument for pharmaceutical supply chain system at primary healthcare units in Ethiopia. Methods Phase 1: Tool development and validation The Digital Maturity Assessment (DMA) tool system was developed and validated in accordance with instrument and psychometric principles to ensure its validity, reliability, and responsiveness9–11. The Dagu system refers to Ethiopia’s national digital pharmaceutical supply chain management platform implemented at the primary healthcare unit level. This tool development and validation study was conducted at the facility level focusing on healthcare facilities as the unit of analysis. The tool helps to measure facilities’ performance on the path of Dagu implementation maturity. The following steps were considered during the tool development and validation process. Item identification and primary domains At this stage, the research team started reviewing existing evidence on the maturity assessment tools12, the Ethiopian digital health blueprint13, the supply chain maturity framework and the national Dagu monitoring tool9,14 to comprehensively understand the existing facts in the business process. Following the item pooling from the review, we organized sessions for detailed discussion and gathered expert opinions on the relevant domains and possible items for the Dagu maturity assessment. Accordingly, the preliminary domains and subdomains were identified through consultations with national and regional Dagu experts, pharmacy professionals, supply chain specialists, and the research team engaged in the development and validation process of Dagu maturity assessment tool9. Experts who have prior knowledge and experience on the system described essential pharmaceutical supply chain maturity components through in-depth interviews, free listening and pile sorting techniques for categorization of possible domains, sub domains and items for Dagu maturity continuum (Fig. 1). Developing and validation of Dagu maturity assessment tool in Ethiopia. Face and content validity A panel of seven experts from multiple levels of the health system were participated in a three-day workshop for face and content validity process. The selection of panellists was based on their experiences in pharmaceutical supply chain management at the national and sub-national levels, to acquire relevant information during validation of the tool [Table 1]. A series of panel discussions was carried out on the possible domains and sub-domains, and experts evaluated the instrument for clarity, grammar, readability, logical flow, alignment with intended constructs, relevance, appropriateness, and suitability for measuring digital supply chain maturity[Annex1]. Iterative rounds of revision took place for improvement of the tool based on the comments given at each stage of the validation process until the revised version met the minimum critical agreement value15. Table 1Composition of the team for face and content validity of Dagu maturity assessment tool.Participants’QualificationProfessionFacilityYears of ExperiencesR1MSCPharmacist and health promotion, supply chain professionHospital14R2BSCPharmacistRegional health bureau13R3BSCPharmacistHospital8R4BSCPharmacistHealth center8R5BSCField IT and area supervisorDHA national team5R6BSC in pharmacy and MPHPharmacist and public health expertHospital15R7BSC in pharmacy and MSC in supply chain managementPharmacist and supply chain professionalHospital9MSc: Master of Science; BSc: Bachelor of Science; MPH: Master in Public Health; DHA: Digital Health Activity; IT: Information Technology. Composition of the team for face and content validity of Dagu maturity assessment tool. MSc: Master of Science; BSc: Bachelor of Science; MPH: Master in Public Health; DHA: Digital Health Activity; IT: Information Technology. Disagreement was resolved through discussion with experts in the process and research team members. Each stage indicates clear implementation maturity, and the item was rated for content validity, and the content validity index (CVI) and Kappa coefficient were estimated. Items with CVI ≥ 0.78 and overall scale CVI ≥ 0.90 were considered acceptable indices16. Study design, sample size and sampling procedures A cross-sectional study design was conducted from April 03 to 27, 2024, targeting all facilities implementing the Dagu system across regions in Ethiopia. Sample size for the psychometric validation was estimated using a subjects-to-item ratio of 12:1 and a 10% non-response rate from facilities that implemented Dagu was taken into consideration17. Thus, 317 facilities were expected according to the formula given. However, the initial survey sample was 365 facilities implementing the Dagu system in the country, which was above the initially estimated sample size, and thus we considered the larger sample size for factor analysis. Thus, all facilities that implemented the Dagu system in all regions of Ethiopia were included in the study. To track stages of progression, we developed a 4-point growth marker scale ranging from 1 to 4. A score of 1 indicates minimal implementation, where facilities have only initiated the required items. A score of 4 represents full implementation and optimization where facilities have fully adopted all required components in alignment with the established business processes. At this final stage, facilities are expected to achieve optimal performance in implementing the defined standards [Figure 2]. Dagu maturity assessment conceptual framework; TP: Transaction Practice, DQ:Data Quality, DU: Data use and visibility, LMG: Leadership and management, Eff:Efficiency, DM: Dagu Maturity. Data collection tool, procedures, management and statistical analysis Initially, the tool was developed in English version and utilized to validate both face and content validity. Dagu tool users in the implementation facilities were healthcare professionals and have advanced language skills. Thus, we used the English version for validation process at each stage and have no locally translated version of the tool. The tool was developed and pre-tested in two selected facilities. After a pre-test, again experts were gathered for discussion on the pre-test findings. Each domain, sub domains and stages of progression were verified and the tool was refined based on the feedback from the panellists before the data collection was commenced. Three days training was given to data collectors and supervisors on data collection tool regarding on the objective of the study, data collection tools, procedures of the data collection, and handling of the data. Daily supervision was conducted by the supervisory team to ensure quality of the data before synchronizing the data to central server. The data synced to central server during data collection were managed by research team on daily basis. Data were collected using Open Data Kit (ODK) and exported to R Studio 3.4.6 version for analysis. Operational definition of the tool items included in the study found (Appendix 3). Psychometric evaluation Confirmatory factor analysis (CFA) was employed to test the loading of items. The CFA assumption was tested, and the model fitness indices were reported accordingly. The reliability test was applied to domain-level items within each domain, and a statistical procedure such as CFA was used to test item loadings under each construct18, and sampling adequacy was checked18. Accordingly, Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy with the cut-off point ≥ 0.5019 and Bartlett’s test of Sphericity were conducted to assess correlation among items before conducting CFA20. The correlation matrix showed that inter-item correlations ranged from 0.080 at artificial stock out and stock status utilization to 0.910 at issue completeness and issue accuracy. The correlation matrix showed that the majority of the items have inter-item correlations above the cut-off points, i.e., 0.3 and were suitable for factor analysis. The KMO measure of sampling adequacy was 0.93, which revealed that the sample size was adequate for factor analysis. Bartlett’s test of Sphericity which tests the null hypothesis, indicated that the correlation matrix was statistically significant at χ2 = 5,410.59, p = .000. The overall model fitness was measured and assessed using goodness of fit indices as indicated [Table 2], showing Chi-square ratio (< 3), goodness of fit index (GFI > 0.9), comparative fit index (CFI > 0.9), Tucker-Lewis’s index (TLI > 0.9), and root mean square error of approximation (RMSEA < 0.08) and standardized root mean squared residual (SRMR)21. The square multiple correlation (R2) was used to report the proportion of variance that the model could be explained by the specific construct. Table 2Confirmatory factor analysis model fitness indices for Dagu maturity assessment tool in Ethiopia.Fit indicesThreshold valueAuthorsResults obtainedConclusionTucker-Lewis’s Index (TLI)> 0.9Chau (1997)0.904AcceptedComparative Fit Index (CFI)> 0.9Bentler (1990)0.921AcceptedRoot Mean Square Error of Approximation (RMSEA)< 0.08Ryo SASAKI (2013)0.079AcceptedStandardized Root Mean Square Residual (SRMR)< 0.08Bentler andBonett (1980)0.062Accepted Confirmatory factor analysis model fitness indices for Dagu maturity assessment tool in Ethiopia. Bentler and Bonett (1980) Reliability and validity Internal consistency of the overall tool and the sub-domains was checked using Cronbach’s alpha coefficient where Cronbach’s alpha coefficient ≥ 0.70 will be accepted as evidence of good internal consistency22. Thus, reliability coefficients were computed to determine the internal consistency of the constructs [Table 3]. Table 3Reliability of the factors on Dagu maturity assessment tool validation.ConstructsSample sizeNumber of itemsCronbach alphaTP36530.96DQ36590.95DU36540.86LMG36550.83EFF36530.59TP: transaction practice; DQ: Data quality; DU: Data use; LMG: Leadership and governance; EFF: Efficiency. Reliability of the factors on Dagu maturity assessment tool validation. TP: transaction practice; DQ: Data quality; DU: Data use; LMG: Leadership and governance; EFF: Efficiency. Ethical consideration Written consent was obtained from the regional health bureau research institute and facility directors about the study objective, and importance of the study, procedure and duration, risk and discomfort, benefits of participating in the study, and confidentiality. Study approval and ethical clearance were obtained from the University of Gondar ethical review board. A formal letter of approval was taken from the respective regional health bureaus and zonal departments, and facility administration. Informed Consent was obtained by all participants and experts. All methods were performed in accordance with the relevant guidelines and regulations. Results A total of 365 facilities were included in the factor analysis during the tool development and validation process. Face validity was calculated by dividing the number of agreements of an individual to total number of raters to get the percentage of agreement per question15. To obtain the overall percentage agreement, we summed all total percentage agreements of each item and divided them by the total number of questions. The initial overall agreement score in the first round was 78%, indicating the need for restructuring of the tool. Based on the feedback provided by the panel of experts during this initial phase, the tool was revised as presented in [Table 4]. Table 4First round face validity assessment of Dagu maturity assessment tool.Item noExpert agreement (N = 10)Agreement percentage (%)I-CVI scoreQ18800.80Q24400.40Q3101001.00Q47700.70Q59900.90Q6101001.00Q77700.70Q87700.70Q97700.70Q109900.90Scale overall (Mean)7.878S-CVI: 0.78 First round face validity assessment of Dagu maturity assessment tool. A second-round panel discussion was convened to evaluate the updated tool using predefined criteria (Supplementary 1). Of the ten experts initially invited, three were excluded from this round due to their involvement in revising the tool after the first discussion. Consequently, seven panellists participated in the second-round validation process as shown in [Table 5]. Table 5Second round face validity after considering first round comments.Item no.Expert agreement (N = 7)Agreement percentage (%)I-CVI ScoreQ171001.00Q271001.00Q371001.00Q471001.00Q571001.00Q671001.00Q771001.00Q8571.40.71Q9685.70.86Q1071001.00Scale overall (Mean)6.795.7S-CVI0.96 Second round face validity after considering first round comments. In the second round, a critical agreement value of 96% was achieved, which is considered acceptable according to established content validity criteria15. The experts evaluated the tool to ensure that each item accurately reflected its intended purpose. The panellists assessed the relevance and importance of the domains, sub-domains, and stages of progression related to Dagu implementation maturity (Supplementary 2), as well as the items presented under each domain and sub-domain that illustrate the stages of progression (Supplementary 3). To assess domain-level content validity, the percentage of agreement was calculated for each item. The Content Validity Ratio (CVR) was computed by dividing the number of agreed-upon item ratings by the total number of questions23 as shown in [Table 6]. The average CVR across all items represents the overall content validity of the tool. Table 6Domain level content validity of the Dagu maturity assessment tool.No.DomainsEssential agreement (N = 7)Agreement percentage (%)CVR Score1Transaction Practice71001.002Data quality71001.003Data use and visibility71001.004Leadership, Management, and Governance (LMG)71001.005Efficiency71001.00Overall scale (Mean)7.0100Content validity ratio (CVR)1.00 Domain level content validity of the Dagu maturity assessment tool. The calculated CVR was compared with the critical value established for a panel of seven experts. As the initial CVR exceeded the critical value, subsequent validation was performed focusing on the sub-domains within each domain to ensure content credibility and importance. The content validity for the sub-domains was estimated at 0.99, which is close to 1. According to the content validity rule regarding the number of subject matter experts (SME) involved24, the high CVR found in the sub-domain content validity indicates strong content validity and confirms the adequacy of the tool as presented in [Table 7]. Table 7Sub-domain level content validity of the Dagu maturity assessment tool.No.DomainSub-domainExpert agreement (N = 7)Agreement percentage (%)CVR Score1Transaction ManagementReceive practice71001.00Issue practice71001.00Inventory practice71001.002Data QualityReceive completeness71001.00Receive timeliness71001.00Receive accuracy71001.00Issue completeness71001.00Issue timeliness71001.00Issue accuracy685.70.71Loss data quality71001.00Adjustment data quality71001.00Inventory data quality71001.003Data use and visibilityReport generation71001.00eRRF synchronization71001.00Stock status synchronization71001.00eRRF utilization71001.004LMGResource planning71001.00M and E71001.00Staffing71001.00Governance71001.00Facility level data utilization71001.005EfficiencyWastage rate71001.00Artificial stock out71001.00Stocked according to plan71001.00Scale overall (Mean)99.3Content Validity Ratio (CVR)0.99 Sub-domain level content validity of the Dagu maturity assessment tool. Stages of progression Stages of progression indicate the development or advancement of a given process or system. The panelists were requested to evaluate the progression of maturity levels from Level 1 to Level 4 to ensure each stage appropriately contributed to the tools’s alignment with the maturity continuum [Appendix 1]. Item analysis The overall mean score of the Dagu maturity assessment tool was 2.29, with item scores ranging from 1.55 (for the stock status synchronization item) to 2.91 (for the receive completeness item). The lowest mean, found in the stock status synchronization component, indicates that the Dagu users have limited stock status synchronization using the existing platform. In contrast, the highest mean was found in receive completeness, indicating that users experience good practice in receiving complete fields when receiving commodities within primary healthcare units [Table 8]. Table 8Item analysis of the dagu maturity assessment tool.Sub domainItemsMeanSDTransaction practiceReceive practice1.6370.75Issue practice1.920.86Inventory practice2.530.91Data qualityReceive completeness2.910.98Receive timeliness2.310.84Receive accuracy2.8170.93Issue completeness2.850.89Issue timeliness2.4930.92Issue accuracy2.8470.88Loss data quality2.4830.94Adjustment data quality2.660.96Inventory data quality2.5130.91Data use and visibilityReport generation1.9970.98eRRF synch1.7270.89Stock status synchronization1.550.73ERRF utilization1.720.82Leadership, management and governanceResource planning2.3370.88Monitoring and Evaluation1.6570.92Staffing2.3170.87Governance2.5730.89Facility level data utilization2.260.85EfficiencyWastage rate2.470.85Artificial stock out2.6230.91Stocked according to plan (SAP)1.8230.86SD: Standard Deviation; eRRF: Electronic report and requisition. Item analysis of the dagu maturity assessment tool. SD: Standard Deviation; eRRF: Electronic report and requisition. Factor loading and internal consistency The Dagu maturity assessment tool comprises five domains: transaction practice, data quality, data use and visibility, leadership, management, and governance, and efficiency. Factor loadings for all items were statistically significant at p<.001, ranging from a minimum of 0.488 (for stocked according to plan, under efficiency component) to a maximum of 0.977 (for issue data quality, under data quality domain) across all domains and sub-domains [Appendix 2]. The data quality domain revealed the highest reliability (0.949), which indicates strong factor loading across items such as completeness, timeliness, and accuracy for both receiving and issuing data. Data use and visibility exhibited a robust reliability (0.856), particularly in report generation and synchronization practices. Transaction practice and leadership, management, and governance showed moderate reliability at 0.758 and 0.827, respectively. Conversely, the efficiency domain indicated the lowest reliability (0.593) [Appendix 2]. Null model of confirmatory factor analysis In the null model, the domain and sub-domain were correlated without adding covariance. Accordingly, the factor loadings of each path from the latent variable to the indicators were reported [Figure 3]. The factors are presented within circles (transaction practice, data quality, data use, leadership and management, efficiency, receive data quality, and issue data quality). The arrows connecting each factor to its corresponding items represent the factor loadings, while the arrows pointing toward the items indicate the residuals. Null model for Dagu maturity assessment tool. Domains and sub-domains were correlated, including covariance, which shows the relationship between variables in terms of transaction management practice. The correlation and loadings for each path are reported in [Figure 4]. Final model for Dagu maturity assessment tool. The contribution of each item to explaining the overall Dagu maturity construct was assessed using square multiple correlation (R2). The R2 value indicates the proportion of variance in Dagu maturity that is accounted by each item in the model [Table 9]. Table 9Square multiple correlation (R2) of items used in confirmatory factor analysis.R-SquareEstimateR-SquareEstimateReceive practice0.323eRRFSynchronization0.770Issue practice0.557Stock status synchronization0.254Inventory practice0.595eRRF utilization0.800Receive completeness0.904Resource planning0.514Receive timeliness0.487Monitoring and evaluation0.492Receive accuracy0.806Staffing0.329Issue completeness0.954Governance0.365Issue timeliness0.642Facility level data utilization0.731Issue accuracy0.878Wastage rate0.407Loss data quality0.598Artificial stockout0.309Adjustment data quality0.527Stocked according to plan0.249Inventory data quality0.707Receive data quality0.746Report generation0.746Issue Data quality0.767errf: electronic report and requisition. Square multiple correlation (R2) of items used in confirmatory factor analysis. eRRF Synchronization errf: electronic report and requisition. Model fitness indices Model fit indices indicated that the χ2 test was significant at α = 0.05 with χ2 753 (P=.000). All fit indices, such as RMSEA = 0.079, SRMR = 0.062, TLI = 0.904, and CFI = 0.921, suggested a good model fit [Table 10]. Table 10Model fit indices of the confirmatory factor in Dagu maturity assessment tool.VariableGoodness of fit indices Dagu maturity ModelsNo of itemsX2(df)P valuedfCFITLIRMSEASRMRNull model2410940.0002400.8710.8520.0990.069Fullmodel247530.0002760.9210.9040.0790.062RMSEA: Root Mean Square Error of Approximation; CFI: Comparative Fit Index; TLI: Tucker–Lewis Index; SRMR: Standardized Root Mean Square Residual. Model fit indices of the confirmatory factor in Dagu maturity assessment tool. RMSEA: Root Mean Square Error of Approximation; CFI: Comparative Fit Index; TLI: Tucker–Lewis Index; SRMR: Standardized Root Mean Square Residual. Dagu maturity assessment scores and levels The Dagu maturity assessment model comprises five domains and twenty-four sub-domains, each with four stages or levels to evaluate Dagu implementation maturity. The model was developed following extensive revisions in item generation, face and content validation through panel discussions. Domain points were weighted according to their contribution to implementation maturity, and sub-domain weights were assigned based on the decisions made through panel discussions and expert judgment. The score for each item within a sub-domain was calculated by multiplying the maturity level (n) by its assigned weight (w). Domain scores were then summed to get the total maturity score for each domain [Table 11]. Each sub-domain was assigned a specific weight percentage (w), and maturity levels (L-n) were determined accordingly. The corresponding weighted scores (Score1 = n × w) were calculated for each sub-domain and aggregated to derive the total domain score (Score2). The overall maturity level was then computed by averaging the domain scores across all assessed domains. Based on the scoring matrix, maturity levels were categorized as “Initiate” if the score falls between maturity level 25%–49.99%, “Developing” if it ranges from 50% to 74.99%, “Defined” if the range falls from 75% to 99.99%, and “Optimized” if maturity level is 100% [Table 11]. Table 11Overall Dagu maturity assessment, scoring, and level matrix.DomainSub domainsSub-domain weight%(w)Maturity level (L-n)Score1(n*w)Score2=∑(Score1 for all subdomains∑(Score2 for all domains)/total number of domainsMaturity level (initiate (25%–49.99%), developing (50%–74.99%), defined (75%–99.99%%), optimized (100%) Overall Dagu maturity assessment, scoring, and level matrix. Discussion The Dagu maturity assessment tool is the first of its kind to measure the implementation maturity of a digital supply chain system in Ethiopia. The tool consists of five domains: transaction practice (3 items), data quality (9 items), data use and visibility (4 items), leadership, management, and governance (5 items), and efficiency (3 items). Transaction practice and data quality revealed the strongest factor loadings, highlighting that reliable data and consistent transaction practices play a major role in the implementation maturity of the digital tool. Literature vindicates that receive and issue of health commodities were the most frequently used features among activity dashlets logs at public health facilities indicating the importance of reliable underlying data for digital tool maturity1. The higher loading for inventory practice suggests that regular inventory management supports the overall inventory of supply chain system and enables digital tool maturity. Conversely, the lower loading for receiving practices indicates logistical delays or constraints that may hinder the implementation process and could be linked to limited ICT infrastructure and professional readiness, skill and perception to use the system2. The finding implies that regular inventory activities in the EPSS system have a direct impact on Dagu maturity and the reliability component is higher than the expected cut-off point22,25. The higher factor loading of this indicator suggests that the importance of strengthening the culture of inventory practice across Dagu implementing facilities. Every year EPSS conducts an inventory to improve the appropriate and efficient transaction of commodities during a dedicated time period from July 1–30 ofevery Ethiopian Fiscal Year (EFY). This dedicated inventory practice helps to measure the facilities’ performance and enable facilities to progress in system maturity25. Adherence to the mandated annual inventory period ensures that physical stock aligns with digital records, thereby providing the high-quality data necessary to reach advanced stages of the Dagu maturity scale26. The high loadings of the issue and receive completeness of commodities indicate that the maturity of the supply chain digital tool maturity is determined by how well facilities maintain data consistency during the receive and issue processes. In routine transaction processes, facilities receive pharmaceutical products from EPSS and private suppliers and issue them to dispensing units and clinics for utilization. The high loadings for issue and receive completeness indicates that maturity is largely determined by the consistency between the receive and issue of commodities. A possible justification could be that the automation of the supply chain system reduces errors that are common in manual systems, and the tool helps to reduce gaps between received and issued of health commodities. In addition, digitalization of the transaction process could enhance the operational performance of healthcare providers26. Data quality issues remain a major problem in manual systems; thus, healthcare providers may perceive that the digitalization of the healthcare system will improve health data quality and service provision. Ethiopia has been implementing structured and clear guidelines, procedures and policies at all levels of the health system to ensure data quality and its use for evidence-based decision-making27. Literature shows that digital tool implementation has shown to improve health system data28 and help healthcare providers to be effective in their job-aid29. The finding showed that data use and visibility enhanced Dagu implementation maturity, where eRRF utilization was found to be the factor with the highest loading, while stock-status synchronization had the lowest loading. The EPSS supplies health commodities every two months according to facilities’ requests and requisitions, as facilities register every business process and provide a request based on their activities. Dagu uses internal custom reports and enables the redistribution of products before expiry, such as those with a near expiry or short shelf life, based on stock status without waiting periods for reports and requests. The current findings align with evidence from the national guidelines and recent empirical evidence, which show that digitalizing the eRRF and stock-status synchronization enhances the data-capturing process and real-time supply chain management30. Dagu helps to manage product shortages by preparing custom reports for internal decision making so that high-level leaders can make decisions based on the data available at the facility level. Literature showed that evidence-based decision-making is a key component of improving the health system31, as the evidence generated from the health system for practice is a vital component in the provision of quality healthcare32. Data use and visibility enhance the ability to track, monitor and access real-time information and data across the entire supply chain management system. The finding showed that strengthening the integration of leadership and governance into routine and non-routine transaction process helps facilities sustain their progress along the maturity continuum. Leadership and management are key components in the healthcare sector, which enhance the effective and efficient implementation of health interventions and enable evidence-based decision-making to improve the health system12,33. Strengthening the leadership and governance component with clear and defined regulatory strategies helps drive digital tool maturity34. Moreover, strong health leaders’ engagement in the implementation process of digital tools, such as utilizing evidence generated for decision-making and clinical practice, could contribute to achieving the nationally desired level of digital tool maturity. Literature highlights the importance of a robust framework to monitor, direct and control digital tool maturity and create accountability among implementers in addition to ensuring the availability of relevant resources for digital tool maturity35,36. The finding implies that the efficiency component demonstrated a positive impact on Dagu maturity. Dagu helps to monitor product loss using the artificial stock out feature, which helps to identify stock discrepancies at the national, sub-national and facility levels. The finding showed that stock according to plan was found to be a low-loading factor compared to other items aggregated to measure Dagu maturity, while the waste rate and artificial stock-out alignment with routine transactions presented promising results within the Dagu maturity continuum. A previous study found that the internal consistency of the efficiency domain yielded a Cronbach’s alpha below the acceptable threshold suggested that presence of structural constraints and operational complexities that need to be emphasized during the implementation of the digital supply chain system26. Thefore, the findings related to the efficiency domain should be interpreted with caution, and the further refinement of items in this domain should be emphasized to ensure the future replicability of the tool and to enhance its practical usage in primary healthcare supply chain systems. The fit indices indicated that overall model fitness parameters were within the acceptable threshold and supported the adequacy of the model for the Dagu maturity assessment tool. The marginal fit of the model indices is likely due to the model complexity inherent in a 94-item instrument. Moreover, the study emphasized the theoretical and content validity of the tool for operationalization within different regions in Ethiopia over statistical optimization or modification indices. The cut-off points for Dagu maturity were categorized into four levels and weighted averages were assigned to the domain and sub-domain levels based on the weight of routine and non-routine transaction components contributed for Dagu maturity. Conclusion This study aimed to develop and validate a digital tool maturity assessment tool for the supply chain system at primary healthcare units in Ethiopia. The Dagu maturity assessment tool, developed through a thorough validation process, comprises five domains, twenty-four sub-domains and ninety four stages of progression. The tool evaluates facilities’ performance in implementing a supply chain digital system along a maturity continuum ranging from the initiating to the optimizing stage. The methodological approach, which includes expert validation, psychometric testing, and construct reliability, ensures the adequacy of the tool for capturing digital tool maturity progression. The high factor loadings in the data quality component indicate the importance of routine and non-routine practices in digital tool maturity. Moreover, the model fitness falling withing the acceptable threshold indicates that the items in the Dagu maturity tool help to track facilities’ progression in transitioning from a paper-first to an electronic-first system. This tool can be utilized for assessing the digital tool implementation maturity of health facilities in similar contextual settings. Supplementary Information Below is the link to the electronic supplementary material. Supplementary Material 1 Supplementary Material 2 Supplementary Material 3 Supplementary Material 4 Supplementary Material 5 Supplementary Material 6 Acknowledgements We thank all regional state health bureaus, district health offices, University of Gondar and DHA/JSI for their comprehensive support during the study. We are also grateful to data collectors from the selected Dagu implementation facilities. Our gratitude also goes to the facility store managers and facility directors for their directions provision of necessary documents during evaluation of facilities. This work would not be possible without the financial support from the DHA/JSI. The funder had no role in data design, data collection, analysis, and interpretation of the study findings. The mission of the DHA/JSI is to improve the quality of people’s lives through improving the quality and availability of healthcare services in Ethiopia. Author contributions TH: Conceptualized the study, analysed the data, wrote the manuscript, and edited the manuscript; TA and AAM: A series of revisions on manuscript, edited the manuscript, approved the manuscript, and guided throughout the study; AK, TAB, ST, AD and SW: Edited the manuscript, approved the manuscript, and guided throughout the study; TN: Conceptualized the study, a series of revisions on manuscript, edited the manuscript, approved the manuscript, and guided throughout the study. All authors have read and approved the final manuscript. Data availability All relevant data are within the manuscript. The data upon which these findings were developed can also be available upon request from the corresponding author. Declarations Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References Associated Data Data Citations Supplementary Materials Supplementary Material 1 Supplementary Material 2 Supplementary Material 3 Supplementary Material 4 Supplementary Material 5 Supplementary Material 6 Data Availability Statement All relevant data are within the manuscript. The data upon which these findings were developed can also be available upon request from the corresponding author.