Validation of the PECUNIA reference unit costs templates in Spain: a useful tool for multi-national economic evaluations of health technologies Authors: Lidia García-Pérez, Renata Linertová, Aránzazu Hernández-Yumar, Cristina Valcárcel-Nazco, Jhoner Perdomo-Vielma, Pedro Serrano-Aguilar, Mencia R Gutierrez-Colosia, Luis Salvador-Carulla, Enrique Fernández-Vega, Susanne Mayer, Judit Simon, the PECUNIA Group Published: 2024 ABSTRACT AbstractBackgroundThe PECUNIA Project was funded by the H2020 programme in which 10 partners from six countries participated. The aim was to develop standardized, harmonized and validated methods and tools to calculate costs in different sectors (such as health and social care, education among others), with the purpose of facilitating comparability of economic evaluations of health technologies across European countries. In this paper we report the first validation of the developed reference unit cost templates in Spain.MethodsThe evaluation of the PECUNIA Reference Unit Cost (RUC) Templates involved usability, transferability and feasibility assessment. Applicability tests were performed to estimate the cost of a selection of 15 resource items by means of the RUC templates in Spain and in four Spanish regions. External validation involved comparison with existing unit costs.ResultsIt was possible to estimate the cost of five services (dental care and general practitioner in the Canary Islands, general practitioner in Spain [tariffs], health-related day care centre and education services provided in a special education school in the Basque Country), car vandalism as an example of potential health-related consequences, and informal care in Spain. The templates were feasible although data completeness depended on the type of data needed to estimate the costs. The templates are transferable across countries although comparability depends on the services available in each jurisdiction.ConclusionsThe PECUNIA RUC Templates are free and feasible tools to estimate comparable reference unit costs across countries. Although more validation exercises are needed, they seem useful tools to perform robust multi-national economic evaluations and increase the transferability of cost-effectiveness studies of health technologies in Europe. However, they cannot compensate for the lack of data across jurisdictions.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12962-024-00601-9.Keywords: Costing, Economic evaluation, Europe, Services, Societal perspective, Spain, Unit costs, Validation Introduction A (full) economic evaluation (often called cost-effectiveness analysis) of a health technology is a type of study where the costs and consequences of two or more possible interventions are compared. This type of study is important in health technology assessment (HTA) given that cost-effectiveness is a criterion on which to make decisions, for example, about the reimbursement of health care technologies by several health care systems in Europe and other countries [1, 2]. The unit costs used in economic evaluations have been identified as one of the most important factors that can impact the transferability of these studies between jurisdictions or countries [1, 3–5]. Adapting a study in one country to another country in the framework of a multi-national economic evaluation (primary study with individual data or model-based study) would require comparable costs, that is, costs estimated according to similar methods for similar services to make fair comparisons in terms of unit costs and results [6, 7]. Otherwise, different methods and non-comparable parameters would affect the policy decisions, based on heterogeneous results [8]. For example, Mayer et al. detected a difference of 173% between the lowest and highest unit cost estimated for the general practitioner (GP) consultation using Austrian data adopting costing methods from four European countries [9]. A recent scoping review found heterogeneity of costing methods between sectors and country guidelines, often driven by data availability and reimbursement systems [4]. Multi-national economic evaluations would reduce research costs and make the process of reimbursement decision-making itself quicker and more efficient, especially in times of restrictive budgets. However, this would need further harmonization in terms of methods and comparable and standardized country-level unit costs [10]. The main aim of the PECUNIA (ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions) project was to set out standardised costing and outcome assessment systems that directly enable comparability, applicability and transferability of cost-effectiveness evidence for health-related interventions within and across countries [8] [https://www.pecunia-project.eu]. To attain this goal, six tools denominated Reference Unit Costs (RUC) templates were devised by PECUNIA partners from Austria, Germany, Hungary, The Netherlands and the United Kingdom [11, 12]. By RUC we mean the cross-sectoral, cross-country standardized methodology applied as reference for unit cost development in the PECUNIA project. One of the objectives in the project was to go further than health sector costs, therefore, several resource use items (mostly services) for several sectors were selected to test the tools. These sectors were health care, social care, education, criminal justice, employment, and patient, family and informal care. The reason to include all these sectors lays on the recommendations by many national guidelines, including the Spanish ones, of adopting a societal perspective when conducting economic evaluations [13–16]. Depending on the health problem and the technology, specific services from other sectors than health care would be needed. Nevertheless, the truth is services from certain sectors are not usually included in economic evaluation from societal perspective, such is the case of criminal justice or education [17]. Although this type of services is not relevant in many assessments, they could be crucial in the cost-effectiveness analysis of mental health-related technologies. One of the aims of the project was to validate the costing tools in an additional country. Following pilot testing in Austria, Germany, Hungary, the Netherlands and the United Kingdom/England, the templates were validated in Spain. Spain was chosen for this objective due to two main reasons. Firstly, Spain does not have an established national unit cost programme. There is a health-related unit costs database in Spain by a private company and accessible after paying an annual fee [18]; no other databases of unit costs for social care nor education sectors exist, apart from some statistics. Secondly, Spain has a decentralised health care system structure, which enables the evaluation of intra-country variations as well. The Spanish public health care system is a national health system, Sistema Nacional de Salud (SNS). It is organized at two levels—national and regional—mirroring the country’s administrative division. Health competences are transferred to the 17 Autonomous Communities or regions with the national level responsible for certain strategic areas (such as vaccination or newborn screening programmes) as well as for overall coordination of the health system [19]. The provision of education and social care is also decentralised. The aim of this paper was to present the process of validation in Spain of the intra-country and inter-country transferability of unit costing tools developed in the PECUNIA Project. Methods Description of the PECUNIA RUC Templates The PECUNIA RUC Templates (Table 1) were drawn up as Microsoft Excel® (2013)-based worksheets [8, 20, 21]. The PECUNIA RUC Templates are jointly presented in a Microsoft Excel file with several sheets for different types of resource items (services/consequences/other) and functions (data collection, estimation of unit cost). Three templates were specifically designed to estimate the unit cost of services delivered in any sector based on different costing methods: “SERVICE-1 top-down micro-costing template”; “SERVICE-2 top-down gross-costing template”; “SERVICE-2 short form top-down gross-costing template”. These allow the user to calculate RUCs per different units: per working hour (with or without travel) or per average direct client contact in the case of SERVICE-1 and per day, night, contact or other unit of measurement in the case of SERVICE-2. All include direct costs (personnnel), other direct costs such as travel or equipment for example, and other overhead costs, including administrative staff. The difference between templates is the level of detail [12]. Table 1Reference unit cost tools developed by PECUNIA partners and items used to test the toolsToolItems used to test the toolSERVICE-1: SERVICE-1 top-down micro-costing template, plus a SERVICE-1 data collection sheetDental careSERVICE-2: SERVICE-2 top-down gross-costing template, plus a SERVICE-2 data collection sheetHealth-related day care centreEducation services provided in a special education schoolSERVICE-2 short: SERVICE-2 short form top-down gross-costing template, plus a SERVICE-2 short data collection sheetDental careGeneral practitioner*TANGIBLE CONSEQUENCES template, plus a TANGIBLE CONSEQUENCES data collection sheetCar vandalismPRODUCTIVITY LOSS template, plus a PRODUCTIVITY LOSS data collection sheetAbsenteeism / PresenteeismUnpaid workPERSONAL TIME templateInformal care*SERVICE RAW data sheet (auxiliary sheet) was alto tested for general practitioner Reference unit cost tools developed by PECUNIA partners and items used to test the tools Health-related day care centre Education services provided in a special education school Dental care General practitioner* Absenteeism / Presenteeism Unpaid work *SERVICE RAW data sheet (auxiliary sheet) was alto tested for general practitioner An additional RUC Aggregation/Weighting (RAW) data sheet (“SERVICE RAW data sheet”) was also developed for the harmonised aggregation of data collected from multiple sources to be used with any of the three service templates as needed. Another template (“TANGIBLE CONSEQUENCES template”) was designed for the potential tangible consequences of human behaviour potentially related to ill-health, whose costs could be relevant in an economic evaluation (for example, car vandalism). Another two templates were designed to estimate the unit cost of productivity loss due to ill-health (“PRODUCTIVITY LOSS template”) and personal time for informal care (“PERSONAL TIME template”). The template to estimate the productivity loss costs includes both the human capital method and the friction cost method. All templates share an aesthetic, friendly and homogeneous appearance. There is an initial sheet with an introduction to the PECUNIA project and the organization of the other sheets in the file. There are sheets to collect data and sheets to estimate unit costs. Templates may be self-completed by a researcher, based on existing secondary data or primary data collected with the data collection templates. These data collection templates were designed to be self-completed by the service provider. The sheets have three types of cells: cells with instructions, cells with data to be entered by researcher and cells with pre-specified entry instructions or automatic calculation (all of them shaded differently). The structure is also similar across the templates, although the length (number of data or cells to be filled in) varies depending on the topic. All sheets include modules to collect basic information on the calculated RUC. For example, in the templates designed to estimate the unit cost of services, there is a module for the service description (resource item name, unit of measurement, service definition, sector, funding source, etc.) and there is another module to collect basic information related to the unit cost (unit cost year, currency, country, data source, etc.). There are other modules for inputs and for results. Furthermore, the meta-data displayed in the linked RUC Compendium indicates the type of funding source of the services that were used to calculate the RUC, classified as public, private or mixed. SERVICE-1 and SERVICE-2 unit cost (also the SERVICE-2 short form) templates have a ‘STOP’ message at the end to make the user consider whether the calculated estimates are sensible (‘Do these unit cost estimates look sensible?’) [12]. Applicability tests The “validation” task was led by SESCS (the HTA unit in the Canary Islands, Spain) and supported by PSICOST Scientific Association (a Spanish scientific society for the study of mental health-related costs). Several tests were performed along 2020 to test the applicability of the RUC templates by means of the RUC estimation of the 15 resource items nationally in Spain and in four Spanish regions (Andalusia, Basque Country, Canary Islands and Catalonia). The selected Spanish regions represent a variety of regions in terms of geography, wealth and development. The 15 items by sector were: health care sector: dental care service; GP service; health-related day care centre (mental health); social care sector: nursing home; health-related support helpline, online and/or telephone; education sector: education services provided in a special education school; educational therapy (primary school); educational therapy (secondary school); criminal justice sector: police contact; jail (night); car vandalism; employment: absenteeism/presenteeism; unpaid work; patient, family and informal care: informal care. The applicability test was conducted following these steps: The templates and service names and their descriptions were translated into Spanish. The instructions for use were analysed carefully to be able to explain them to users if needed. The templates and service names and their descriptions were translated into Spanish. The instructions for use were analysed carefully to be able to explain them to users if needed. Key people and bodies were contacted in each sector and each region or at national level. A short questionnaire was sent to key contacts to ask them whether the selected services were available in their region (identification) and whether the definitions corresponded to the definition of the service available in their region. By replying to our requests, they tacitly consented to participate in the project. Key people and bodies were contacted in each sector and each region or at national level. A short questionnaire was sent to key contacts to ask them whether the selected services were available in their region (identification) and whether the definitions corresponded to the definition of the service available in their region. By replying to our requests, they tacitly consented to participate in the project. The key contacts were asked to complete the data collection templates with the relevant data to estimate the cost of the service / resource item (in Euros from 2019, the year the research was conducted). Reviews of scientific studies and other sources including databases and collaboration with experts and other agents were occasionally required to complete the templates. The key contacts were asked to complete the data collection templates with the relevant data to estimate the cost of the service / resource item (in Euros from 2019, the year the research was conducted). Reviews of scientific studies and other sources including databases and collaboration with experts and other agents were occasionally required to complete the templates. Finally, the qualitative assessment of the usability, transferability and feasibility was conducted by one researcher (economist) and discussed with other researchers in the team. To guide this assessment three tools were used. The ‘System Usability Scale’ (ten items, Likert scale) was used to guide the evaluation of the usability of the templates [22]. Transferability was analysed by means of a set of criteria (aspects) devised based on the EURONHEED tool [23]; and feasibility was analysed by means of a set of indicators developed based on Bouwmans et al. [24] and structured into three categories: response, completion time and data completeness. Finally, the qualitative assessment of the usability, transferability and feasibility was conducted by one researcher (economist) and discussed with other researchers in the team. To guide this assessment three tools were used. The ‘System Usability Scale’ (ten items, Likert scale) was used to guide the evaluation of the usability of the templates [22]. Transferability was analysed by means of a set of criteria (aspects) devised based on the EURONHEED tool [23]; and feasibility was analysed by means of a set of indicators developed based on Bouwmans et al. [24] and structured into three categories: response, completion time and data completeness. A table with the classification of items by sector and planned template tests is included in Additional file 1 (Table S1). The results of the tests per service are presented as cases. External validation of unit costs The calculated unit costs were externally validated. Estimations were compared to existing national/regional unit costs. These were obtained by searching bibliographic databases, government gazettes, health costs databases [18], statistics and contacting some experts who consented to participate in the project. Results Feedback from the validation team to the development team in the PECUNIA project resulted in modification of some template features. Nevertheless, the following results are valid for the current version (version 1.1, May 2023) of the PECUNIA RUC Templates. Applicability tests To reach the key organizations/individuals that could complete the unit costing templates, 69 persons from 55 organizations or departments were contacted. Most contacts were employees in the public sector (mental health experts or economists, from HTA units, accountancy departments, (mental) health care departments, among others). In the end, it was possible to estimate the cost of seven out of 15 items with the PECUNIA RUC Templates, i.e. dental care, general practitioner, health-related day care centre, education services provided in a special education school, car vandalism, absenteeism/presenteeism, informal care (see Table S2 in Additional file 1). Despite the effort, it was not possible to reach any key contact in the social care sector who could fully or partially fill in a template with data. The same applies to other health care and education services in some regions and for criminal justice sector items. Usability The evaluated templates were Microsoft Excel files. Although this is not free software, it is broadly known and used, and could be easily converted into a free and open software file. We found the PECUNIA RUC Templates highly usable for (health) economists and researchers/technicians with some knowledge of (health) economics and/or accountancy. For this type of professional, the templates would be easy to use, with a fast learning curve. They would be helpful and efficient, as well, to keep a register of data and methods, including sources, for future reference. Some of the cells were protected against errors and the templates were potentially customizable at the time. The latest version of the templates is protected against customization [21]. In any case, the main limiting issues identified were not the templates themselves but the lack of data and the fact that those who could easily access the data do not normally need to know the unit costs of services or compare them with other countries. Feasibility Given the number of geographic areas and items/services selected, there were enough opportunities to test the PECUNIA RUC Templates. However, the response was disappointing. Overall, we managed to estimate three health care services in two different regions (Box 1, 2, 3 4), one education service in one region (Box 2), and the cost of three other resource items regionally and in Spain: vandalism (Box 5), productivity loss (Box 6) and informal care (Box 7). This was achieved using five out of six templates, plus the auxiliary RAW sheet. Example: Dental care service SERVICE-1 Top-down micro-costing template was partially filled in for the estimation of dental care contacts by key contacts in the Canary Islands. According to the methods in the top-down micro-costing template, the data collected were not sufficient to estimate the RUC of dental care services. It was possible to estimate the RUC for dental care in the Canary Islands using the SERVICE-2 short template, a less data-demanding template. Example 1: Health-related day care centre, mental health It was possible to obtain data to fill in the “SERVICE-2 data collection” template with data from a mental day care hospital in Bizkaia, one of the three provinces in the Basque Country. After completing the template, the contacts confirmed that: a) the costs do not include pharmaceutical-related costs except for some imputation of hospital drugs; and b) the number of actual annual working hours is not really the actual number of hours but the number of hours agreed for work conditions in the public regional health service. The data provided by the contact was sufficient to estimate the RUC, although it was not deemed fully precise. One limitation of this RUC is that it may not represent all regions in the Basque Country. A code would be needed to compare this service to other similar services in other regions and countries. Example 2: Education services provided in a special education school A very specialized health-education centre in Bizkaia, one of the three provinces in the Basque Country, was identified. This service is provided by more than one sector, or to be precise, is funded by two sectors: health and education sector. A contact in the regional health service in the Basque Country reported the costs paid by the public health system (psychiatrist, psychologist, nurse overheads). However, it was not possible to contact the education department to gather data related to the resources and costs paid for by the education government. Consequently, the cost of the service estimated with the “SERVICE-2 data collection” template was underestimated Other limitations were: a) the use of the unit “contact”, instead of that recommended‚ per student per school day’; b) the number of annual working hours is not really the actual number of hours but the number of hours agreed for work conditions in the mental health system (OSAKIDETZA) (maximum number of working hours, overtime not considered); c) this service is only one type of special education for people with special needs and mental health problems; d) it was based on just one centre in one province in one region in Spain. A code would be needed to compare this service to other similar services in other regions and countries. Example: General practitioner (GP) SERVICE-2 Top-down gross costing template was partially filled in for the estimation of GP contacts by key contacts in the Canary Islands. However, according to the methods in the top-down gross template, the data collected were not sufficient to estimate the RUC. This was observed for other services tested in Spain and other PECUNIA countries. As a result, a less-data demanding template, the SERVICE-2 short template, was designed. The data provided by the regional health service in the Canary Islands (annual costs, contacts and unit costs) enabled estimation of the RUC for GP (contact at the primary care centre for people aged over 14 years) by means of the SERVICE-2 short template. This RUC coincided with the unit cost provided by the source, which was the costing department of the body that is both provider and payer of the service. Consequently, we deemed that the RUC was reliable and close to the actual unit cost (in the Canary Islands), where no other source or unit cost could be considered a better comparator/gold standard. The data would enable the estimation of RUCs for a contact with the GP in the centre, a visit at home or a phone contact, independently. We were also able to estimate the RUCs for other services such as dental care (at primary care level), paediatricians or GPs acting as paediatricians at the primary care level, psychologists and psychiatrists. Example: General practitioner (GP) The PECUNIA SERVICE RAW sheet was used to estimate the Spanish mean cost for the GP contact, using the tariffs or public prices published in the 17 Spanish autonomous government gazettes (there are 17 regions with their corresponding 17 regional health services in the Spanish NHS). The mean was weighted by the population in each region in Spain. These tariffs are used very often when there are no other alternative values available, although they are not considered reliable proxies for economic costs. The PECUNIA SERVICE RAW sheet is overall a useful way of collecting data and performing this type of analysis in a harmonized and methodologically robust way. It also enables amending the methodology (for example, if the researcher considers the estimation of standard deviations or exclusion of the lowest and the highest value/s). Example: Car vandalism The cost of car vandalism was estimated using the methods recommended by PECUNIA, that is, the average cost per incident is the total cost attributed to the annual number of incidents divided by the annual number of incidents. It was possible to fill in the “TANGIBLE CONSEQUENCES data collection sheet” with the data provided by The Consorcio de Compensación de Seguros (Spanish Insurance Compensation Consortium). This is a public business organization in Spain that performs many functions within the insurance field, including cover of extraordinary risk, mandatory vehicle insurance, etc. [https://www.consorseguros.es]. This contact provided a database with the cost of each damaged vehicle associated with riots and disturbances for the last ten years in Spain and in each Spanish region. There was some variety in RUCs observed depending on the region and the period chosen for the estimation. It was not possible to externally validate the cost of vandalism as no other source of data, such as insurance companies, could be contacted successfully. Nevertheless, it appears plausible that the unit cost of minor incidents could be lower than that estimated here. The “PRODUCTIVITY LOSS template” includes, among others, a module for absenteeism, a module for presenteeism and a module for unpaid work. The template offers the two approaches to estimate the cost of productivity loss, HCA and FCM. If the method to estimate the productivity loss is HCA, then the national average hourly wage is recommended for absenteeism and presenteeism. The national average hourly wage is available for Spain from Eurostat. This is a labour cost as it includes the remuneration of employees plus taxes minus subsidies [46]. This is an existing unit and validated cost estimate from an available and reliable source. The Spanish Office for National Statistics (Instituto Nacional de Estadística) reports quarterly data [47] and this enables calculation of the average wage per hour per region in Spain. To estimate productivity loss by means of FCM, it was possible to identify the length of the friction period from a secondary source, which is 72.60 days in Spain [48]. To estimate unpaid work, it was not possible to identify the national average costs for home help. Alternatively, and following PECUNIA’s recommendations, we used the calculation of national average costs of home help/informal care from market prices (Box 7). Example: Informal care The “PERSONAL TIME template” includes, among others, a module for Proxy good method for informal care unit cost calculation (primary data collected by the researcher). The average cost per hour is recommended to be the average price per hour of ten providers offering care. According to the Spanish decree that regulates the special work relationship of family home services between families and workers [49], the reference minimum salary for external home employees working by hours in Spain is the Minimum Interprofessional Salary (MIS) in Spain. As this is a national decree, there is no expected variability among regions in Spain. The work relationship between families and home help companies is not regulated by that decree, so we used it to test the template with market-based prices. We contacted/searched 53 companies/websites and requested quotes for a month of home service of five hours/day and five days/week for two different types of contracts: (i) contract between the family and the worker where the company is only an intermediary handling the paperwork; (ii) contract between the family and the company which offers the service. We received 30 quotes that enabled estimation of cost per hour. It was not possible to obtain ten prices from ten providers for all regions. There was some observed variety in RUCs depending on the region and type of contract. It is unclear which cost should be used in an economic evaluation, but we could estimate a mean and range for the RUC of home care in Spain. Knowing the percentage workers per type of contract in the country would enable a weighted estimation of unit cost. For the external validation, it was possible to compare the estimated RUC with the legal and mandatory maximum costs per hour set in some regions for those home services provided by authorized companies and paid by public authorities. Response Given the low response we did not calculate the response rate but describe the efforts needed to estimate 7 out of 15 items. It was necessary to contact several people and organizations before reaching the right person who could complete the data collection sheet. In total, five participants were able to facilitate data to complete templates. A total of 22 persons/organizations did not answer. Four sources did not provide data once they were contacted and agreed to participate. Three contacts could only contribute with unit costs already estimated by other means. It was impossible to get answers from private companies such as insurance companies and dental practices (after unsuccessfully trying public services). More than 40 literature sources and/or websites were consulted looking for data and information related to the resource items, except for informal care as the sources contacted were mainly private companies and we had to require quotes or prices from them (Box 7). The COVID-19 pandemic also affected the response. More information is detailed in Table S1 in Additional file 1. Completion time The time needed to understand and manage a template for the first time is short (less than two hours) from the point of view of a health economist (even without instructions). However, the time required to attain the best potential unit cost could vary depending on the available data meeting the template’s required level of information. Once the collection data template is completed, the time needed to use the PECUNIA RUC calculation template and estimate a unit cost is very short, given that both templates are very similar in several sections. Data completeness on the template items The data needed to estimate the RUC by means of the SERVICE-1 template, i.e. the most extensive service costing templates adopting a top-down micro-costing approach, were unavailable (working hours, time spent with clients). We only managed to receive a partially completed template for one service (dental care) (Box 1). In some instances, the problem was the lack of data within an organization, in other cases the problem was the need to collect data in multiple organizations and/or sectors (Box 2). In contrast, the SERVICE-2 short template was completed using data provided by contacts for several services/regions (Box 3). This template is not very demanding in terms of data (description of services, total annual cost and number of contacts per year). It appears this prompted the response from the data owner. The development of RUCs for a tangible consequence such as car vandalism (Box 5) and productivity loss (Box 6) were both successful as the data needed were available at national level. Transferability The items included in the templates are sufficiently reported and/or can be defined according to needs/settings/jurisdictions. The methods needed to be followed to use the templates correctly are provided, including instructions for use of input data and local sources of unit costs. The templates are designed to select the perspective; define the type of funding and potential co-payments; change the unit of measurement; indicate the currency, the date(s) of the calculations/price year, even a subnational entity (region, for example); specify local values, sources and users’ comments; and present intermediate data and calculations in detail and separately. All this makes the templates good tools for the transferability of costs and methods. However, there were some issues identified during the process that can affect transferability. Firstly, terminological problems with implications for resource use measurement and cost calculation were identified [25, 26]. In one country similar services can have different names among regions, and services with similar names can involve different resources. A high degree of similarity between two services proved to be key to be able to compare services and their costs among regions/jurisdictions. When the service of interest does not exist or it is not possible to estimate its cost, one alternative option is to estimate the cost of an existing similar service. This requires a clear explicit definition of the service (as with ‘dental care’), the inclusion of the name of the service in the local language, and the use of standard codes provided by classifications such as the Description and Evaluation of Services and Directories (DESDE system) that could be combined with other classifications such as the World Health Organization Family of International Classifications [27, 28]. The PECUNIA methods and the used RUC templates, therefore, incorporate fields to add a code by means of the DESDE system. Secondly, the templates are primarily designed to estimate RUCs based on costs. If the only available input were not costs but tariff prices, expenditure or budget, the results should be interpreted with caution (Box 4). Consequently, the methods rely on some user knowledge about economics and accountancy. Thirdly, the methods followed by one organization to estimate unit costs based on analytical accounting could be more complex (regarding imputation for example) than the PECUNIA methods. In these circumstances, the PECUNIA RUC Templates could be seen as a burden by the data sources given the need for a homogeneous system for comparative reasons not being perceived. Finally, despite these drawbacks, the main problem in all instances appears to be the lack of sufficient available data related to the resource use and/or costs in the region. The SERVICE-2 short template, designed to be easier and overcome the lack of detailed data, has the slight disadvantage of being too simple, potentially introducing unintended shortcuts and enabling the omission of relevant information as data inputs are on a higher level of aggregation than in the longer SERVICE-2 template. Each provider in each country could fill in this information vaguely or with different data. This goes against harmonization and could make the results less comparable between countries. Discussion One of the main aims of the PECUNIA Project was to develop methods and tools to harmonize the estimation of RUCs that would be useful for comparability of costs and economic evaluations of health technologies across Europe. In this paper, we presented the first validation of the PECUNIA RUC Templates in Spain. The findings were used for improvement of the PECUNIA RUC Templates, which were further revised based on the feedback coming from this validation exercise. The versions validated during the project could be considered a software at TRL 5 according to the Technology Readiness Level (TRL) scale [12]; that is, the software was a prototype, the alpha version of the software functionalities tested by outsiders of the development team. Nevertheless, they were also tested in a relevant environment (TRL 6) as part of an HTA project and in other countries. The most current versions of the PECUNIA RUC Templates can be found online [21]. A secondary aim of this paper is to contribute to the scarce evidence on the validation of tools helpful for the economic evaluation of health technologies, as is the case of the instruments to collect resource use and cost from the patient perspective [29], and hopefully also useful for other researchers in other sectors. In fact, to our knowledge this is the first publication about the validation of a computer-based standardised comprehensive template for the collection of data from providers or secondary sources and the estimation of unit costs [30]. To validate the templates, key regional and national bodies in Spain were identified and contacted to complete the templates. The whole external validation process was used to draw conclusions about the overall feasibility and transferability of the PECUNIA templates. See Mayer et al. (2022) for more details about the methods and results of the quality assessment and the certainty of the RUC estimates by the PECUNIA consortium [31]. The time needed to understand and manage a template is short from the point of view of a health economist. However, this time depends on the number of data required and the number of sources that must be contacted to access the data. While some templates were not feasible (such as the SERVICE-1 top-down micro-costing templates), other templates could be completed by just accessing published data or looking for market prices. The user guide developed after the validation process should prove very helpful for those not used to this type of task. The development of the SERVICE-2 short template was deemed necessary due to the limited feasibility of primary data collection during the COVID-19 pandemic. In the end it was possible to estimate PECUNIA RUCs with this short template for dental care and GP because of the small amount of data required to estimate the RUC and because of its flexibility. A finding during the process of development and validation was the fact that the less precise the costing methods are, the more feasible they will be, as other researchers have stated before [32]. Hence, the added value of SERVICE-2 short template lies in its simplicity. PECUNIA approached the economic evaluation from a multi-sector point of view as the analysis of some health interventions or health problems should include spillover costs from other sectors to fulfil the requirements of a full economic evaluation from a societal perspective [33–36]. This prompted the creation of tools also usable for the estimation of costs from sectors different from the health care sector [12, 31, 37, 38]. This approach deserved a broad validation that would test resource items from as many sectors as possible. An applied project focusing on the mapping and deployment of the templates within the mental health services landscape in Vienna, Austria, is currently ongoing to fully explore these aspects [39]. There are multiple providers of public social services in Spain. Although there is a Spanish Ministry with social care competences, most are devolved to Autonomous Communities and services are usually planned, funded and provided by regional and local authorities. There is also private provision of social services paid out-of-pocket by patients and families. This heterogeneity made it impossible to estimate the cost of any social care service in the framework of the project. The selected services related to the criminal justice sector were police contacts and a night in jail. The police system in Spain is complex as there are several police bodies. Unfortunately, despite the major effort made to contact the key sources of data, it was not possible to apply the templates. By contrast, the experience with the TANGIBLE CONSEQUENCES templates was slightly different as they were completed easily thanks to the collaboration of a third party that facilitated a database with cost of vehicle vandalism in Spain. Spain is one of the countries in Europe that recommends both a (public) healthcare perspective and a societal perspective for economic evaluations [15, 16, 40]. Patient, family and informal care costs and productivity costs must also be included in this perspective according to Spanish guidelines [16, 40]. The PECUNIA PRODUCTIVITY LOSS template enables collection of the data needed to calculate homogenous costs across countries by means of different methods (Human Capital Approach or Friction Costs Method) to use them in multinational economic evaluations. In short, the templates were considered good tools for transferability in terms of methods, user instructions, descriptions of items and design among other aspects. This is supported by some qualitative research conducted during the study (see Additional file 2). The main problem encountered that could affect their transferability across countries was terminological variability. Across countries/regions we can find similar services with different names and different services with similar names. This issue has previously been raised by studies analysing ambiguity in the comparison of lists of services drawn from literature reviews, and studies evaluating efficacy and cost-effectiveness in specific services such as psychotherapy [25, 41]. These reveal the necessity and usefulness of classification systems and taxonomies that include standard codes for the analysis of service costs. The PECUNIA RUC Templates include cells to identify the services using several codes such as the DESDE Codes and others. There are services that can be provided jointly by several bodies within a sector or even by several sectors. This means that resources and costs can be collected from different organizations and, usually, different information systems. The lack of integrated information systems and protocols for sharing data is a limitation for the estimation of RUCs, irrespective of the PECUNIA methods. In Spain special education is funded by at least two sectors (health care and education sectors). In public schools some services are provided by other level of administration (cleaning services are funded by towns; education and healthcare are funded by the regional administration). Finally, some private schools are partially funded by public funds while other centres are fully paid for by families. All this makes it difficult to obtain all the relevant data to estimate the RUC. This study has a series of limitations. Firstly, we could not complete the estimation of RUCs for all the proposed resource items due to unexpected circumstances that delayed and/or hampered the collection of input data (i.e. unexpected challenges to access the targeted experts due to the COVID-19 pandemic, lack of time and/or (trained) staff, lack of data that could fulfil the PECUNIA RUC Templates’ requirements, and confidentiality of data). Secondly, it was not feasible to estimate the unit cost of any selected service for more than one region. This prevented robust conclusions about the intra-country variability of RUCs developed using the PECUNIA methods. On the other hand, we know that the estimated regional unit costs of some services are not representative of the whole country due to the lower prices in the Canary Islands in comparison to the Basque Country or Catalonia. Unit costs from different regions calculated following homogeneous methods would be needed to know how different prices between regions affect costs and to know which region could be used as a proxy. Thirdly, the scarcity of estimated unit costs by means of the PECUNIA RUC Templates prevented use of formal feasibility and transferability indicators and the calculations of statistics as planned. Finally, it was not possible to identify valid alternative costs to externally validate the unit costs estimated with the PECUNIA RUC Templates for most services. Despite these limitations, we believe we were able to demonstrate the validity of the PECUNIA RUC Templates. The main issue that remains in the harmonization efforts is the lack of input data or difficulties accessing the necessary input data, and these are context-related. We could say that, apart from the COVID pandemic, the main reasons behind the limitations are the two sides of the same coin: (1) the lack of national representative sources for most costs in Spain, but also (2) our exigency when testing the templates that made us look for data from public providers instead of private sources (given the fact that the biggest provider of health care in Spain is the public sector) and reject the use of some low-quality data (i.e. proxy statistics for the education sector or salaries reported on the news for the criminal justice sector). Ultimately, more applied validation exercises testing the PECUNIA RUC Templates should be conducted in the near future in Spain and other countries different from those involved in the PECUNIA project. This could be facilitated by the fact that the use of the PECUNIA RUC Templates is conditional upon sharing the calculated RUCs with the wider research community via the PECUNIA RUC Compendium (https://www.pecunia-project.eu/tools/ruc-templates). Several experts raised concerns over the barriers of several initiatives to harmonize costing methods over the years [42]. There are different analytical accounting systems between regions/countries and sectors. In a country such as Spain, without a national unit cost programme and with many services provided at regional level, the barriers encountered during the PECUNIA Project could be faced again by any researcher team in the process of conducting an economic evaluation as they are not circumstantial and hamper the estimation of ‘nationally valid’ unit costs [4]. Some of these issues are beyond PECUNIA’s control and will hardly be solved in the short term. Nevertheless, having tools such as PECUNIA costing templates, which can enable researchers to go from different systems to reference data, is a crucial step towards comparability across regions and countries. Moreover, there is a consensus among the research community about the need for harmonization of costing methods and standardization of unit costs in the future [4, 30, 43–45]. Hopefully, in the medium term we will be able to plan and invest resources in information systems to overcome the current shortcomings in data access. Conclusions The main conclusions of this work are two. Firstly, although due to aforementioned factors the response, completion time and data completeness were not as foreseen for all services and sectors, we could say that the templates’ structure and content make them feasible for use in different jurisdictions, for the estimation of unit costs and as a register for future reference. Secondly, as the PECUNIA templates were drawn up keeping in mind evidence about the lack of transferability of costs between settings and the recommendations to make economic evaluations more transferable, the templates will contribute to enhance the transferability of economic evaluations of health technologies or interventions. Further validation research should be conducted in the framework of real-life HTA. Supplementary Information Additional file 1: Table S1: Items by sector and templates, plans and goals. Table S2: Unit costs estimated with the PECUNIA RUC Templates and unit costs obtained from other sources. Additional file 2: Qualitative research. Acknowledgements We wanted to acknowledge the collaboration of: Sebastián Fuentes Tarajano (Costing Department at the Servicio Canario de la Salud, (regional health service in the Canary Islands)), Álvaro Iruin Sanz and Carlos Pereira Rodríguez (Mental Health department of OSAKIDETZA (regional health service in the Basque Country)), and other organizations, experts and collaborators, Àngels Fumàs Coma, Bibiana Prat Pubill, Carlos González Rodríguez, Eduardo Sánchez Iriso, Federico Alonso Trujillo, Juan Cabasés Hita, Laura Vallejo Torres, Pilar Campoy Muñoz, Pilar Tirado Valencia, Rosa Vidal Álvarez, Rosario Santana Ordóñez. Finally, we are grateful to Jason Willis-Lee, professional copyeditor specialised in biomedicine, for his assistance in drawing up the final manuscript. Members of the PECUNIA Group are: Simon, Judit; Medizinische Universität Wien, Austria. Berger, Michael; Medizinische Universität Wien, Austria. Fischer, Claudia; Medizinische Universität Wien, Austria. Łaszewska, Agata; Medizinische Universität Wien, Austria. Mayer, Susanne; Medizinische Universität Wien, Austria. Perić, Nataša; Medizinische Universität Wien, Austria. König, Hans-Helmut; Universitätsklinikum Hamburg-Eppendorf, Germany. Brettschneider, Christian; Universitätsklinikum Hamburg-Eppendorf, Germany. Duval, Marie Christine; Universitätsklinikum Hamburg-Eppendorf, Germany. Hinck, Paul; Universitätsklinikum Hamburg-Eppendorf, Germany. Hohls, Johanna Katharina; Universitätsklinikum Hamburg-Eppendorf, Germany. Konnopka, Alexander; Universitätsklinikum Hamburg-Eppendorf, Germany. Muntendorf, Louisa-Kristin; Universitätsklinikum Hamburg-Eppendorf, Germany. Brodszky, Valentin; Budapesti Corvinus Egyetem, Hungary. Gulácsi, László; Budapesti Corvinus Egyetem, Hungary. Evers, Silvia M.A.A.; Universiteit Maastricht, Netherlands. Drost, Ruben M.W.A.; Universiteit Maastricht, Netherlands. Janssen, Luca M.M.; Universiteit Maastricht, Netherlands. Paulus, Aggie T.G.; Universiteit Maastricht, Netherlands. Pokhilenko, Irina; Universiteit Maastricht, Netherlands. Hakkaart-van Roijen, Leona; Erasmus Universiteit Rotterdam, Netherlands. Hubens, Kimberley; Erasmus Universiteit Rotterdam, Netherlands. Sajjad, Ayesha; Erasmus Universiteit Rotterdam, Netherlands. Serrano-Aguilar, Pedro; Servicio Canario de la Salud, Spain. García-Pérez, Lidia; Servicio Canario de la Salud, Spain. Linertová, Renata; Servicio Canario de la Salud, Spain. Perestelo-Pérez, Lilisbeth; Servicio Canario de la Salud, Spain. Valcárcel-Nazco, Cristina; Servicio Canario de la Salud, Spain. Salvador-Carulla, Luis; Asociación Científica Psicost, Spain. Almeda, Nerea; Asociación Científica Psicost, Spain. Campoy-Muñoz, Pilar; Asociación Científica Psicost, Spain. García-Alonso, Carlos R.; Asociación Científica Psicost, Spain. Gutiérrez-Colosía, Mencía R.; Asociación Científica Psicost, Spain. Romero-López-Alberca, Cristina; Asociación Científica Psicost, Spain. Park, A-La; London School of Economics and Political Science, United Kingdom. Hollingworth, William; University of Bristol, United Kingdom. Noble, Sian; University of Bristol, United Kingdom. Thorn, Joanna; University of Bristol, United Kingdom Abbreviations General practitioner Health technology assessment Reference unit costs Technology readiness level Author contributions LGP, RL, CVN, PSA, LSC, SM, and JS designed the study. LGP, AHY, JPV, MRGC, and EFV acquired the data. LGP, AHY, JPV, SM, and JS analysed and interpreted the results of the study. LGP, CVN, JPV, MRGC, and EFV wrote the draft of the article. All the authors (LGP, RL, AHY, CVN, JPV, PSA, MRGC, LSC, EFV, SM, JS) reviewed the article and contributed with important intellectual content. PSA, LSC, and JS (principal investigators) obtained the funding. AHY, CVN, and JPV also contributed with administrative, technical, and logistic support. LGP, PSA, SM, and JS supervised the whole project. Funding This research is part of the PECUNIA (ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions) project. This project has received funding from the European Union’s Horizon 2020 research and innovation program under Grant agreement No 779292. The grant was assigned to Professor Judit Simon as principal investigator of the project and Dr Pedro Serrano as leader of work package 6 ‘Validation’. All views and opinions expressed in this article are those of the authors and do not necessarily reflect the views of the funding agency. Availability of data and materials No datasets were generated or analysed during the current study. Declarations Ethics approval and consent to participate Ethics approval was not needed at the time given the nature of the study. Ethical guidelines allowed implied or tacit consent by answering and participating in proposed activities (emails, filling in templates, focus groups) as no personal data was collected. Consent for publication Not applicable. Competing interests The authors declare no competing interests. Footnotes Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Contributor Information Lidia García-Pérez, Email: lidia.garciaperez@sescs.es. the PECUNIA Group: Judit Simon, Michael Berger, Claudia Fischer, Agata Łaszewska, Susanne Mayer, Nataša Perić, Hans-Helmut König, Christian Brettschneider, Marie Christine Duval, Paul Hinck, Johanna Katharina Hohls, Alexander Konnopka, Louisa-Kristin Muntendorf, Valentin Brodszky, László Gulácsi, Silvia M. A. A. Evers, Ruben M. W. A. Drost, Luca M. M. Janssen, Aggie T. G. Paulus, Irina Pokhilenko, Leona Hakkaart-van Roijen, Kimberley Hubens, Ayesha Sajjad, Pedro Serrano-Aguilar, Lidia García-Pérez, Renata Linertová, Lilisbeth Perestelo-Pérez, Cristina Valcárcel-Nazco, Luis Salvador-Carulla, Nerea Almeda, Pilar Campoy-Muñoz, Carlos R. García-Alonso, Mencía R. Gutiérrez-Colosía, Cristina Romero-López-Alberca, A.-La Park, William Hollingworth, Sian Noble, and Joanna Thorn References Associated Data Supplementary Materials Additional file 1: Table S1: Items by sector and templates, plans and goals. Table S2: Unit costs estimated with the PECUNIA RUC Templates and unit costs obtained from other sources. Additional file 2: Qualitative research. Data Availability Statement No datasets were generated or analysed during the current study.