Developing and validating an abortion care quality metric for facility and out-of-facility settings: an observational cohort study in Bangladesh, Ethiopia, and Nigeria Authors: Erin E. Pearson, Nirali M. Chakraborty, Sarah E. Baum, Jamie L. Menzel, Sally Dijkerman, Rezwana Chowdhury, Bekalu Mossie Chekol, Yinka Abiola Adojutelegan, Chiara Bercu, Bill Powell, Dominic Montagu, Andrea Sprockett, Caitlin Gerdts Published: 2023 ABSTRACT SummaryBackgroundDespite progress in assuring provision of safe abortion, substantial disparities remain in quality of abortion care around the world. However, no consistent, valid, reliable method exists to routinely measure quality in abortion care across facility and out-of-facility settings, impeding learning and improvement. To address this need, the Abortion Service Quality Initiative developed the first global standard for measuring quality of abortion care in low-income and middle-income countries.MethodsThis prospective cohort study was conducted in Bangladesh, Ethiopia, and Nigeria in 2020–2022. Participants included sites and providers offering abortion care, including health facilities, pharmacies, proprietary and patent medicine vendors (PPMVs), and hotlines, and clients aged 15–49 receiving abortion care from a selected site. 111 structure and process indicators were tested, which originated from a review of existing abortion quality indicators and from qualitative research to develop additional client-centred quality indicators. The indicators were tested against 12 clinical and client experience outcomes at the site-level (such as abortion-related deaths) and client-level (such as whether the client would recommend the service to a friend) that were expected to result from the abortion quality indicators. Indicators were selected for the final metric based on predictive validity assessed using Bayesian models to test associations between indicators and outcomes, content validity, and performance.FindingsWe included 1915 abortion clients recruited from 131 sites offering abortion care across the three countries. Among the 111 indicators tested, 44 were associated with outcomes in Bayesian analyses and an additional 8 were recommended for inclusion by the study's Resource Group for face validity. These 52 indicators were evaluated on content validity, predictive validity, and performance, and 29 validated indicators were included in the final abortion care quality metric. The 29 validated indicators were feasibility tested among 53 clients and 24 providers from 9 facility sites in Ethiopia and 57 clients and 6 PPMVs from 9 PPMV sites in Nigeria. The median time required to complete each survey instrument indicated feasibility: 10 min to complete the client exit survey, 16 min to complete the provider survey, and 11 min to complete the site checklist. Overall, the indicators performed well. However, all providers in the feasibility test failed two indicators of provider knowledge to competently complete the abortion procedure, and these indicators were subsequently revised to improve performance.InterpretationThis study provides 29 validated abortion care quality indicators to assess quality in facility, pharmacy, and hotline settings in low-income and middle-income countries. Future research should validate the Abortion Care Quality (ACQ) Tool in additional abortion care settings, such as telemedicine, online medication abortion (MA) sellers, and traditional abortion providers, and in other geographical and legal settings.FundingThe 10.13039/100000008David and Lucile Packard Foundation and the 10.13039/100010409Children's Investment Fund Foundation. Research in contextEvidence before this studyA plethora of abortion quality indicators are currently in use, and there is no validated standard for high quality abortion care. Existing indicators do not adequately capture client-centred elements of quality, nor are they relevant for increasingly common out-of-facility abortion care. To address this gap, we aimed to develop the first global standard for measuring quality of abortion care in low-income and middle-income countries.Added value of this studyThis prospective cohort study, conducted in Bangladesh, Ethiopia, and Nigeria in 2020–2022, provides the first validated indicators to assess abortion care quality in facility and out-of-facility settings in low-income and middle-income countries. A total of 111 indicators that originated from existing abortion quality indicators and from qualitative research to develop additional client-centred quality indicators were pilot tested with 1915 abortion clients from 131 sites across the three countries. The 29 validated indicators were feasibility tested among 53 clients and 24 providers from nine facility sites in Ethiopia and 57 clients and 6 proprietary and patent medicine vendors from nine sites in Nigeria. Overall, the final set of 29 validated indicators performed well.Implications of all the available evidenceValidated indicators in the Abortion Care Quality (ACQ) Tool could be used by abortion care sites, program implementers, and Ministries of Health to monitor quality of abortion care and implement quality improvement initiatives. Future research should validate the ACQ Tool in a wider variety of geographical and abortion care settings. Research in context Evidence before this study A plethora of abortion quality indicators are currently in use, and there is no validated standard for high quality abortion care. Existing indicators do not adequately capture client-centred elements of quality, nor are they relevant for increasingly common out-of-facility abortion care. To address this gap, we aimed to develop the first global standard for measuring quality of abortion care in low-income and middle-income countries. Added value of this study This prospective cohort study, conducted in Bangladesh, Ethiopia, and Nigeria in 2020–2022, provides the first validated indicators to assess abortion care quality in facility and out-of-facility settings in low-income and middle-income countries. A total of 111 indicators that originated from existing abortion quality indicators and from qualitative research to develop additional client-centred quality indicators were pilot tested with 1915 abortion clients from 131 sites across the three countries. The 29 validated indicators were feasibility tested among 53 clients and 24 providers from nine facility sites in Ethiopia and 57 clients and 6 proprietary and patent medicine vendors from nine sites in Nigeria. Overall, the final set of 29 validated indicators performed well. Implications of all the available evidence Validated indicators in the Abortion Care Quality (ACQ) Tool could be used by abortion care sites, program implementers, and Ministries of Health to monitor quality of abortion care and implement quality improvement initiatives. Future research should validate the ACQ Tool in a wider variety of geographical and abortion care settings. Introduction The Institute of Medicine and the World Health Organization (WHO) define quality healthcare, regardless of the type of services provided, as that which is safe, effective, patient-centred, timely, efficient, equitable, and accessible.1,2 These components of quality care have important and unambiguous implications for abortion service provision. However, no uniform standard that is valid, actionable, simple to collect and interpret exists with which to measure quality abortion care. Existing indicators for abortion care quality are poorly defined, inconsistently used, incomplete, and no consensus has been formed around their relevance within quality frameworks.3, 4, 5 Furthermore, existing measures do not fully address client-centred elements of quality and are not applicable to increasingly common out-of-facility administration of medication abortion (MA). A 2017 systematic review by Dennis, Blanchard and Bessenaar found 75 unique indicators used to measure abortion care quality with varying application.5 The authors articulated the need for consensus on a streamlined set of validated, evidence-based, and client-centred indicators.5 A primary challenge with a plethora of indicators is that researchers and program implementers may select, or independently develop, different indicators to measure the same, or similar, concepts. The use of such a wide variety of indicators makes it difficult to compare quality across programs and settings, to assess changes over time, and to improve care delivery. Further, little is known about how existing indicators were created and the evidence base to support them. When considering the utility of quality indicators, there are no valid or reliable data on the accuracy of how current indicators relate to abortion outcomes nor have they routinely captured the client's perspective on the quality of their care,6, 7, 8 which may have an important impact on outcomes. These gaps make it difficult to make improvements in the quality of care for patients and ultimately in reproductive health outcomes at scale. To address these gaps, in 2018, Metrics for Management, Ibis Reproductive Health, and Ipas launched the Abortion Service Quality (ASQ) Initiative, to develop the first global standard for measuring the quality of abortion care in facility and out-of-facility settings in low- and middle-income countries (LMICs). The initiative focused on LMIC settings due to the higher rates of abortion compared to high-income countries9 and the growing evidence of the prevalence of out-of-facility compared to in-facility abortion care in these settings.10 The ASQ Initiative used a “stakeholder first approach” to identify and select indicators for validation through regular engagement with the project's Resource Group.11 This manuscript describes the process for validating the indicators against selected outcomes and presents the final, validated indicators of abortion care quality resulting from the ASQ Initiative. Methods Study design The ASQ Initiative used a two-stage process to develop the abortion care quality metric. In the first stage, which is described in detail in Chakraborty et al. (2022),11 the ASQ team gathered tools used by abortion service delivery organisations to measure abortion quality and extracted 1860 existing technical quality indicators, which were streamlined and prioritised with input from the ASQ Resource Group. The ASQ team also conducted formative research with abortion clients in Bangladesh, Ethiopia, Nigeria, and Argentina to develop 39 person-centred quality of care indicators,12 which were underrepresented among the existing indicators. The first stage of the process resulted in 111 technical and person-centred abortion care quality indicators to be validated against abortion outcomes.11 In the second stage, described in detail in this manuscript, the ASQ team conducted a pilot test of the 111 abortion care quality indicators, which were collected from abortion clients, providers, and sites in Bangladesh, Ethiopia, and Nigeria between February 2020 and October 2021. The three countries were selected to represent a variety of regional and abortion legal contexts, which was critical for the study of abortion quality as restrictive abortion laws and abortion provider restrictions are known to negatively impact the quality of abortion care.13 Ethiopia has a liberal abortion law, allowing abortion on broad social and economic grounds.14 In Bangladesh, abortion is only permitted to save a person's life, but menstrual regulation is widely available in the public sector to establish non-pregnancy up to 12 weeks from the last menstrual period.15 Nigeria has a restrictive abortion law, allowed only to save a person's life, but MA drugs are widely available in pharmacies and through patent and proprietary medicine vendors (PPMVs), as drug sellers who do not have formal pharmacist training are known in Nigeria.16 After pilot testing, indicators were selected for the final abortion care quality metric using a two-step process. In the first step, indicators were considered for the final metric if they were associated with outcomes in the pilot testing analysis or were recommended for inclusion based on expert review by the study's Resource Group. In the second step, the ASQ study team evaluated indicators selected for consideration in Step 1 on content validity, predictive validity, and performance. The second stage of the process resulted in 29 validated abortion care quality indicators that were selected for the final metric and feasibility-tested in February and March 2022 in Ethiopia and Nigeria. Abortion care quality indicators, rather than the selected outcome measures, were selected for the final metric based on ease of measurement. All indicators can be measured via site checklists, provider surveys, and client exit interviews, while all client-level outcomes were measured 30 days post-abortion, which would require expensive follow-up and would require service delivery organizations to collect contact information for their clients, which can be sensitive in contexts with restrictive abortion laws. Ethics Prior to the start of data collection, this study received ethical approval from the Marie Stopes International (MSI) Ethics Review Committee (026-20AA), the Bangladesh Medical Research Council (BMRC/NREC/2019-2022/884), the National Health Research Ethics Committee of Nigeria (NHREC/01/01/2007-27/01/2020), the Ethiopian Public Health Institute (EHI-IRB-223-2019), and Allendale Investigational Review Board (SAFE032019). All participants provided written informed consent prior to taking part in the study. In consultation with the ethics committees, it was decided that site-level quality data would not be shared back with participating sites to protect the privacy of participants, especially providers working in small or low caseload sites where anonymity would be impossible. Sample For the pilot test, three regions were selected in each country to represent the diversity of the population, including Dhaka, Sylhet, and Rangpur divisions in Bangladesh; Abia, Kano, and Nasarawa states in Nigeria; and Amhara, Benishangul Gumuz, and Oromia regions in Ethiopia. Data were collected from facility sites in Ethiopia and Bangladesh and from out-of-facility sites in Bangladesh (hotline and pharmacies) and Nigeria (hotline and PPMVs). Detailed information on sample size determination can be found in Appendix I. The sample was drawn from existing sampling frames in each country, which varied by country and site type. In Bangladesh and Ethiopia, the sampling frame for facilities consisted of non-governmental organization (NGO) supported facilities in the public and private sector. In Ethiopia, the facility sampling frame consisted of 441 public sector health facilities, including 400 primary-level facilities and 41 secondary- and tertiary-level facilities, and 232 private sector health facilities, all of which were NGO-run. In Bangladesh, the sampling frame for facilities consisted of 70 public sector health facilities, including 45 primary-level facilities and 25 secondary- and tertiary-level facilities, and 44 private sector facilities, including 19 that were NGO-run and 25 that were private for-profit facilities. The sampling frame for pharmacies in Bangladesh came from a pharmaceutical company, which identified 94 pharmacies where MA drugs (mifepristone/misoprostol combi-packs or misoprostol alone) were sold. In Nigeria, the sampling frame for PPMVs came from NGOs providing support to PPMVs on the sale of MA drugs and included 169 PPMVs. One hotline was purposively selected in Bangladesh and one in Nigeria based on their orientation toward supporting abortion self-care (rather than providing information to locate clinic-based abortion care) and their call volume. The hotlines were based in Dhaka (Bangladesh) and Lagos (Nigeria) but served the entire country. Sites were stratified by region and site type (secondary/tertiary-level public facility, primary-level public facility, private/NGO facility, pharmacy/PPMV), ordered by the average number of abortion cases per month, and probability proportional to size (PPS) sampling was used to select study sites. If a selected site declined to participate, was no longer providing abortion care, or if the security situation did not allow for data collection, the site was replaced with the next site sampled through PPS. In addition to pharmacies (Bangladesh) and PPMVs (Nigeria) selected via PPS sampling, we included pharmacies and PPMVs that were geographically near selected sites. In Ethiopia, sites with a higher proportion of clients with a gestational age at or after 13 weeks were oversampled to ensure representation of this group. In Bangladesh, induced abortion clients were oversampled in sites that primarily served postabortion care (PAC) clients to ensure representation of both client types. Data collection Pilot testing Data collection began in February 2020 and was paused in March 2020 due to the COVID-19 pandemic. Data collection resumed in October 2020 with COVID-19 risk mitigation measures and was completed in October 2021. In addition to the COVID-19 pandemic, data collection was challenging due to the security situation in Ethiopia. Data collection procedures were adapted throughout the study in collaboration with the ethics committees to maximise safety of participants and research assistants. All data were collected using a tablet with data collection tools pre-programmed in the CommCare data collection application (Dimagi) or Qualtrics. Data were collected from clients (client exit surveys and 30-day follow-up surveys), providers (surveys and observations), and sites (site checklist). Detailed information on pilot test eligibility criteria and data collection procedures can be found in Appendix I. Resource Group engagement The ASQ Resource Group included 69 global experts, including physicians, researchers, community and youth advocates, representatives from Ministries of Health, and donors, from across South Asia, East and West Africa, North America, Latin America, and Europe who were engaged throughout the study to provide guidance at each stage. After the pilot test, the Resource Group members participated in group and one-on-one meetings to review pilot testing results.11 In addition to these meetings, a survey was sent to all Resource Group members in January 2022 to gather quantitative data that could be used in the process of validating the ASQ indicators. The ASQ Resource Group survey included two questions: 1) respondents were asked to provide input to calculate weights for an outcome index, and 2) they were asked to indicate which categories of indicators should be maintained in a final product, among those that were not associated with an outcome based on pilot testing results. The goal of the second question was to ensure that the ASQ metric did not exclude a category of indicators that the Resource Group felt was vital for measuring abortion quality (i.e., required for face validity). Indicator selection and feasibility testing Results from the pilot test and Resource Group survey were combined to select indicators for feasibility testing. As mentioned above, the ASQ Resource Group members were asked to indicate which categories of indicators (if any) should be maintained in the set of indicators for face validity, among those that were not associated with an outcome based on pilot testing results. The ASQ study team used the following criteria to create the pool of indicators for possible inclusion in the feasibility test: 1) Associated with at least 1 individual outcome or the client-level or site-level outcome index in the hypothesised direction based on pilot testing data, and 2) In a category that at least 50% of Resource Group members recommended including. The study team carefully reviewed the resulting indicators and evaluated each one based on content validity, predictive validity, and performance. A rating of high or low was assigned based on the following criteria: 1) Content validity (High: well-represents the quality domain; Low: partial representation of the quality domain); 2) Predictive validity (High: associated with >1 outcome [individual outcomes or outcome indices] or was the only indicator associated with an individual outcome; Low: associated with fewer outcomes); and 3) Performance (High: simple indicator [single tool and few questions] and low proportions of “don't know” and “refused” responses; Low: complex indicator [multiple tools and/or many questions] and high proportions of “don't know” or “refused” responses). A total of 29 indicators rated in the “high” category for content validity and at least one of the other criteria (predictive validity or performance) were selected for feasibility testing. Content validity was given greater weight than predictive validity or performance given the goal of creating a metric that would represent the quality domains deemed most important by the Resource Group.11 The rating of content validity was based on the ASQ study team's assessment of how well the indicator represented the quality domain compared to other indicators in the domain. Most of the indicators deemed to have low content validity were specific to only one aspect of the domain compared to other indicators that better represented the domain as a whole. For example, in the domain of Accessibility, the indicator “Site does not turn away women eligible for abortion” was rated as having high content validity because it better represented the Accessibility domain compared to other more specific indicators such as “Women seeking abortion have not been turned away due to stockouts of surgical abortion equipment over the past 3 months”, which were rated as having low content validity. Feasibility testing of the 29 validated indicators was conducted in a convenience sample of public health facilities in Ethiopia and PPMV sites in Nigeria in two regions in each country. Feasibility testing was not conducted in Bangladesh due to logistical challenges. The 29 indicators selected for feasibility testing were assessed using three data collection tools: client exit surveys, provider surveys, and site checklists. Provider observations were dropped for the feasibility test based on Resource Group feedback that observations are too expensive for routine quality monitoring, requiring a highly trained data collector (often a provider), and because observations are infeasible at low abortion caseload sites where quality may be lowest. Indicators that were measured on the provider observation during the pilot test were moved to the provider survey for the feasibility test, which necessarily meant that in the feasibility test, these indicators reflected “provider knowledge” as a proxy for “provider competence” of the skill being assessed. The provider survey consisted of true/false knowledge questions and clinical vignettes that were developed in consultation with the clinical sub-group of the ASQ Resource Group to ensure that the newly added questions would capture the same concepts as the provider observation. Measures Outcomes Twelve abortion outcomes, including nine client-level outcomes and three site-level outcomes, were used in the pilot test. Outcomes were selected by a sub-group of the ASQ Resource Group to include both clinical outcomes and person-centred outcomes expected to result from high quality abortion care. Additional information on outcome selection can be found in Chakraborty et al., (2022).11 All client-level outcomes, including clinical outcomes, were measured based on client self-report on the 30-day follow-up survey. Site-level outcomes including abortion-related severe adverse events and deaths were measured through the site checklist at facilities and through the provider survey at pharmacies, PPMVs, and hotlines. The list of outcomes and their definitions can be found in Table 1.Table 1Abortion Service Quality (ASQ) outcome definitions.OutcomeDefinitionIndex weightSite-level outcomes1No abortion-related deaths in past 12 monthsSite or provider reports no abortion-related deaths in the past year.5.212Abortion-related severe adverse events in past 12 months within expected rangeIf number of abortions in past year (induced + PAC) < 100: site or provider reports severe adverse events in <1.5% of abortion cases (can include 0 severe adverse events)If number of abortions in past year (induced + PAC) ≥ 100: site or provider reports severe adverse events in >0% and <1.5% of abortion cases (must be more than 0 severe adverse events, assuming that at high caseload sites, at least some severe adverse events are expected and 0 indicates poor reporting)Severe adverse events include blood transfusions, surgery, IV antibiotics, fluid resuscitation, missed ectopic pregnancy, hospital admission, or referral to a higher level facility.9.393No eligible clients turned away for abortion servicesProvider does not report denial of abortion services for eligible clients. Clients were considered to be turned away if they were denied services for any reason other than “sex-selective abortion” (Bangladesh) or “client did not have a valid legal reason” (Ethiopia and Nigeria).7.64Client-level outcomes4Client was treated with respect and kindness throughout the abortion processClient responded “Strongly agree” or “Agree” to both of the following questions: 1) “I was treated with kindness throughout my entire abortion experience at the site” and 2) “I was treated with respect throughout my entire abortion experience at the site.”12.945Client felt that they could cope with their painClient responded “Yes” to “Did you feel that you were able to cope with the pain you experienced during the abortion process?”9.456Client felt they knew what to do if adverse events occurredClient responded “Would have known what to do” to “If a problem had occurred such as too much bleeding, would you have: known what to do, not known what to do, or would you have been unsure?”14.887Client was able to access follow-up or intervention for issues related to the abortion, if desiredClient reported that they did not feel that they needed additional treatment at any point after their abortion procedure OR client reported that they felt they did need additional treatment and received it.6.738Client knew their abortion was complete or had a plan for what to doClient reported that they did not think they were still pregnant OR they did think they were still pregnant and had an effective plan for what to do. An effective plan was defined as planning to take more MA pills, planning to visit a health facility for treatment, or deciding that they wanted to continue the pregnancy.7.529Client was able to access ancillary services or referrals, such as contraceptive and STI/HIV services, if desiredClient reported that they wanted ancillary services and received the desired services or a referral OR client reported that they did not want ancillary services and did not receive them6.5510Client was no longer pregnant at 30 daysClient answered “No” to “Do you believe that you may still be pregnant today?”7.2411Client did not experience abortion-related infectionClient was considered to have an infection if they reported at least two of the following: 1) pain scored above 5 that lasted >24 h and did not improve with pain meds OR pain scored above 5 that lasted >24 h and included nausea, 2) fever/chills that lasted over 24 h, or 3) foul-smelling or discoloured discharge that lasted >6 days5.0012Client would recommend the service to a friendClient answered “Yes” to “If your friend needed an abortion and asked you where to go, would you recommend that she seek care at the same site where you had the abortion?”7.45 Table 1Abortion Service Quality (ASQ) outcome definitions.OutcomeDefinitionIndex weightSite-level outcomes1No abortion-related deaths in past 12 monthsSite or provider reports no abortion-related deaths in the past year.5.212Abortion-related severe adverse events in past 12 months within expected rangeIf number of abortions in past year (induced + PAC) < 100: site or provider reports severe adverse events in <1.5% of abortion cases (can include 0 severe adverse events)If number of abortions in past year (induced + PAC) ≥ 100: site or provider reports severe adverse events in >0% and <1.5% of abortion cases (must be more than 0 severe adverse events, assuming that at high caseload sites, at least some severe adverse events are expected and 0 indicates poor reporting)Severe adverse events include blood transfusions, surgery, IV antibiotics, fluid resuscitation, missed ectopic pregnancy, hospital admission, or referral to a higher level facility.9.393No eligible clients turned away for abortion servicesProvider does not report denial of abortion services for eligible clients. Clients were considered to be turned away if they were denied services for any reason other than “sex-selective abortion” (Bangladesh) or “client did not have a valid legal reason” (Ethiopia and Nigeria).7.64Client-level outcomes4Client was treated with respect and kindness throughout the abortion processClient responded “Strongly agree” or “Agree” to both of the following questions: 1) “I was treated with kindness throughout my entire abortion experience at the site” and 2) “I was treated with respect throughout my entire abortion experience at the site.”12.945Client felt that they could cope with their painClient responded “Yes” to “Did you feel that you were able to cope with the pain you experienced during the abortion process?”9.456Client felt they knew what to do if adverse events occurredClient responded “Would have known what to do” to “If a problem had occurred such as too much bleeding, would you have: known what to do, not known what to do, or would you have been unsure?”14.887Client was able to access follow-up or intervention for issues related to the abortion, if desiredClient reported that they did not feel that they needed additional treatment at any point after their abortion procedure OR client reported that they felt they did need additional treatment and received it.6.738Client knew their abortion was complete or had a plan for what to doClient reported that they did not think they were still pregnant OR they did think they were still pregnant and had an effective plan for what to do. An effective plan was defined as planning to take more MA pills, planning to visit a health facility for treatment, or deciding that they wanted to continue the pregnancy.7.529Client was able to access ancillary services or referrals, such as contraceptive and STI/HIV services, if desiredClient reported that they wanted ancillary services and received the desired services or a referral OR client reported that they did not want ancillary services and did not receive them6.5510Client was no longer pregnant at 30 daysClient answered “No” to “Do you believe that you may still be pregnant today?”7.2411Client did not experience abortion-related infectionClient was considered to have an infection if they reported at least two of the following: 1) pain scored above 5 that lasted >24 h and did not improve with pain meds OR pain scored above 5 that lasted >24 h and included nausea, 2) fever/chills that lasted over 24 h, or 3) foul-smelling or discoloured discharge that lasted >6 days5.0012Client would recommend the service to a friendClient answered “Yes” to “If your friend needed an abortion and asked you where to go, would you recommend that she seek care at the same site where you had the abortion?”7.45 Abortion Service Quality (ASQ) outcome definitions. In addition to measuring individual outcomes, an outcome index was constructed to give a holistic view of quality on all 12 dimensions measured by the outcomes. The outcome index was constructed based on weights provided through a survey of Resource Group members. On the survey, Resource Group members were given 100 points to allocate among the 12 outcome measures. Resource Group members had the option to assign 0 points to a given outcome if they did not think the outcome was important to include in the outcome index. 33 responses to the survey were received and used to construct the outcome indices. The number of points for each outcome were averaged across all survey responses to create weights for each outcome within the index (Table 1). Outcome indices (possible range: 0–100) were created at both the individual client level and the site level. Indicators A total of 111 abortion quality indicators were included in the pilot test and a subset of 29 validated indicators were included in the feasibility test. The process to develop the initial set of 111 indicators and the ASQ framework, which was developed to categorise the indicators, are described in detail in Chakraborty et al. (2022).11 The ASQ framework drew on the seminal quality of care framework by Donabedian (1966), including structure and process indicators, which are theorised to affect outcome measures17 and the Bruce-Jain family planning quality of care framework,18 and built on the domains for health service quality of care developed by Akachi and Kruk19 and the domains specific to abortion care developed by Dennis, Blanchard and Bessenaar.5 All ASQ indicators were categorised under the 12 domains of the ASQ framework. Indicators were defined at three levels: 1) client-level indicators measured through the client exit survey were coded 0/1 at the individual level, 2) indicators measured through the provider observation, provider survey, or a combination thereof were coded as a proportion of observations that met the indicator, and 3) indicators measured through the site checklist were coded 0/1 at the site level. All indicators were binary, indicating whether the abortion care was high quality or not, which reflected the ASQ study's goal of differentiating between high quality care and care that required follow-up from mentors and managers to ensure that all clients in the site received high quality care. Provider-level indicators were aggregated to the site level for analysis and were considered to have met the indicator based on an assigned threshold for high quality care. Most provider-level indicators were assigned a threshold of 80% (the site passed the indicator if 80% of providers passed) with the rationale that most providers on staff should pass the indicator, and those who fail can be supported by the majority of providers who pass. Provider-level indicators of individual patient interactions that could result in harm (e.g., use of uterine evacuation technology that is not WHO-approved) were assigned a threshold of 100%. After the feasibility test, site quality thresholds were set for client-level indicators so that all indicators in the final metric were defined at the site level (as described above, thresholds for provider-level indicators were set prior to the pilot test). Client-level indicators that were related to whether a service was provided (e.g., client reported that pain management was available regardless of economic status) used a threshold of 100% using the rationale that the same services should be available to every client to achieve high quality care. Client-level indicators related to the client opinion (e.g., client felt that the provider took enough time to explain what was going to happen during their abortion process) used a threshold of 90%, recognising natural variation in client opinions and assuming that some of this variation may be based on lived experiences rather than the site's quality. There is no established threshold for site-level quality of abortion care, but this approach is in alignment with similar quality of care literature.20,21 Data analysis To establish predictive validity in the pilot test, Bayesian models were selected due to their ability to handle rare outcomes22 and conservatism in cases of multiple comparisons.23 For each of the 12 individual outcomes, the study team documented hypothesised relationships between the indicators and individual outcomes. Hypotheses were reviewed by a second member of the study team and disagreements were discussed and resolved by the full ASQ study team. All 111 indicators were tested against the outcome indices. In the pilot test, only 3 of the 111 indicators had data missing on more than 2% of observations, and these 3 indicators were imputed using hotdeck imputation. Data were pooled across countries for analysis to increase power, which was justified based on findings from qualitative research that similar quality domains were relevant to clients across country contexts.12 Analyses were run on a client-level dataset for client-level outcomes and a site-level dataset for site-level outcomes. For client-level outcomes, Bayesian multilevel logistic regression models were used that included random intercepts by site and use of weakly informative priors based on an inverse gamma distribution. For site-level outcomes, Bayesian logistic regression models with weakly informative priors based on a normal distribution were used. One site-level outcome, no abortion-related deaths in the past year, was excluded from the Bayesian analysis due to lack of power. We present odds ratios and 95% credible intervals for associations with individual outcomes and assumed that an indicator was associated with a given outcome if the credible interval did not include one. The client-level and site-level outcome indices were continuous (possible range: 0–100), and Bayesian linear regression models were used. The models for the client-level outcome index included random intercepts by site. Means are presented, and we assumed that an indicator was associated with the index if the credible interval did not include zero. Analysis of indicators with individual outcome measures was prespecified in the protocol, though specific outcomes were determined by the ASQ Resource Group after completion of the study protocol (please see details above). Development of the outcome index and analysis of indicators with the outcome indices were conducted post-hoc in an effort to assess site-level abortion care quality holistically. All analyses were conducted using Stata/SE version 17.0 (StataCorp LLC, College Station, TX). Role of the funding source The funders of the study collaborated with the researchers in study oversight, but were not involved in study design, the collection, analysis, or interpretation of data, or writing of the report. The researchers are fully independent from the funders. Results Pilot testing results During the pilot test, a total of 131 sites were included, consisting of 49 facilities (23 in Bangladesh and 26 in Ethiopia), 44 pharmacies (Bangladesh), 36 PPMVs (Nigeria), and 2 hotlines (1 in Bangladesh and 1 in Nigeria). A total of 2168 clients were contacted for participation and 2077 clients (96%) were enrolled in the study across the 131 sites (Fig. 1). Exit surveys were completed with 2048 clients, and 1939 clients completed 30-day follow-up surveys (95% of those who completed exit surveys). Data from 1915 clients with complete exit and follow-up data were analysed across Bangladesh (n = 919 clients from 68 sites), Ethiopia (n = 550 clients from 26 sites), and Nigeria (n = 446 clients from 37 sites).Fig. 1Pilot test enrolment. Pilot test enrolment. Sociodemographic characteristics of client participants can be found in Table 2. Characteristics varied by country, reflecting the population. A majority of clients in the sample received induced abortion and most were <13 weeks gestation. The highest proportion of clients 13+ weeks gestation was seen in Ethiopia (29%) due to oversampling in this group. Approximately 75% of clients received MA, and in Nigeria 100% of participants were MA clients as recruitment only occurred in out-of-clinic settings (PPMVs).Table 2Client sociodemographic and abortion characteristics (pilot test data).BangladeshEthiopiaNigeriaTotal(n = 919)(n = 550)(n = 446)(n = 1915)n (%)n (%)n (%)n (%)Recruitment site Facility481 (52.3)550 (100)0 (0)1031 (53.8) Pharmacy or proprietary and patent medicine vendor (PPMV)404 (44.0)0 (0)264 (59.2)668 (34.9) Hotline34 (3.7)0 (0)182 (40.8)216 (11.3)Age (years) 15–1841 (4.5)39 (7.1)24 (5.4)104 (5.4) 19–24242 (26.3)228 (41.5)150 (33.6)620 (32.4) 25–48636 (69.2)198 (36.0)271 (60.8)1105 (57.7) Don't know/refused0 (0)85 (15.4)1 (0.2)86 (4.5)Education None/some primary399 (43.4)188 (34.2)27 (6.1)614 (32.1) Primary complete/some secondary285 (31.0)138 (25.1)18 (4.0)441 (23.0) Secondary complete/some university213 (23.2)163 (29.6)303 (67.9)679 (35.5) University complete or higher22 (2.4)61 (11.1)74 (16.6)157 (8.2) Don't know/refused0 (0)0 (0)24 (5.4)24 (1.2)Religion Islam856 (93.1)126 (22.9)88 (19.7)1070 (55.9) Hinduism62 (6.8)0 (0)0 (0)62 (3.2) Christianity1 (0.1)408 (74.2)356 (79.8)765 (40.0) Another religion0 (0)4 (0.7)0 (0)4 (0.2) No religion0 (0)12 (2.2)2 (0.5)14 (0.7)Marital status Unmarried and not cohabitating4 (0.4)0 (0)258 (57.9)262 (13.7) Married/cohabitating909 (98.9)481 (87.4)167 (37.4)1557 (81.3) Formerly married6 (0.7)69 (12.6)21 (4.7)96 (5.0)Residence City113 (12.3)194 (35.3)224 (50.2)531 (27.7) Town273 (29.7)254 (46.2)156 (35.0)683 (35.7) Countryside/village533 (58.0)101 (18.3)63 (14.1)697 (36.4) Don't know/refused0 (0)1 (0.2)3 (0.7)4 (0.2)Paid work in past 12 months Yes211 (23.0)198 (36.0)234 (52.5)643 (33.6) No708 (77.0)350 (63.6)210 (47.1)1268 (66.2) Don't know/refused0 (0)2 (0.4)2 (0.4)4 (0.2)Abortion service type Induced743 (80.8)424 (77.1)427 (95.7)1594 (83.2) Postabortion care (PAC)176 (19.2)113 (20.5)19 (4.3)308 (16.1) Don't know/refused0 (0)13 (2.4)0 (0)13 (0.7)Gestational age <13 weeks817 (88.9)380 (69.1)432 (96.9)1629 (85.1) 13+ weeks86 (9.4)159 (28.9)2 (0.4)247 (12.9) Don't know/refused16 (1.7)11 (2.0)12 (2.7)39 (2.0)Procedure type Medication abortion (MA)584 (63.6)410 (74.6)446 (100)1440 (75.2) Surgical335 (36.4)132 (24.0)0 (0)467 (24.4) Don't know/refused0 (0)8 (1.4)0 (0)8 (0.4) Table 2Client sociodemographic and abortion characteristics (pilot test data).BangladeshEthiopiaNigeriaTotal(n = 919)(n = 550)(n = 446)(n = 1915)n (%)n (%)n (%)n (%)Recruitment site Facility481 (52.3)550 (100)0 (0)1031 (53.8) Pharmacy or proprietary and patent medicine vendor (PPMV)404 (44.0)0 (0)264 (59.2)668 (34.9) Hotline34 (3.7)0 (0)182 (40.8)216 (11.3)Age (years) 15–1841 (4.5)39 (7.1)24 (5.4)104 (5.4) 19–24242 (26.3)228 (41.5)150 (33.6)620 (32.4) 25–48636 (69.2)198 (36.0)271 (60.8)1105 (57.7) Don't know/refused0 (0)85 (15.4)1 (0.2)86 (4.5)Education None/some primary399 (43.4)188 (34.2)27 (6.1)614 (32.1) Primary complete/some secondary285 (31.0)138 (25.1)18 (4.0)441 (23.0) Secondary complete/some university213 (23.2)163 (29.6)303 (67.9)679 (35.5) University complete or higher22 (2.4)61 (11.1)74 (16.6)157 (8.2) Don't know/refused0 (0)0 (0)24 (5.4)24 (1.2)Religion Islam856 (93.1)126 (22.9)88 (19.7)1070 (55.9) Hinduism62 (6.8)0 (0)0 (0)62 (3.2) Christianity1 (0.1)408 (74.2)356 (79.8)765 (40.0) Another religion0 (0)4 (0.7)0 (0)4 (0.2) No religion0 (0)12 (2.2)2 (0.5)14 (0.7)Marital status Unmarried and not cohabitating4 (0.4)0 (0)258 (57.9)262 (13.7) Married/cohabitating909 (98.9)481 (87.4)167 (37.4)1557 (81.3) Formerly married6 (0.7)69 (12.6)21 (4.7)96 (5.0)Residence City113 (12.3)194 (35.3)224 (50.2)531 (27.7) Town273 (29.7)254 (46.2)156 (35.0)683 (35.7) Countryside/village533 (58.0)101 (18.3)63 (14.1)697 (36.4) Don't know/refused0 (0)1 (0.2)3 (0.7)4 (0.2)Paid work in past 12 months Yes211 (23.0)198 (36.0)234 (52.5)643 (33.6) No708 (77.0)350 (63.6)210 (47.1)1268 (66.2) Don't know/refused0 (0)2 (0.4)2 (0.4)4 (0.2)Abortion service type Induced743 (80.8)424 (77.1)427 (95.7)1594 (83.2) Postabortion care (PAC)176 (19.2)113 (20.5)19 (4.3)308 (16.1) Don't know/refused0 (0)13 (2.4)0 (0)13 (0.7)Gestational age <13 weeks817 (88.9)380 (69.1)432 (96.9)1629 (85.1) 13+ weeks86 (9.4)159 (28.9)2 (0.4)247 (12.9) Don't know/refused16 (1.7)11 (2.0)12 (2.7)39 (2.0)Procedure type Medication abortion (MA)584 (63.6)410 (74.6)446 (100)1440 (75.2) Surgical335 (36.4)132 (24.0)0 (0)467 (24.4) Don't know/refused0 (0)8 (1.4)0 (0)8 (0.4) Client sociodemographic and abortion characteristics (pilot test data). Table 3 presents the proportion of clients meeting each client-level outcome and the proportion of sites meeting each site-level outcome. Most outcomes had low variability; for six of the nine client-level outcomes, more than 95% of clients reported that their care met the outcome. The outcome indices had a normal distribution that was skewed right (higher quality). The client-level outcome index ranged from 36.6 to 100 with a mean of 86.5 (standard error = 0.25), and the site-level outcome index ranged from 55.7 to 100 with a mean of 84.3 (standard error = 0.82).Table 3Percentage of clients and sites meeting abortion service quality outcomes (pilot test data).n (%)aSite-level outcomes No abortion-related deaths in past 12 months106 (96.4) Abortion-related severe adverse events in past 12 months within expected range34 (29.6) No eligible clients turned away for abortion services94 (74.6)Client-level outcomes Client was treated with respect and kindness throughout the abortion process1869 (97.8) Client felt that they could cope with their pain1777 (92.9) Client felt they knew what to do if adverse events occurred1611 (84.2) Client was able to access follow-up or intervention for issues related to the abortion, if desired (facilities only)1869 (98.5) Client knew their abortion was complete or had a plan for what to do1894 (99.1) Client was able to access ancillary services or referrals, if desired707 (69.3) Client was no longer pregnant at 30 days1882 (98.3) Client did not experience abortion-related infection1827 (95.4) Client would recommend the service to a friend1816 (96.4)Outcome indicesMean (SE) Site-level outcome index84.3 (0.82) Client-level outcome index86.5 (0.25)aPercentages calculated among non-missing responses. Table 3Percentage of clients and sites meeting abortion service quality outcomes (pilot test data).n (%)aSite-level outcomes No abortion-related deaths in past 12 months106 (96.4) Abortion-related severe adverse events in past 12 months within expected range34 (29.6) No eligible clients turned away for abortion services94 (74.6)Client-level outcomes Client was treated with respect and kindness throughout the abortion process1869 (97.8) Client felt that they could cope with their pain1777 (92.9) Client felt they knew what to do if adverse events occurred1611 (84.2) Client was able to access follow-up or intervention for issues related to the abortion, if desired (facilities only)1869 (98.5) Client knew their abortion was complete or had a plan for what to do1894 (99.1) Client was able to access ancillary services or referrals, if desired707 (69.3) Client was no longer pregnant at 30 days1882 (98.3) Client did not experience abortion-related infection1827 (95.4) Client would recommend the service to a friend1816 (96.4)Outcome indicesMean (SE) Site-level outcome index84.3 (0.82) Client-level outcome index86.5 (0.25)aPercentages calculated among non-missing responses. Percentage of clients and sites meeting abortion service quality outcomes (pilot test data). Percentages calculated among non-missing responses. The Bayesian analysis results can be found in Appendix II. Overall, 44 of the 111 ASQ indicators were associated with at least one of the eleven outcome measures assessed (one outcome, “no abortion-related deaths in the past year” was not assessed as described in the Methods section) or an outcome index in the hypothesised direction. Among the 44 indicators associated in the hypothesised direction, 18 came from the 72 existing technical quality indicators and 26 came from the 39 novel, client-centred indicators developed through the ASQ Initiative.11,12 A total of 11 indicators were associated with at least one outcome or an outcome index in the opposite direction from what was expected. Two outcomes were not associated with any of the 111 ASQ indicators in the expected direction, including “Client was able to access follow-up or intervention for issues related to the abortion if desired” and “Client did not experience abortion-related infection”. Selection of indicators for feasibility testing Fig. 2 provides an overview of the indicator selection process, including the number of indicators eliminated and added at each stage. A total of 52 indicators were considered for feasibility testing, including 44 indicators that were associated with outcomes in the hypothesised direction and 8 indicators recommended for inclusion by the ASQ Resource Group for face validity. Table 4 presents the study team's evaluation of each of the 52 indicators on the criteria laid out in the Methods section. There was a high level of variation in both the site-level and client-level indicators with 9–91% of sites passing the site-level indicators and 34–99% of clients reporting that their care met the quality standard for the client-level indicators. Provider competence to complete the surgical abortion and MA provision was low (8.8% and 9.1% of sites passing, respectively). For surgical abortion this was primarily driven by low rates of paracervical block (providers administered paracervical block in only 45% of observations) and low rates of inspecting the tissue to ensure the abortion is complete (providers inspected tissue in 79% of observations). For MA provision, most sites failed because the providers did not routinely check for all contraindications of MA. Nineteen indicators were eliminated due to low content validity, and five indicators with high content validity were eliminated due to low predictive validity and poor performance. One indicator (continuous supply of surgical abortion equipment) was added because the analogous indicator for MA (continuous supply of medications for MA) had been included based on the criteria, and the study team felt that both indicators should be included for completeness. A total of 29 indicators were selected for feasibility testing.Fig. 2Indicator selection process.Table 4Evaluation of indicator validity (pilot test data).Indicatorn (%)aBasis for Possible InclusionStudy Team EvaluationDecisionAssociated with outcomesAt least 50% of Resource Group members recommendedContent validityPredictive validityPerformanceSite-level Indicators Emergency referral systems are in place, documented, and staff are aware62 (48.4)NoYesHighLowHighInclude The site has had a continuous supply of the necessary equipment and supplies over the past 3 months78 (63.9)YesN/AHighLowHighInclude Women seeking abortion have not been turned away due to stockouts of surgical abortion equipment over the past 3 months42 (91.3)YesN/ALowLowHighExclude Women seeking abortion have not been turned away due to stockouts of medication abortion (MA) over the past 3 months111 (91.0)YesN/ALowLowHighExclude The site has had continuous supply of the necessary medications for medication abortion over the past 3 months54 (54.0)NoYesHighLowHighInclude Site does not turn away women eligible for abortion86 (68.8)YesN/AHighHighHighInclude Sites use World Health Organization (WHO)-approved methods for uterine evacuation based on gestational age38 (84.4)NoYesHighLowHighInclude Clients report trusting their provider is knowledgeable and has the technical skills to provide their abortion service109 (87.2)NoYesHighLowLowExclude The provider inspects aspirated tissue for products of conception (POCs)27 (79.4)NoYesHighLowLowExclude The provider competently completes each step of the surgical abortion procedure3 (8.8)NoYesHighLowHighInclude The provider competently completes each step of MA provision3 (9.1)YesN/AHighLowHighInclude For the MA procedure, the mifepristone plus misoprostol or misoprostol alone are administered according to WHO protocol29 (87.9)NoYesLowLowHighExclude Providers and staff are not subject to targets, quotas, or incentives for contraception110 (89.4)YesN/AHighHighHighInclude Provider explains in detail the common symptoms and side effects of the procedure11 (34.4)NoYesHighLowLowExcludeClient-level Indicators Pain management is available to all clients, and not dependent on economic status311 (63.6)YesN/AHighLowHighInclude Clients felt they received quality medications from a reliable source, or knew where to obtain them1352 (94.0)YesN/AHighLowHighInclude Cost of services are appropriate and feasible for the client, addressing needs of different ages and income levels1361 (71.1)YesN/AHighLowHighInclude Services are reliably available to clients (open during posted hours with a provider present)1550 (81.0)YesN/AHighLowHighInclude Open/service times are convenient to client1781 (93.0)YesN/ALowLowHighExclude Clients felt that providers were attentive to their pain levels throughout service delivery.430 (87.8)YesN/ALowLowHighExclude Clients felt their pain was managed effectively1671 (87.3)YesN/AHighHighHighInclude Clients felt confident that the instruments were clean369 (79.0)YesN/ALowLowHighExclude Clients felt reassured by their provider of their safety throughout their abortion process/procedure1791 (93.5)YesN/AHighHighHighInclude Client was able to make an informed choice about contraception without coercion from their provider793 (82.3)YesN/ALowLowHighExclude Clients felt they had a choice among contraceptive method options1412 (74.6)YesN/ALowLowHighExclude Clients did not feel obligated to use a contraceptive method after their abortion1533 (80.1)YesN/AHighLowHighInclude Clients felt comfortable asking provider(s) questions about contraceptive methods1385 (73.6)YesN/ALowLowHighExclude Client received their desired contraceptive method or referral for desired method1451 (86.2)YesN/AHighHighLowInclude Personal circumstances raised by the client are taken into consideration by staff when discussing abortion options1210 (63.4)YesN/AHighHighHighInclude Client felt provider was supportive of their abortion decision and not coercive1343 (70.4)YesN/AHighHighHighInclude Clients felt supported by provider throughout their abortion process1807 (94.6)YesN/ALowLowHighExclude Time is offered for exploring the client's feelings regarding their decision to have an abortion, fears, and conflicts, including religious issues1039 (54.4)YesN/ALowLowHighExclude Client felt comfortable expressing their needs, questions, and fears if they wanted to1812 (94.8)YesN/AHighHighHighInclude Clients felt put at ease by their provider(s)1871 (97.9)YesN/ALowLowHighExclude Clients were able to make an informed choice about options for pain management1303 (70.8)YesN/AHighLowLowExclude Clients felt their preferences for type of abortion method were asked about and considered1646 (86.0)YesN/ALowHighHighExclude Client received and understood information about the safety of all available abortions methods1361 (71.2)YesN/ALowHighLowExclude Clients understand what to expect during each step of their visit/care provision552 (77.8)YesN/AHighHighHighInclude Client felt the provider took enough time to explain what was going to happen during their abortion process1448 (75.7)YesN/AHighHighHighInclude Client felt they received and understood clear, honest information about the abortion process1759 (92.7)YesN/ALowHighHighExclude Clients felt prepared for the side effects they would experience during their abortion1389 (72.5)YesN/ALowHighHighExclude Clients felt prepared for what to do if they experienced warning signs or in the event of complications1712 (89.4)YesN/AHighLowHighInclude Clients were confident they knew how to determine if their abortion was complete685 (71.7)YesN/AHighHighHighInclude Clients felt they had all of their questions answered1405 (73.6)YesN/AHighLowHighInclude Among women who want information about contraception: Provider gives comprehensive information on contraception637 (33.6)YesN/AHighLowLowExclude The provider checks the client's understanding of the information provided1500 (78.4)YesN/AHighHighHighInclude Clients felt their privacy was maintained in the waiting room666 (64.8)YesN/ALowLowHighExclude Clients felt comfortable sharing personal information with the counsellor/staff1813 (95.0)YesN/AHighHighHighInclude Clients trusted the providers to keep their personal information confidential1711 (89.5)YesN/AHighLowHighInclude Client was treated with respect at all times1891 (98.8)YesN/AHighLowHighInclude Clients felt the provider cared about them1753 (91.5)YesN/ALowHighHighExclude Clients were spoken to kindly by all staff/providers1896 (99.1)YesN/ALowLowHighExcludeaDenominator for percentages varies based on indicator applicability. Indicator selection process. Table 4Evaluation of indicator validity (pilot test data).Indicatorn (%)aBasis for Possible InclusionStudy Team EvaluationDecisionAssociated with outcomesAt least 50% of Resource Group members recommendedContent validityPredictive validityPerformanceSite-level Indicators Emergency referral systems are in place, documented, and staff are aware62 (48.4)NoYesHighLowHighInclude The site has had a continuous supply of the necessary equipment and supplies over the past 3 months78 (63.9)YesN/AHighLowHighInclude Women seeking abortion have not been turned away due to stockouts of surgical abortion equipment over the past 3 months42 (91.3)YesN/ALowLowHighExclude Women seeking abortion have not been turned away due to stockouts of medication abortion (MA) over the past 3 months111 (91.0)YesN/ALowLowHighExclude The site has had continuous supply of the necessary medications for medication abortion over the past 3 months54 (54.0)NoYesHighLowHighInclude Site does not turn away women eligible for abortion86 (68.8)YesN/AHighHighHighInclude Sites use World Health Organization (WHO)-approved methods for uterine evacuation based on gestational age38 (84.4)NoYesHighLowHighInclude Clients report trusting their provider is knowledgeable and has the technical skills to provide their abortion service109 (87.2)NoYesHighLowLowExclude The provider inspects aspirated tissue for products of conception (POCs)27 (79.4)NoYesHighLowLowExclude The provider competently completes each step of the surgical abortion procedure3 (8.8)NoYesHighLowHighInclude The provider competently completes each step of MA provision3 (9.1)YesN/AHighLowHighInclude For the MA procedure, the mifepristone plus misoprostol or misoprostol alone are administered according to WHO protocol29 (87.9)NoYesLowLowHighExclude Providers and staff are not subject to targets, quotas, or incentives for contraception110 (89.4)YesN/AHighHighHighInclude Provider explains in detail the common symptoms and side effects of the procedure11 (34.4)NoYesHighLowLowExcludeClient-level Indicators Pain management is available to all clients, and not dependent on economic status311 (63.6)YesN/AHighLowHighInclude Clients felt they received quality medications from a reliable source, or knew where to obtain them1352 (94.0)YesN/AHighLowHighInclude Cost of services are appropriate and feasible for the client, addressing needs of different ages and income levels1361 (71.1)YesN/AHighLowHighInclude Services are reliably available to clients (open during posted hours with a provider present)1550 (81.0)YesN/AHighLowHighInclude Open/service times are convenient to client1781 (93.0)YesN/ALowLowHighExclude Clients felt that providers were attentive to their pain levels throughout service delivery.430 (87.8)YesN/ALowLowHighExclude Clients felt their pain was managed effectively1671 (87.3)YesN/AHighHighHighInclude Clients felt confident that the instruments were clean369 (79.0)YesN/ALowLowHighExclude Clients felt reassured by their provider of their safety throughout their abortion process/procedure1791 (93.5)YesN/AHighHighHighInclude Client was able to make an informed choice about contraception without coercion from their provider793 (82.3)YesN/ALowLowHighExclude Clients felt they had a choice among contraceptive method options1412 (74.6)YesN/ALowLowHighExclude Clients did not feel obligated to use a contraceptive method after their abortion1533 (80.1)YesN/AHighLowHighInclude Clients felt comfortable asking provider(s) questions about contraceptive methods1385 (73.6)YesN/ALowLowHighExclude Client received their desired contraceptive method or referral for desired method1451 (86.2)YesN/AHighHighLowInclude Personal circumstances raised by the client are taken into consideration by staff when discussing abortion options1210 (63.4)YesN/AHighHighHighInclude Client felt provider was supportive of their abortion decision and not coercive1343 (70.4)YesN/AHighHighHighInclude Clients felt supported by provider throughout their abortion process1807 (94.6)YesN/ALowLowHighExclude Time is offered for exploring the client's feelings regarding their decision to have an abortion, fears, and conflicts, including religious issues1039 (54.4)YesN/ALowLowHighExclude Client felt comfortable expressing their needs, questions, and fears if they wanted to1812 (94.8)YesN/AHighHighHighInclude Clients felt put at ease by their provider(s)1871 (97.9)YesN/ALowLowHighExclude Clients were able to make an informed choice about options for pain management1303 (70.8)YesN/AHighLowLowExclude Clients felt their preferences for type of abortion method were asked about and considered1646 (86.0)YesN/ALowHighHighExclude Client received and understood information about the safety of all available abortions methods1361 (71.2)YesN/ALowHighLowExclude Clients understand what to expect during each step of their visit/care provision552 (77.8)YesN/AHighHighHighInclude Client felt the provider took enough time to explain what was going to happen during their abortion process1448 (75.7)YesN/AHighHighHighInclude Client felt they received and understood clear, honest information about the abortion process1759 (92.7)YesN/ALowHighHighExclude Clients felt prepared for the side effects they would experience during their abortion1389 (72.5)YesN/ALowHighHighExclude Clients felt prepared for what to do if they experienced warning signs or in the event of complications1712 (89.4)YesN/AHighLowHighInclude Clients were confident they knew how to determine if their abortion was complete685 (71.7)YesN/AHighHighHighInclude Clients felt they had all of their questions answered1405 (73.6)YesN/AHighLowHighInclude Among women who want information about contraception: Provider gives comprehensive information on contraception637 (33.6)YesN/AHighLowLowExclude The provider checks the client's understanding of the information provided1500 (78.4)YesN/AHighHighHighInclude Clients felt their privacy was maintained in the waiting room666 (64.8)YesN/ALowLowHighExclude Clients felt comfortable sharing personal information with the counsellor/staff1813 (95.0)YesN/AHighHighHighInclude Clients trusted the providers to keep their personal information confidential1711 (89.5)YesN/AHighLowHighInclude Client was treated with respect at all times1891 (98.8)YesN/AHighLowHighInclude Clients felt the provider cared about them1753 (91.5)YesN/ALowHighHighExclude Clients were spoken to kindly by all staff/providers1896 (99.1)YesN/ALowLowHighExcludeaDenominator for percentages varies based on indicator applicability. Evaluation of indicator validity (pilot test data). Denominator for percentages varies based on indicator applicability. Feasibility testing results A total of 53 clients and 24 providers were recruited from 9 facility sites in Ethiopia, and 57 clients and 6 PPMVs were recruited from 9 PPMV sites in Nigeria. The median time required to complete each survey instrument indicated feasibility of using these tools in a variety of settings: the client exit survey required 10.6 min, the provider survey required 16.5 min, and the site checklist required 11.1 min. Overall, the questions on the survey instruments performed well with very few “Refused” and “Don't Know” responses. The results for the 29 indicators can be found in Table 5. Overall, there was a fairly high level of variation in indicators, and they were considered to have performed well. However, none of the sites participating in the feasibility test passed indicators #12 or #13. These indicators assessed provider knowledge to competently complete the abortion procedure and were measured on the provider survey during the feasibility test rather than the provider observation tool that was used in the pilot test. Based on these findings, the clinical sub-group of the Resource Group adjusted the construction and wording of these questions to simplify and clarify their intent while maintaining the high bar for quality. The revised questions that make up indicators #12 and #13 have not yet been tested.Table 5Percentage of sites passing 29 abortion service quality indicators (feasibility test data).n (%)a1. Emergency referral systems are in place, documented, and staff are aware15 (100)2. The site has had a continuous supply of the necessary equipment and supplies over the past 3 months6 (42.9)3. The site has had continuous supply of the necessary medications for medication abortion over the past 3 months5 (38.5)4. Clients felt they received quality medications from a reliable source, or knew where to obtain them16 (100)5. The site has had continuous supply of the necessary surgical abortion equipment over the past 3 months3 (33.3)6. Pain management is available to all clients, and not dependent on economic status4 (44.4)7. Cost of services are appropriate and feasible for the client, addressing needs of different ages and income levels12 (66.7)8. Services are reliably available to clients (open during posted hours with a provider present)5 (33.3)9. Site does not refuse abortion services to eligible clients2 (13.3)10. Clients felt their pain was managed effectively13 (72.2)11. Sites use World Health Organization (WHO)-approved methods for uterine evacuation based on gestational age8 (88.9)12. The provider has adequate knowledge to competently complete each step of the surgical abortion procedure0 (0)13. The provider has adequate knowledge to competently complete each step of medication abortion (MA) provision0 (0)14. Clients felt reassured by their provider of their safety throughout their abortion process/procedure16 (88.9)15. Clients did not feel obligated to use a contraceptive method after their abortion10 (55.6)16. Client received their desired contraceptive method or referral for desired method4 (22.2)17. Providers and staff are not subject to targets, quotas, or incentives for contraception7 (46.7)18. Personal circumstances raised by the client are taken into consideration by staff when discussing abortion options7 (38.9)19. Client felt provider was supportive of their abortion decision and not coercive7 (38.9)20. Client felt comfortable expressing their needs, questions, and fears if they wanted to16 (88.9)21. Clients understand what to expect during each step of their visit/care provision4 (44.4)22. Client felt the provider took enough time to explain what was going to happen during their abortion process9 (50.0)23. Clients felt prepared for what to do if they experienced warning signs or in the event of complications13 (72.2)24. Clients were confident they knew how to determine if their abortion was complete7 (63.6)25. Clients felt they had all of their questions answered7 (46.7)26. The provider checks the client's understanding of the information provided14 (77.8)27. Clients felt comfortable sharing personal information with the counsellor/staff15 (83.3)28. Clients trusted the providers to keep their personal information confidential16 (88.9)29. Client was treated with respect at all times17 (94.4)aDenominator for percentages varies based on indicator applicability and calculated among non-missing responses. Table 5Percentage of sites passing 29 abortion service quality indicators (feasibility test data).n (%)a1. Emergency referral systems are in place, documented, and staff are aware15 (100)2. The site has had a continuous supply of the necessary equipment and supplies over the past 3 months6 (42.9)3. The site has had continuous supply of the necessary medications for medication abortion over the past 3 months5 (38.5)4. Clients felt they received quality medications from a reliable source, or knew where to obtain them16 (100)5. The site has had continuous supply of the necessary surgical abortion equipment over the past 3 months3 (33.3)6. Pain management is available to all clients, and not dependent on economic status4 (44.4)7. Cost of services are appropriate and feasible for the client, addressing needs of different ages and income levels12 (66.7)8. Services are reliably available to clients (open during posted hours with a provider present)5 (33.3)9. Site does not refuse abortion services to eligible clients2 (13.3)10. Clients felt their pain was managed effectively13 (72.2)11. Sites use World Health Organization (WHO)-approved methods for uterine evacuation based on gestational age8 (88.9)12. The provider has adequate knowledge to competently complete each step of the surgical abortion procedure0 (0)13. The provider has adequate knowledge to competently complete each step of medication abortion (MA) provision0 (0)14. Clients felt reassured by their provider of their safety throughout their abortion process/procedure16 (88.9)15. Clients did not feel obligated to use a contraceptive method after their abortion10 (55.6)16. Client received their desired contraceptive method or referral for desired method4 (22.2)17. Providers and staff are not subject to targets, quotas, or incentives for contraception7 (46.7)18. Personal circumstances raised by the client are taken into consideration by staff when discussing abortion options7 (38.9)19. Client felt provider was supportive of their abortion decision and not coercive7 (38.9)20. Client felt comfortable expressing their needs, questions, and fears if they wanted to16 (88.9)21. Clients understand what to expect during each step of their visit/care provision4 (44.4)22. Client felt the provider took enough time to explain what was going to happen during their abortion process9 (50.0)23. Clients felt prepared for what to do if they experienced warning signs or in the event of complications13 (72.2)24. Clients were confident they knew how to determine if their abortion was complete7 (63.6)25. Clients felt they had all of their questions answered7 (46.7)26. The provider checks the client's understanding of the information provided14 (77.8)27. Clients felt comfortable sharing personal information with the counsellor/staff15 (83.3)28. Clients trusted the providers to keep their personal information confidential16 (88.9)29. Client was treated with respect at all times17 (94.4)aDenominator for percentages varies based on indicator applicability and calculated among non-missing responses. Percentage of sites passing 29 abortion service quality indicators (feasibility test data). Denominator for percentages varies based on indicator applicability and calculated among non-missing responses. Final indicator list The final list of indicators, termed the Abortion Care Quality (ACQ) Tool, can be found in Table 6, which includes the threshold set for high quality abortion care at the site level, applicability to abortion care settings (facilities, pharmacies, hotlines), the origin of the indicator, and justification for inclusion. More detailed information, including data collection instruments and resources for collecting and analysing the indicators can be found at www.acqtool.org.Table 6Abortion Care Quality (ACQ) tool indicators.DomainIndicatorSite Quality ThresholdApplicabilityOriginJustification for InclusionReferral Systems1. Emergency referral systems80% of providers report that emergency referral systems are in place and documented and have accurate knowledge of the systemFacility, Pharmacy, HotlineExisting technical quality indicatorsRecommended by Resource GroupSupplies, Medicines, & Equipment2. Equipment and supply availabilityAll equipment and supplies necessary for abortion services continuously available for past 3 monthsFacility, PharmacyExisting technical quality indicatorsAssociated with•Severe adverse events in expected range3. Medication availabilityAll necessary medications for medical abortion continuously available for past 3 monthsFacility, Pharmacy (offering MA)Existing technical quality indicatorsRecommended by Resource Group4. Client perception of medication quality90% of MA clients reported confidence in medication qualityFacility, Pharmacy, Hotline (offering MA)Novel client-centred indicatorsAssociated with•Site-level outcome index5. Surgical abortion supply and availabilityAll necessary equipment and supplies for surgical abortion continuously available for past 3 monthsFacility (offering surgical abortion)Existing technical quality indicatorsCompletenessAccess6. Equitable access to pain management100% of clients reported pain management was available regardless of economic statusFacilityNovel client-centred indicatorsAssociated with•Client-level outcome index7. Affordability100% of clients reported services were affordableFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client would recommend to a friend8. Hours of operation90% of clients reported services reliably available during posted hoursFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client-level outcome index9. Service refusal100% of providers report they have not refused abortion services to eligible clientsFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client-level outcome index•Site-level outcome index•Site does not turn away women eligible for abortionTechnical Competence10. Pain management90% of clients reported their pain was managed effectivelyFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client-level outcome index•Site-level outcome index•Client felt that they could cope with their pain11. Appropriate uterine evacuation method100% of providers using WHO-approved uterine evacuation methods based on gestational ageFacilityExisting technical quality indicatorsRecommended by Resource Group12. Surgical abortion procedure competence80% of providers have adequate knowledge to competently provide surgical abortion proceduresFacility (offering surgical abortion)Existing technical quality indicatorsRecommended by Resource Group13. Medical abortion procedure competence80% of providers have adequate knowledge to competently provide medical abortion servicesFacility, Pharmacy, Hotline (offering MA)Existing technical quality indicatorsAssociated with•Client knew their abortion was complete or had a plan for what to do14. Client perception of safety90% of clients reported feeling reassured of their safety throughout their abortion procedure/processFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Site-level outcome index•Client felt treated with respectDecision-Making15. Contraceptive method coercion90% of clients reported not feeling obligated to use a contraceptive method after abortionFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt that ancillary services were available if desired16. Desired contraceptive received100% of clients reported receiving their desired contraceptive method or a referral for their desired methodFacility, PharmacyExisting technical quality indicatorsAssociated with•Client felt that ancillary services were available if desired•Client would recommend to a friend17. Provider contraceptive quotas80% of providers reported not being subject to targets, quotas, or incentives for contraceptionFacility, PharmacyExisting technical quality indicatorsAssociated with•Client-level outcome index•Site-level outcome index•Client would recommend to a friend18. Personalised care options100% of clients reported that personal circumstances they raised were taken into consideration when discussing abortion optionsFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client-level outcome index•Client felt treated with respect19. Provider support for client decision90% of clients reported that the provider was supportive of their abortion decision and not coerciveFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client-level outcome index•Client felt treated with respect•Client felt they could cope with pain•Client felt that ancillary services were available if desired•Client would recommend to a friendInformation Provision20. Client communication comfort90% of clients reported feeling comfortable expressing their needs, questions, and fears if they wanted toFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client-level outcome index•Client felt treated with respect21. Client understands abortion process100% of clients reported that they understood what to expect during each step of their visit/callFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt treated with respect•Client would recommend to a friend22. Sufficient provider explanation90% of clients reported feeling that the provider took enough time to explain what was going to happen during their abortion processFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt treated with respect•Client felt they knew what to do if an adverse event occurred23. Prepared if complication occurs90% of clients reported feeling prepared for what to do if they experienced warning signs or in the event of complicationsFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt they knew what to do if an adverse event occurred24. Client can determine complete abortion90% of clients reported feeling confident they knew how to determine if their abortion was completeFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt confident their abortion was complete or had a plan for what to do•Client was no longer pregnant at 30 days25. All questions answered90% of clients reported having all of their questions answeredFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt they knew what to do if an adverse event occurred26. Client understanding checked100% of clients reported that the provider checked their understanding of the information providedFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client felt treated with respect•Client felt they could cope with their pain•Client felt they knew what to do if an adverse event occurred•Client would recommend to a friendClient–Provider Interaction27. Comfortable sharing information90% of clients reported feeling comfortable sharing personal information with the counsellor/staffFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client-level outcome index•Client felt treated with respect28. Confidentiality90% of clients reported trusting the providers to keep their personal information confidentialFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt treated with respect29. Respect90% of clients reported feeling that they were treated with respect at all timesFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client-level outcome index Table 6Abortion Care Quality (ACQ) tool indicators.DomainIndicatorSite Quality ThresholdApplicabilityOriginJustification for InclusionReferral Systems1. Emergency referral systems80% of providers report that emergency referral systems are in place and documented and have accurate knowledge of the systemFacility, Pharmacy, HotlineExisting technical quality indicatorsRecommended by Resource GroupSupplies, Medicines, & Equipment2. Equipment and supply availabilityAll equipment and supplies necessary for abortion services continuously available for past 3 monthsFacility, PharmacyExisting technical quality indicatorsAssociated with•Severe adverse events in expected range3. Medication availabilityAll necessary medications for medical abortion continuously available for past 3 monthsFacility, Pharmacy (offering MA)Existing technical quality indicatorsRecommended by Resource Group4. Client perception of medication quality90% of MA clients reported confidence in medication qualityFacility, Pharmacy, Hotline (offering MA)Novel client-centred indicatorsAssociated with•Site-level outcome index5. Surgical abortion supply and availabilityAll necessary equipment and supplies for surgical abortion continuously available for past 3 monthsFacility (offering surgical abortion)Existing technical quality indicatorsCompletenessAccess6. Equitable access to pain management100% of clients reported pain management was available regardless of economic statusFacilityNovel client-centred indicatorsAssociated with•Client-level outcome index7. Affordability100% of clients reported services were affordableFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client would recommend to a friend8. Hours of operation90% of clients reported services reliably available during posted hoursFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client-level outcome index9. Service refusal100% of providers report they have not refused abortion services to eligible clientsFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client-level outcome index•Site-level outcome index•Site does not turn away women eligible for abortionTechnical Competence10. Pain management90% of clients reported their pain was managed effectivelyFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client-level outcome index•Site-level outcome index•Client felt that they could cope with their pain11. Appropriate uterine evacuation method100% of providers using WHO-approved uterine evacuation methods based on gestational ageFacilityExisting technical quality indicatorsRecommended by Resource Group12. Surgical abortion procedure competence80% of providers have adequate knowledge to competently provide surgical abortion proceduresFacility (offering surgical abortion)Existing technical quality indicatorsRecommended by Resource Group13. Medical abortion procedure competence80% of providers have adequate knowledge to competently provide medical abortion servicesFacility, Pharmacy, Hotline (offering MA)Existing technical quality indicatorsAssociated with•Client knew their abortion was complete or had a plan for what to do14. Client perception of safety90% of clients reported feeling reassured of their safety throughout their abortion procedure/processFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Site-level outcome index•Client felt treated with respectDecision-Making15. Contraceptive method coercion90% of clients reported not feeling obligated to use a contraceptive method after abortionFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt that ancillary services were available if desired16. Desired contraceptive received100% of clients reported receiving their desired contraceptive method or a referral for their desired methodFacility, PharmacyExisting technical quality indicatorsAssociated with•Client felt that ancillary services were available if desired•Client would recommend to a friend17. Provider contraceptive quotas80% of providers reported not being subject to targets, quotas, or incentives for contraceptionFacility, PharmacyExisting technical quality indicatorsAssociated with•Client-level outcome index•Site-level outcome index•Client would recommend to a friend18. Personalised care options100% of clients reported that personal circumstances they raised were taken into consideration when discussing abortion optionsFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client-level outcome index•Client felt treated with respect19. Provider support for client decision90% of clients reported that the provider was supportive of their abortion decision and not coerciveFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client-level outcome index•Client felt treated with respect•Client felt they could cope with pain•Client felt that ancillary services were available if desired•Client would recommend to a friendInformation Provision20. Client communication comfort90% of clients reported feeling comfortable expressing their needs, questions, and fears if they wanted toFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client-level outcome index•Client felt treated with respect21. Client understands abortion process100% of clients reported that they understood what to expect during each step of their visit/callFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt treated with respect•Client would recommend to a friend22. Sufficient provider explanation90% of clients reported feeling that the provider took enough time to explain what was going to happen during their abortion processFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt treated with respect•Client felt they knew what to do if an adverse event occurred23. Prepared if complication occurs90% of clients reported feeling prepared for what to do if they experienced warning signs or in the event of complicationsFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt they knew what to do if an adverse event occurred24. Client can determine complete abortion90% of clients reported feeling confident they knew how to determine if their abortion was completeFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt confident their abortion was complete or had a plan for what to do•Client was no longer pregnant at 30 days25. All questions answered90% of clients reported having all of their questions answeredFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt they knew what to do if an adverse event occurred26. Client understanding checked100% of clients reported that the provider checked their understanding of the information providedFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client felt treated with respect•Client felt they could cope with their pain•Client felt they knew what to do if an adverse event occurred•Client would recommend to a friendClient–Provider Interaction27. Comfortable sharing information90% of clients reported feeling comfortable sharing personal information with the counsellor/staffFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client-level outcome index•Client felt treated with respect28. Confidentiality90% of clients reported trusting the providers to keep their personal information confidentialFacility, Pharmacy, HotlineNovel client-centred indicatorsAssociated with•Client felt treated with respect29. Respect90% of clients reported feeling that they were treated with respect at all timesFacility, Pharmacy, HotlineExisting technical quality indicatorsAssociated with•Client-level outcome index Abortion Care Quality (ACQ) tool indicators. •Severe adverse events in expected range Severe adverse events in expected range •Site-level outcome index Site-level outcome index •Client-level outcome index Client-level outcome index •Client would recommend to a friend Client would recommend to a friend •Client-level outcome index Client-level outcome index •Client-level outcome index Client-level outcome index •Site-level outcome index Site-level outcome index •Site does not turn away women eligible for abortion Site does not turn away women eligible for abortion •Client-level outcome index Client-level outcome index •Site-level outcome index Site-level outcome index •Client felt that they could cope with their pain Client felt that they could cope with their pain •Client knew their abortion was complete or had a plan for what to do Client knew their abortion was complete or had a plan for what to do •Site-level outcome index Site-level outcome index •Client felt treated with respect Client felt treated with respect •Client felt that ancillary services were available if desired Client felt that ancillary services were available if desired •Client felt that ancillary services were available if desired Client felt that ancillary services were available if desired •Client would recommend to a friend Client would recommend to a friend •Client-level outcome index Client-level outcome index •Site-level outcome index Site-level outcome index •Client would recommend to a friend Client would recommend to a friend •Client-level outcome index Client-level outcome index •Client felt treated with respect Client felt treated with respect •Client-level outcome index Client-level outcome index •Client felt treated with respect Client felt treated with respect •Client felt they could cope with pain Client felt they could cope with pain •Client felt that ancillary services were available if desired Client felt that ancillary services were available if desired •Client would recommend to a friend Client would recommend to a friend •Client-level outcome index Client-level outcome index •Client felt treated with respect Client felt treated with respect •Client felt treated with respect Client felt treated with respect •Client would recommend to a friend Client would recommend to a friend •Client felt treated with respect Client felt treated with respect •Client felt they knew what to do if an adverse event occurred Client felt they knew what to do if an adverse event occurred •Client felt they knew what to do if an adverse event occurred Client felt they knew what to do if an adverse event occurred •Client felt confident their abortion was complete or had a plan for what to do Client felt confident their abortion was complete or had a plan for what to do •Client was no longer pregnant at 30 days Client was no longer pregnant at 30 days •Client felt they knew what to do if an adverse event occurred Client felt they knew what to do if an adverse event occurred •Client felt treated with respect Client felt treated with respect •Client felt they could cope with their pain Client felt they could cope with their pain •Client felt they knew what to do if an adverse event occurred Client felt they knew what to do if an adverse event occurred •Client would recommend to a friend Client would recommend to a friend •Client-level outcome index Client-level outcome index •Client felt treated with respect Client felt treated with respect •Client felt treated with respect Client felt treated with respect •Client-level outcome index Client-level outcome index Discussion This study provides the first validated indicators to assess abortion care quality in facility and out-of-facility settings in low- and middle-income countries. To our knowledge, no standard for measuring abortion quality exists in out-of-facility settings, and measurement of in-facility quality does not have a unifying standard, nor have existing quality indicators been validated against health outcomes of interest. This study identified 29 indicators, valid across diverse abortion care settings in three countries in Asia and Africa, which are feasible to collect routinely, are indicative of abortion care quality, and are relevant for various stakeholder constituencies. The indicator selection process began with 111 indicators of quality identified by the ASQ Resource Group as theoretically associated with quality to be assessed against 12 client- and site-level outcomes. Dennis, Blanchard and Bessenaar (2016)5 identified 75 abortion quality indicators based on a systematic literature review, and this study extended that work by extracting quality indicators utilised by abortion service delivery organisations and Ministries of Health and developing novel client-centred indicators through qualitative formative work to identify 111 indicators for validation. The pilot test demonstrated that approximately half of the indicators were not associated with any of the selected outcomes, and 11 of the 111 indicators were associated with an outcome in the direction opposite from that hypothesised. Among those not associated were 16 of the 18 indicators of technical skills for in-facility procedures, often assessed via direct observation, which is resource-intensive to conduct. In contrast, 34 of the 44 indicators associated with an outcome in the hypothesised direction represented the client-centred domains of decision-making, information provision, and client and provider interactions. There was no statistical association with selected outcomes for indicators measuring the following domains from the ASQ Framework11: Infrastructure; Referral systems; Health information systems; and Continuum of care and service integration. Despite lack of statistical association, an indicator on emergency referral systems was reintroduced by the Resource Group to improve face validity of the tool. Twelve outcomes were assessed, including both person-centred and clinical outcomes, ensuring that indicators of quality were not solely centred on morbidity and mortality. Several outcomes had limited variability in this sample, and future studies of quality could consider focusing on outcomes with greater variability. Despite low variability, three client-level outcomes were associated with five or more indicators of high-quality care, including, “Client would recommend the service to a friend”, “Client was treated with respect and kindness throughout the abortion process”, and “Client felt they knew what to do if an adverse event occurred”. The two site-level outcomes that were analysed, “Abortion-related severe adverse events in past 12 months within expected range” and “No eligible clients turned away for abortion services” had greater variation compared to client-level outcomes and predicted high quality care. The ASQ Initiative sought to create a metric that was actionable by actors at various levels, including individual abortion care providers, site administrators, NGOs offering abortion care, and Ministries of Health, and included indicators across quality domains that experts deemed critical to assessing abortion care quality at the site level. Though there are many approaches to defining indicators (e.g., a Likert scale defining each indicator as high, medium, or low quality), all ASQ indicators were binary, defined as meeting the standard for high quality abortion care or not meeting the standard, which reflected the ASQ Initiative's objective to focus on “high quality” care rather than a more general assessment of “quality” or “minimum standards” for abortion care. In addition, individual indicators were identified rather than a composite or scaled measure due to the potential to mask different aspects of quality that would prevent the metric from being actionable.24 The ASQ Initiative had a strong focus on assessing both technical quality and client-centred quality of care indicators that would apply to various procedure types and across abortion care sites (facilities, pharmacies, and hotlines),25 and use of individual indicators rather than a composite measure recognises that some sites might perform well on indicators related to one domain but not others. In addition, the goal was to enable abortion care providers or those supervising at the site or higher levels to focus their quality improvement efforts on areas of greatest need. The provider competency indicators had the lowest rate of passing among all of the indicators. Over 90% of sites failed on provider competency to complete the surgical abortion procedure and to provide MA (indicators in pilot test were measured through provider observations) and all sites failed on provider knowledge to competently complete surgical abortion and MA provision (indicators in feasibility test were measured through provider surveys). The ASQ team intentionally set a high bar for quality for all indicators, and these two indicators required that providers complete all recommended steps of the abortion procedure/MA provision for the site to pass. For MA provision, most sites failed during the pilot test (measured via provider observations) due to failure to fully assess contraindications for MA, especially rarer contraindications such as haemorrhagic disorders and allergy to MA drugs. For surgical abortion, most sites failed on the pilot test due to missing one step in the abortion care process: administering paracervical block (PCB) prior to placing the tenaculum and inserting the cannula, which is recommended by WHO for management of abortion pain.26 During the feasibility test, sites failed the provider knowledge indicators (measured via the provider survey) for a wider variety of reasons. Several of the questions were multiple-select, and we found that providers often chose only the first correct answer on the survey without carefully considering all of the answer options. The clinical sub-group of the ASQ Resource Group worked to improve clarity of the questions in the final version of the provider survey, including changing questions that were multiple-select to have a single correct answer. Despite no sites passing the provider knowledge indicators in the small feasibility test, the clinical sub-group determined that the content of the questions should remain the same and that providers should still be required to answer all questions correctly to maintain a high bar for quality. This study has some key strengths and limitations. This study used a novel, stakeholder-centred approach to select indicators for pilot testing and analysed quality and outcome data from over 1900 abortion clients recruited from 131 sites in three countries in Asia and Africa. It was further informed by qualitative data in Argentina, but no Latin American sites were a part of the quantitative research. The indicators were tested in countries where abortion is widely available in the public sector (Ethiopia and Bangladesh) and in one more restrictive legal setting where MA is still widely available (Nigeria), but the indicators have not been tested in very restrictive settings where abortion is less available and/or criminalised, and validity of the indicators in these contexts is unknown. Three types of care models were included (facilities, pharmacies/PPMVs, and hotlines), but other sites offering abortion care such as online MA sellers or traditional abortion providers were not included. In addition, existing sampling frames were used that were not representative of all sites providing abortion care in the selected regions of each country, and many of the sites selected for this study were already participating in an abortion quality improvement intervention. As a result, it is likely that quality is higher among sites participating in the study than the average abortion site. These limitations may impact the generalisability of the overall metric. All client-level outcomes, including clinical abortion outcomes, were measured based on client self-report rather than provider report or medical record review. Use of client-reported clinical abortion outcomes is in alignment with similar studies,27,28 but this approach may introduce bias, particularly if some subgroups of abortion clients are more likely to report certain outcomes. This study used a rigorous approach to validating the final abortion quality metric, but also sought to be stakeholder-centred to ensure uptake and use of the final abortion quality metric, which meant that individuals participating the ASQ Resource Group had an influence on the final metric. The ASQ Initiative included a broad group of stakeholders in the Resource Group, but some members participated more than others (e.g., participation in surveys and working groups) and may have had a greater impact on the final metric compared to others. Finally, the feasibility test changed the measurement approach for indicators of providers’ technical skills, to assess knowledge based on clinical vignettes as a proxy for technical competence, which was measured via provider observation during the pilot test. This allowed for easier implementation and was a less resource-intensive data collection approach. However, the clinical vignettes were revised after the feasibility test to improve clarity, and the final versions have not yet been tested. The ASQ Initiative used a stakeholder-first approach to rigorously develop and test a global standard for measuring the quality of abortion care in facility and out-of-facility settings in LMICs. The resulting 29 indicators of the Abortion Care Quality Tool (acqtool.org) comprise the first tool to comprehensively measure structural, technical, and client-centred aspects of abortion care with expert-derived benchmarks for high quality. Future research should validate the ACQ Tool in a wider variety of settings, including other models of abortion care, such as telemedicine, online MA sellers, and traditional abortion providers, and in other geographical and legal settings. Contributors All authors contributed to the conception, design of the study, and interpretation of the analyses. RC, BMC, YA, and CB managed data collection. RC, BMC, YA, CB, JLM, and SD managed and cleaned the data. EP, JLM, and SD performed the analysis and prepared the tables and figures with input from NMC, SEB, BP, and CG. EP, NMC, JLM, SD, RC, BMC, YA, and CB have accessed and verified the data. EP was responsible for the decision to submit the manuscript and wrote the first draft of the manuscript. All authors contributed to, reviewed, and approved the final version of the manuscript. All authors had full access to the data and take responsibility for the integrity of the data and accuracy of the analysis. Data sharing statement Deidentified data that support the findings of this study are available on reasonable request. The data are not publicly available due to privacy concerns for abortion clients, providers, and sites. Declaration of interests SD and BP's time on this work was supported by a grant to Ipas from Global Affairs Canada, which provided funding for integration of the ACQ Tool indicators into Ipas's global monitoring and evaluation system. All other authors declare no competing interests.