Patient safety measures for virtual consultations in primary care: a systematic review Authors: Tetiana Lunova, Katherine-Helen Hurndall, Ulrik Bak Kirk, Bryony Dean Franklin, Ara Darzi, Ana Luisa Neves Published: 2025 ABSTRACT AbstractObjectivesWith the growing adoption of virtual consultations in primary care, the need for tailored metrics to evaluate their safety became increasingly urgent. This systematic review seeks to identify and review existing safety measures that could be used for safety evaluation of virtual consultations in primary care.MethodsThis has been conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and followed a published protocol. A systematic literature search was performed in Ovid MEDLINE/PubMed, Embase and Cochrane Library databases from 2014 to 2024. Studies comparing virtual consultations with face-to-face consultations in the primary care setting were included. An inductive thematic analysis was performed to systematically extract and group the safety measures into overarching themes, with a narrative synthesis to summarise the results.ResultsA total of 47 studies (31 experimental and 16 observational studies) were included (n=2 223 697 patients). All studies assessed the safety of virtual versus face-to-face consultations via one or both of the following domains: (1) factors that influence the safety of virtual consultations and (2) tangible outcomes of virtual care safety. The former were categorised into provider-related, patient-related and system-related factors. Tangible outcomes were evident through three subthemes—adverse events, health outcomes and patient perception of safety.ConclusionsThis review provides a systematic synthesis of measures for the safety evaluation of virtual consultations. Further research into patient and physician perspectives is needed to identify aspects and indicators not captured in this study, followed by a consensus study to finalise safety metrics. Ultimately, having a robust methodology for safety evaluation of virtual consultations in place will enable safety monitoring, root cause analyses and safety improvement.PROSPERO registration numberPROSPERO CRD42023464878.Keywords: Patient Safety, Primary care, Health services research WHAT IS ALREADY KNOWN ON THIS TOPIC Despite the growing use of virtual consultations in primary care, there is no standardised methodology to evaluate their safety, which represents a significant gap in evidence. Despite the growing use of virtual consultations in primary care, there is no standardised methodology to evaluate their safety, which represents a significant gap in evidence. WHAT THIS STUDY ADDS This review is a fundamental first step in developing a set of patient safety measures for virtual consultations. Having reviewed 47 publications that evaluated the safety of virtual versus face-to-face consultations, we identified the key aspects of virtual consultations that have been assessed for safety and extracted the corresponding safety measures. This review is a fundamental first step in developing a set of patient safety measures for virtual consultations. Having reviewed 47 publications that evaluated the safety of virtual versus face-to-face consultations, we identified the key aspects of virtual consultations that have been assessed for safety and extracted the corresponding safety measures. HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY This study identified safety aspects that may need to be assessed in virtual consultations and presented the corresponding safety measures. It also recommends next steps for research to produce a final set of measures and highlights the need for robust data infrastructure in primary care. This study identified safety aspects that may need to be assessed in virtual consultations and presented the corresponding safety measures. It also recommends next steps for research to produce a final set of measures and highlights the need for robust data infrastructure in primary care. Introduction Over recent years, increased adoption of virtual consultations has been observed, driven by the demand for convenient, cost-effective and geographically flexible care.1 2 Current evidence suggests that virtual consultations can be as effective as face-to-face care.3 4 However, limited evidence exists on the safety of virtual versus face-to-face consultations, and no validated methodology for assessing the safety of virtual consultations in primary care has been introduced. Patient safety is the basis of resilient and high-quality healthcare.5 6 Prioritising patient safety improves patients’ outcomes, reduces preventable harm and increases cost-efficiency by avoiding medical errors.5 Effective patient safety strategies rely on evidence-based tools that help identify, track and mitigate risks. In this context, quantitative metrics are essential to systematically assess the safety of medical services and shape healthcare improvement policies. Over recent years, considerable efforts have been made to identify effective measures of safety in healthcare, among which patient safety and quality metrics and indicators have emerged as valuable tools for evaluating and improving care.7,10 These standardised metrics are designed to help healthcare systems systematically monitor, evaluate and enhance safety across various care settings. By identifying adverse events or unsafe practices, safety and quality indicators provide critical insights into areas that require improvement, supporting efforts to protect patients from harm and enhance the overall quality and delivery of care.8 Several prominent institutions, such as the Agency for Healthcare Research and Quality (AHRQ) in the USA, the Organisation for Economic Co-operation and Development (OECD) and the WHO, have played pivotal roles in developing and validating patient safety and quality indicator frameworks globally.7 8 11 12 For instance, the AHRQ has developed a set of 26 indicators that benchmark the safety and quality of in-hospital care, addressing adverse events related to childbirth, medical procedures and surgeries.7 8 Similarly, the European Society for Quality in Healthcare Office for Quality Indicators has contributed to the European Union health system by presenting a selection of patient safety indicators that encompasses medication errors and surgical complications, falls, obstetrics quality and safety, and infection control.10 Despite progress in the development of indicators, existing metrics are mostly tailored to evaluate the safety of hospital care. In primary care, Engels et al developed a framework and quality indicators for general practice management that focus on aspects such as infrastructure, staff, information and finance.13 While this framework also includes a safety domain, it is mainly centred around organisational aspects (eg, ‘the practice has fire extinguishers’). At the same time, a significant gap persists in safety metrics specific to the context of virtual consultations with a focus on tracking adverse events, unsafe practices and consultation outcomes. Virtual consultations differ fundamentally from in-person visits in care delivery processes and risk exposures, particularly for vulnerable populations, which may alter safety dynamics and introduce new safety risks that traditional indicators miss.3 Greenhalgh et al proposed a novel conceptual framework for planning and evaluating remote consultation services, incorporating considerations of complexity and practical ethics.14 Similarly, Bell et al and Spencer and Campbel developed tools for measuring patient safety in primary care,15 16 and the Royal College of General Practitioners developed a toolkit for improving continuity of care,17 but quantitative metrics tailored to the virtual setting are still lacking. Developing safety measures specific to virtual consultations could significantly improve the safety evaluation of these services in primary care. Such measures could help identify and mitigate risks unique to digital care settings, including technological failures, data security issues and communication gaps. They would also equip healthcare providers with essential tools to implement targeted safety strategies and drive continuous quality improvement in virtual care. This systematic review aims to identify and review safety measures applicable to virtual consultations, whether used as standalone interventions or as part of multicomponent models. Methods This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.18 The protocol was prospectively registered with the International Prospective Register of Systematic Reviews (CRD42023464878) and published elsewhere.19 Search strategy A systematic literature search was performed in Ovid MEDLINE/PubMed, Embase and Cochrane Library databases from 2014 to 2024. The reference lists of included articles and grey literature sources (ie, gov.uk data on Medicine safety, European Frameworks for patient safety in primary care) were also searched. The search strategy was developed and agreed through multiple discussions with the academic liaison librarian of Imperial College London. Search strings combined free terms and controlled vocabulary, such as “virtual consultation”, “remote consultation”, “online consultation”, “patient safety”, “patient harm” and “medical errors”. We also used some disease-specific strings to broadly capture relevant outcome measures for the most prevalent conditions in primary care. An example of a search strategy is presented in online supplemental file 1. Study selection Studies were included if they assessed any aspect of safety, either directly (eg, incidence of harm) or indirectly (eg, factors that may influence safety). Experimental studies (randomised and non-randomised controlled trials (RCTs)) and observational studies (cohort, case–control and cross-sectional) involving adults were included, while descriptive studies and those involving children or adolescents were excluded. Studies investigating paediatric care were excluded due to the unique challenges it presents, such as safeguarding and communication dynamics, which warrant a separate review. Included studies had to examine: (1) synchronous patient-provider virtual consultations delivered via telephone or videoconference by primary healthcare professionals or (2) multicomponent interventions involving synchronous remote consultations in primary care. Studies were excluded if they lacked synchronous consultations with healthcare professionals, or if consultations were delivered solely by non-healthcare professionals or by secondary/tertiary care specialists. No limitations on language or country were introduced. More detailed information on the exclusion and inclusion criteria is provided in the protocol19 and in online supplemental file 2. Screening and data extraction Abstract screening and full-text screening were performed by two independent reviewers (TL and K-HH), with inter-rater agreement assessed using Cohen’s kappa.20 One investigator extracted data into a structured electronic form, which was reviewed by the second investigator for consistency. Information extracted included study title, name of author, year of publication, sample size, disease studied, intervention and comparator measures, participant and setting characteristics, and outcome measures used. Discrepancies were resolved through discussion. Where key information was missing, study authors were contacted personally. Quality assessment RCTs were assessed using the Cochrane Risk of Bias Tool, and non-randomised studies with the Newcastle-Ottawa Scale.21 22 Studies meeting ≥50% of low-risk criteria were considered ‘overall low risk’. No studies were excluded based on quality. Data analysis An inductive thematic analysis was performed by TL and ALN, where safety measures identified in the studies were systematically extracted and grouped into overarching themes based on their recurring patterns and similarities. Where possible, they were also categorised according to disease category. This was done to provide a clearer overview of which metrics are most relevant for assessing the health outcomes of virtual consultations, depending on the condition being managed, and to highlight the need to consider disease-specific markers. The themes identified were mapped into a conceptual framework of domains for patient safety evaluation of virtual consultations (figure 1). A narrative synthesis was conducted to summarise the results and present the extracted safety metrics. All extracted measures and key study descriptives, such as population demographics, study design, diseases studied and study setting, are summarised in online supplemental tables 1 and 2. Framework of domains for patient safety evaluation of virtual consultations. Patient and public involvement The local research partners group was convened to gather patient and public perspectives on the study’s relevance and benefits, as well as to outline the strategy for future work.19 Results Search results The database search retrieved 2271 articles, among which 99 were duplicates, and a further 1868 were deemed ineligible after abstract screening (figure 2). Full-text screening was conducted for the remaining 304 articles, of which 257 did not meet the inclusion criteria. A total of 47 studies (31 experimental and 16 observational studies) were included in the review. The inter-rater reliability demonstrated substantial agreement (Cohen’s kappa 0.66). Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram of study selection and screening. Characteristics of included studies The studies involved a total of 2 223 697 participants and investigated the safety of virtual consultations in patients with various medical conditions, such as diabetes (n=12),23,34 hypertension (n=7),35,41 mental health disorders, such as depression, anxiety, post-traumatic stress disorder (PTSD) and sleep disorders (n=12),42,53 heart failure (HF) (n=3),54,56 chronic obstructive pulmonary disease (COPD) (n=3),57,59 osteoarthritis (n=1)60 and skin conditions (n=1).61 Six studies compared virtual and face-to-face consultations in patients with multiple conditions,62,67 while two studies focused on smoking cessation and obesity management.68 69 Most studies were conducted in the USA (n=29)24,2729 and the UK (n=6).32 46 53 58 59 63 Other studies originated from Spain (n=2),55 61 the Netherlands (n=2),57 69 Australia (n=2),37 60 Germany (n=2)47 49 Singapore (n=2),28 38 Sweden (n=1)48 and Denmark (n=1).23 Detailed characteristics of the studies included are summarised in online supplemental table 2. Risk-of-bias assessment Most of the experimental studies were assessed as having an overall low risk of bias (n=22), scoring ‘low’ in at least 50% of the evaluated domains based on the Cochrane Risk of Bias Tool.2325 32 35 37 39 43 47,52 55 56 58 However, due to the nature of the interventions, participants and care providers were aware of their group assignments, leading all studies scoring ‘high risk’ in the domains of blinding of participants and personnel. Nine studies were classified as having a moderate risk of bias, scoring ‘low’ in fewer than 50% of the evaluated domains2429 38 42 44,46 57 66 (online supplemental table 3). Among the 16 observational studies included, 13 were assessed as having a low risk of bias, with total scores ranging from 7 to 9,33 40 54 while three were assessed as having a moderate risk of bias26,2830 31 34 36 41 53 61 64 67 69 (online supplemental table 4). Intervention and comparator Most of the studies investigated single-component interventions, using telephone consultation (n=13),2842 43 45 46 50,52 58 59 65 66 69 video consultation (n=9)23 24 33 44 47 49 53 60 61 or both (n=12)26 27 29 31 32 34 36 40 41 64 67 68 while 13 investigated multicomponent interventions, which combined virtual consultation with additional elements, such as telemonitoring (blood pressure, body temperature, weight, blood oxygen saturation or blood glucose monitoring), text messaging or posted medication reminder charts.2530 35 37,39 48 54 In most studies, usual care served as the comparator, while two studies evaluated patient outcomes by comparing data from before and after the consultations.29 33 Outcomes Analysis of the 47 studies revealed safety measures that were grouped into two overarching themes: (1) factors that may influence safety of virtual consultations and (2) tangible outcomes of virtual consultation safety (online supplemental table 1). Factors that may influence safety included provider-related, patient-related and system-related factors. Tangible outcomes included adverse events, health outcomes and patient experience and perception of safety. Adverse events 15 studies assessed the incidence of adverse events following virtual care as a key aspect of patient safety.3337 40 48 54,62 64 65 For this purpose, a wide range of measures was employed, including all-cause or disease-specific emergency department visits, all-cause or disease-specific hospitalisations, incorrect diagnosis, 7-day (after consultation) primary care office visit, persistent or significant disability as a result of the intervention and disease-specific adverse events (eg, severe hypotension, hypoglycaemic episodes, hypertensive emergencies, acute HF events). Health outcomes 39 studies assessed health outcomes of virtual consultations compared with face-to-face care as a key domain of safety.23,3234 These studies introduced various clinical safety measures, which were grouped by disease categories such as diabetes, hypertension, depression, anxiety, PTSD, chronic pain, insomnia, COPD and osteoarthritis. Two additional categories, covering lifestyle counselling (obesity and smoking cessation), were also included. The most frequently reported clinical measure in diabetes consultations23,34 was glycosylated haemoglobin (HbA1c), while studies focusing on virtual care for hypertension management consistently used systolic and diastolic blood pressure as the primary outcome measure.35,41 All mental health and cognitive–behavioural therapy interventions predominantly relied on validated questionnaires to assess health outcomes, such as Patient Health Questionnaire (PHQ-9), Generalised Anxiety Disorder-7 (GAD-7), Posttraumatic Stress Disorder Checklist – Military Version (PCL-M) and Chronic Pain Acceptance Questionnaire (CPAQ).42,5370 For osteoarthritis consultations, patient-assessed knee pain intensity (numeric rating scale) and physical function Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) have been introduced.60 COPD studies used respiratory symptom questionnaires (eg, dyspnoea score) along with more objective measures such as oxygen saturation levels.57,5962 Patient-related factors Seven studies investigated patient adherence to treatment as a fundamental patient-related factor of the safety of virtual consultations. Outcome measures used included ‘non-attendance of the assessment visit’, ‘no show’, ‘number of missed pills’ and ‘adherence to self-monitoring’. Several scales, such as the Diagnostic Adherence to Medication Scale, were employed to measure medication adherence.25 29 33 39 40 61 63 Provider-related factors The safety of virtual consultations at the provider level was assessed through measures of physician adherence to evidence-based practices in five studies.36 58 64 66 67 Baughman et al provided a detailed framework for evaluating evidence-based practices in primary care consultations, categorising them into three domains: medication-based, testing-based and counselling-based. For medication-based indicators, the authors reported metrics, such as the rates of evidence-based medication prescribing (eg, ‘cardiovascular disease (CVD) receiving antiplatelet’, ‘CVD receiving statin’, ‘β-blocker prescription for HF’, etc). Testing-based metrics included lipid testing in CVD and HbA1c/nephropathy testing for diabetes. Counselling-based metrics evaluated rates of screening uptake and vaccinations.67 Other studies highlighted additional safety metrics, such as ‘lab and image ordering’, absence of recorded blood pressure (‘no blood pressure recorded’), medication errors and unsafe prescribing as critical measures of safety in primary care consultations.36 64 75 76 System-related factors Patients’ perceptions of system factors—accessibility, timeliness and continuity of care—were assessed via questionnaires, such as the Patient Assessment of Chronic Illness Care, Revised Assessment of Service Satisfaction-General (RAS-G) and Satisfaction with Care Questionnaire (SCQ).29 30 47 49 52 All questionnaires collected responses on Likert scales and assessed aspects such as ease of accessing services and appointment scheduling (RAS-G), coordination among providers and follow-up care (SCQ) and Patient Assessment of Chronic Illness Care (PACIC) and waiting times (SCQ). Discussion Key findings This review systematically appraised 47 articles, comprising 31 experimental and 16 observational studies, to identify measures used to evaluate the safety of virtual compared with face-to-face consultations. To our knowledge, this is the first systematic review to specifically examine safety measures for virtual consultations in primary care, which underscores the significance and novelty of this work. All extracted safety measures assessed patient safety in virtual versus face-to-face consultations through two primary domains: factors that may influence safety and tangible outcomes (figure 1). Safety measures assessing factors that influence the safety of virtual consultations were categorised into patient-related, centred around patient adherence to treatment; provider-related, focusing on physicians’ evidence-based practices; and system-related, encompassing aspects such as accessibility, timeliness and continuity of care. Tangible outcomes of patient safety in virtual consultations were assessed through three dimensions: adverse events, health outcomes, and patient experience and perception of safety. Interpretation and comparison with the previous literature Adverse events serve as a core component of many patient safety evaluation metrics, including AHRQ Patient Safety Indicators7 8 and emerged as key measures for evaluating the safety of virtual consultations in this review.3337 40 48 54,62 64 65 The extracted patient safety measures included disease-specific adverse events, such as hypoglycaemic episodes in diabetes,33 hypertensive crises in hypertension37 and acute HF exacerbations,55 as well as treatment-related adverse events,40 48 56 60 mortality rates,37 52 55 56 65 hospital admissions5455 57,59 62 64 65 and emergency department visits.54 57 62 Tracking the incidence of adverse events following virtual consultations has broad implications for identifying systemic risks, such as technology failures, provider errors or patient barriers in using telemedicine.77 However, distinguishing whether adverse outcomes result from virtual care itself, or stem from broader systemic issues can sometimes be challenging. Therefore, contextual factors must be considered before flagging a consultation as ‘unsafe’. ‘Health outcomes’ was one of the most frequently assessed safety domains of virtual consultations, with 39 studies citing a wide range of disease-specific measures. Evaluating health outcomes is crucial as they reflect whether virtual consultations achieve their intended goal, manage patients’ symptoms and provide continuity of care.4 However, due to a high heterogeneity of measures, robust standardisation is needed to ensure comparability and consistency in reporting. In recent years, patient perception of quality and safety of care has been increasingly acknowledged as vital to the overall safety culture in healthcare services.78 Evidence indicates that patients are highly perceptive of the care they receive, and their insights can identify safety issues that might otherwise go unnoticed.79 The questionnaires captured by this review, although factored in patients’ general perspectives on quality and safety, did not explicitly focus on patient-reported safety issues in virtual consultations. Currently, there is no validated questionnaire specifically designed to assess safety incidents in virtual consultations from patients’ perspectives. However, the OECD’s question set for hospital care ‘Patient-reported safety indicators’80 could be adapted to virtual consultations and this review highlights the need for such an initiative. 11 studies in this systematic review explored patient-related safety aspects in virtual consultations, centred primarily around patient adherence to treatment.2529 30 33 39 40 47 49 61,63 It is important to highlight that while patient adherence and patient experience and perception of safety are distinct concepts, they are closely interconnected. For instance, Momin et al reported that satisfaction with care positively correlated with adherence to treatment in their study.29 Similarly, Doyle et al suggested that positive patient experience is associated with improved clinical effectiveness and enhanced patient safety.81 This interplay highlights the importance of addressing both dimensions to optimise the safety and quality of virtual care. Incorporating evaluation measures that track patient compliance into standardised safety metrics for virtual consultations is essential, as they can flag suboptimal-quality consultations and identify at-risk patients.82 Another key theme identified in the review was evidence-based practice, which reflected provider-related aspects of safe care. Evaluating provider adherence to evidence-based care can foster the reduction of diagnostic and treatment errors, enhanced standardisation of care, minimising harm and, consequently, better patient outcomes.83 The framework for virtual consultations developed in this study intersects with the general practice management framework proposed by Engels et al, particularly in areas such as accessibility and evidence-based care. At the same time, it extends the focus to include the tracking of adverse events and health outcomes associated with virtual consultations and highlights the importance of incorporating patient feedback into the quality and safety assurance processes.13 Alongside the review findings, several additional frameworks, such as those by Bell et al and Spencer and Campbell, and Starfield outline further safety-related domains.15 16 84 The Starfield framework identifies comprehensiveness and coordination as key domains of safety, while the frameworks by Spencer et al and Bell and Campbell also address aspects such as workforce, management, patient involvement, background systems, premises, interface and others.15 16 Strengths and limitations This review offers a novel contribution by addressing the evidence gap around safety measures for virtual consultations. To our knowledge, this is the first review of its kind, as no prior work has systematically developed or evaluated a set of safety measures specific to virtual primary care. Second, this study adhered to a predefined systematic review protocol, ensuring a robust methodological approach and transparency in the reporting of results. This structured framework enhances the credibility and reliability of the findings. However, several limitations should be noted. First, given the complexity and multifaceted nature of the research question, the review may not fully capture all relevant aspects. Some condition-specific measures, such as HbA1c and blood pressure, while common, offer only a partial view of overall consultation safety, whereas aspects such as communication quality, shared decision-making and continuity of care remain under-represented in the existing literature. Systems-level factors were assessed through patient-reported data rather than objective metrics, which may not accurately reflect actual safety risks. Finally, several studies have been assessed as having a moderate-to-high risk of bias. As the primary objective of this review was to extract and map outcome measures rather than to evaluate the effectiveness of interventions, studies were not excluded based on quality. However, we acknowledge this as a potential limitation. Implications for research, policy and practice Based on the findings of this systematic review, we have developed a framework of domains (figure 1), with corresponding recommendations for patient safety evaluation of virtual consultations in primary care (figure 3). Tangible safety outcomes can be assessed by monitoring the frequency of adverse events, such as mortality, hospitalisations, emergency department visits or persistent disability resulting from care, as well as by evaluating health outcomes through disease-specific measures and assessing patient experience and perceived safety. However, to provide a holistic evaluation of patient safety in virtual consultations, not only tangible outcomes, but also factors influencing safety should be assessed. These include provider adherence to evidence-based care, patient adherence to treatment and system factors, such as accessibility, timeliness and continuity of care. Key considerations for patient safety evaluation in virtual consultations. HF, heart failure, RCGP, Royal College of General Practitioners Although this review identified many crucial safety aspects of virtual consultations that need to be evaluated, further work is needed to finalise the associated safety metrics. Developing a set of safety measures requires a stepwise approach: evidence synthesis, original data collection of service user perspectives and validation studies. To fully embrace the potential of safety measures in monitoring safety, a robust data infrastructure must be developed. Information on the modality of consultation (face to face or virtual) needs to be routinely collected for research and quality assurance purposes. Establishing a robust methodology for safety evaluation of virtual consultations will enable safety monitoring, root cause analyses and safety improvement. It will also help identify best practices, which can inform the development of tailored interventions for patient safety improvement. Conclusion This review systematically examined measures that could be used to benchmark the safety of virtual consultations in primary care. The safety measures were categorised into key domains: factors that may influence safety, such as patient-related factors, provider-related factors and system-related factors, and tangible outcomes of virtual consultation safety, including adverse events, health outcomes and patient perception of safety. Further research is needed to validate these measures, ensuring their reliability, consistency and applicability in digital primary care settings. Supplementary material Footnotes Funding: This study/research is funded by the National Institute for Health and Care Research (NIHR) North West London Patient Safety Research Collaboration (NIHR NWL PSRC). ALN is also funded by the NIHR Applied Research Collaboration Northwest London (NIHR ARC NWL) with infrastructure support from the Imperial NIHR Biomedical Research Centre. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. Provenance and peer review: Not commissioned; externally peer reviewed. Patient consent for publication: Not applicable. Ethics approval: Not applicable. Correction notice: This article has been corrected since it was first published online. The funding statement has been updated. Data availability statement Data are available in a public, open-access repository. Data are available on reasonable request. References Associated Data Supplementary Materials Data Availability Statement Data are available in a public, open-access repository. Data are available on reasonable request.