Measuring provider-level differences in perioperative workflow using computer vision-based artificial intelligence Authors: Theoren Loo, Brandon Mcglennen, Stephen Incavo, Nate Hilger Published: 2025 ABSTRACT AbstractObjectivesTo evaluate provider-level variability across the full perioperative workflow using a computer vision-based artificial intelligence (AI) system that automatically detects and timestamps operating room events.MethodsA cross-sectional study of total knee arthroplasty cases performed between September 2022 and March 2025 at a regional health system was conducted. An ambient surgical platform equipped with wall-mounted cameras continuously captured perioperative activity. A YOLO-based model identified patients, staff and equipment, and a transformer-based event detector predicted key perioperative events in real time. Detected events were used to segment cases into eight workflow phases: anaesthesia induction, patient preparation, final preparation, active procedure, postoperation, patient exit, room cleanup and room setup. Provider-level variability in segment durations was evaluated after adjusting for case characteristics, daily surgical volume and team composition.ResultsThe computer vision event detection system achieved high agreement with ground truth annotations. Across 2502 surgical cases, significant provider-level variability was observed in all workflow segments except for room exit. Active procedure showed the greatest variation among surgeons (F=28.4, p<0.001; β IQR=−20.9 to 8.8 min) followed by room setup among circulating nurses (F=1.3, p<0.001; β IQR=−5.2 to 4.4 min) and room setup among scrub nurses (F=1.4, p<0.001; β IQR=−3.7 to 3.2 min).ConclusionsAutomated workflow segmentation using computer vision provides a scalable method to evaluate perioperative efficiency with greater granularity. Broader case segmentation may support more targeted and effective surgical quality improvement initiatives.Keywords: Artificial intelligence, Convolutional Neural Networks, Data Science, Data Systems, BMJ Health Informatics WHAT IS ALREADY KNOWN ON THIS TOPIC Operating room inefficiency is a major contributor to hospital costs, yet most improvement initiatives focus narrowly on turnover times rather than the full perioperative workflow. Operating room inefficiency is a major contributor to hospital costs, yet most improvement initiatives focus narrowly on turnover times rather than the full perioperative workflow. WHAT THIS STUDY ADDS This study shows that computer vision-based artificial intelligence can automatically segment surgical workflows and reveal significant provider-level variability across perioperative phases. This study shows that computer vision-based artificial intelligence can automatically segment surgical workflows and reveal significant provider-level variability across perioperative phases. HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY Automated workflow segmentation can enable scalable benchmarking and guide targeted interventions to improve surgical efficiency beyond traditional turnover metrics. Automated workflow segmentation can enable scalable benchmarking and guide targeted interventions to improve surgical efficiency beyond traditional turnover metrics. Introduction Operating room (OR) time is a significant driver of hospital spending,1 costing US$46.04 per OR-minute.2 These expenses accumulate rapidly, contributing to millions of dollars in annual operational costs. Consequently, there is a strong financial incentive to optimise OR time by improving efficiency throughout the entire surgical procedure. Although hospitals have sought to reduce unnecessary delays, quality improvement initiatives tend to focus narrowly on turnover times.3,5 While turnover efficiency is an important operational metric, an exclusive focus on this phase oversimplifies the complexity of the surgical workflow and overlooks other perioperative segments where inefficiencies may occur. Addressing inefficiencies across the full surgical case not only helps control costs and improves satisfaction for patients, but also allows hospitals to maximise the utilisation of expensive OR resources. Improving OR efficiency can translate to higher revenue generation by accommodating more procedures within the same timeframe,6 reducing overtime expenses7 and minimising staff turnover.6 Given these high stakes, hospitals are increasingly adopting data-driven strategies and technological innovations to optimise surgical workflows and resource allocation. Recent studies have demonstrated that partitioning case durations into event-based segments is effective in identifying and improving OR inefficiencies.3 8 Goldhaber et al divided turnover times into five segments with a timestamping system using clickable buttons on the electronic health record (EHR) default prelogin screen. Sharing segment duration data with OR team members and managers was associated with a reduction in environmental response time by 1.7 min and cleaning time by 4.5 min.3 Meyers et al segmented case durations into eight distinct stages and found that specific members of the OR team contributed significant variability within different segments of the procedure.8 By identifying workflow inefficiencies within team roles and providing appropriate training, this variability can be minimised, leading to shorter and more consistent case duration segments. These studies highlight the value of segmenting case durations into discrete phases, offering a more precise approach to identifying inefficiencies compared with traditional frameworks that treat turnover time and total case duration as undifferentiated periods of time. Accurate and robust measurement is needed to identify important gaps in operational performance. Benchmarking plays a critical role in identifying performance outliers and driving continuous improvement in surgical workflows.9,12 In the context of OR operation efficiency, benchmarking involves comparing individual providers or service lines against institutional averages. With granular segmented case duration data, benchmarking enables highly specific performance assessments which can reveal workflow inefficiencies across providers or service lines. This facilitates a data-driven approach that allows OR managers to target training, allocate resources and redesign workflows towards specific inefficiencies. However, collecting detailed event timestamps often places an additional burden on OR nurses who already spend 9.2 min of operating time on documentation.13 To alleviate this burden and improve data fidelity, several automated event-tracking technologies have been explored. Early work using computer vision demonstrated the ability to recognise actions in the OR using spatiotemporal Red–Green-Blue-Depth feature encoding.14 However, this study was limited to classifying actions within preselected video clips and did not detect or timestamp perioperative events in real time. Other approaches have leveraged pressure sensors, ventilator-mounted motion detectors, ambient light sensors and Radio Frequency Identification Device systems.15 16 Although these sensor-based solutions offer the potential for real-time, hands-free data collection, they also present significant limitations. The complexity of system integration, dependence on precise sensor placement and need for frequent calibration and maintenance pose challenges for long-term reliability. Computer vision has been applied in laparoscopic and minimally invasive surgery where continuous video streams allow for automated recognition of surgical phases, instrument use and workflow segmentation.17 Deep learning models such as EndoNet and Convolutional Neural Network Long Short-Term Memory have achieved high performance in phase recognition and intraoperative decision support.18 19 However, these advances have largely remained confined to endoscopic settings and have not been applied to the full perioperative workflow. In response to the limitations of existing sensor-based event tracking systems in the OR, we developed and implemented a computer vision-based artificial intelligence (AI) system that autonomously detects perioperative events in real time using cameras and deep learning algorithms. In this study, we describe the AI system and analyse segment durations and variability for total knee arthroplasty (TKA) cases as one of many targeted surgical workflows under evaluation. We hypothesise that segment durations vary among OR team members and that this variability can serve as a foundation for a benchmarking framework to identify systemic inefficiencies and design targeted interventions. Methods Study design This cross-sectional study used data from TKA cases performed between September 2022 and March 2025 at a large regional academic health system in Texas. Even though the computer vision model can detect events that are relevant to many procedure types, TKAs were selected as the focus of this analysis because they are the most commonly performed orthopaedic procedure in ORs in the USA and are expected to grow to 1.3 million procedures by 2030.20 21 To ensure data consistency and comparability, TKA cases were excluded if they involved more than one surgical procedure or involved robotic assistance as these factors may introduce workflow variation not representative of standard TKA procedures. This study was conducted as a quality improvement initiative and adheres to the Standards for Quality Improvement Reporting Excellence 2.0 guidelines for transparent and rigorous reporting of healthcare improvement studies.22 Computer vision and AI event tracking system An ambient surgical operation platform equipped with wall-mounted cameras was used to identify and timestamp OR events in real time without oversight from OR staff (Apella Technology, San Francisco, California, USA). Four high-resolution cameras installed on each wall of the OR captured continuous streams of images which were used in a YOLO-based model to identify patients, staff and equipment. This model is trained on data from multiple sites and organisations which enable it to learn visual patterns that generalise across different OR configurations, lighting conditions and equipment setups. The output is used in a transformer-based model that analyses temporal sequences of detected objects and their spatial relationships to infer surgical events. Surgical domain experts annotate a random 10% sample of events which are reviewed by a senior annotator to establish the ground truth. During the study period, the annotation process was broadened such that nearly all events were manually annotated. Any discrepancies in annotations were resolved through consensus discussion between the original annotator and reviewer. All image and video data were retained on secure, access-controlled servers for a maximum of 28 days and then automatically deleted in accordance with institutional data security protocols. Data collection Cases were deconstructed into the following segments using a selection of events identified by the computer vision model: anaesthesia induction, patient preparation, final preparation, active procedure, postoperation, patient exit, room cleanup and room setup (figure 1). These segments were used due to their relevance across all cases. Timeline of case segments and events. EHR data were used to obtain staff identification and service role assignments for each case. With guidance from surgeons and OR managers, we developed a segment-role mapping framework: the anaesthesiologist for anaesthesia induction; the circulating nurse for patient preparation; the surgeon for final preparation and active procedure; the scrub nurse for postoperation; the circulating nurse for patient exit; and the circulating nurse and scrub nurse for room setup. Room cleanup was excluded from analysis due to the absence of environmental service staff data in the EHR. In cases where multiple individuals shared the same role, the primary assigned staff member was used for indexing. Statistical analysis The computer vision event detection system was evaluated using F1 scores, accuracy, precision, recall and median time error compared against ground truth annotations. For TKA case segments, medians and interquartile ranges (IQRs) were calculated, both in aggregate and stratified by individual staff members within each role. To evaluate provider-level variability in segment durations, Kruskal–Wallis H tests were performed. Multivariable linear regression models were fit for each segment duration to examine the association between provider adjusting for case-specific covariates (case acuity, month of surgery, day of week, scheduled start hour, outpatient status and flip room status), daily case volume and team composition (surgeon, anaesthesiologist, circulating nurse and scrub nurse). Using effect coding, provider β was interpreted as the difference in segment duration relative to the average across all providers. An F-test was conducted to evaluate whether providers explained a statistically significant proportion of the variance in segment duration after adjusting for other covariates. Data analysis was performed using SciPy V.1.0.18 and statsmodels V.0.14. Results At the time of this study, the computer vision event detection system was trained and validated on a global dataset comprising 137 517 surgical cases performed in 315 ORs by 1853 surgeons. When evaluated at the study site, the system achieved high agreement with ground truth annotations, with F1 scores, accuracy, precision and recall values ≥0.97 across all events and median time errors ranging from 6.4 to 16.9 s (table 1). Table 1Computer vision event detection evaluation at study siteEventEvent predictionsGround truth eventsF1 scoreAccuracyPrecisionRecallMedian time error (seconds)Wheels in184 984184 1720.990.990.990.996.9Patient draped179 892177 4240.980.970.980.9916.9Patient transferred to bed171 660170 0600.990.980.990.996.6Cleanup end173 678173 0320.990.990.990.996.4Wheels out185 126184 2460.990.990.990.997.4Anaesthesia ready, case start and case closed event data were extracted from the EHR and are excluded from this table. Computer vision event detection evaluation at study site Anaesthesia ready, case start and case closed event data were extracted from the EHR and are excluded from this table. The overall median case duration for TKAs was 178 min (n=2502; IQR: 146–209 min) with median segment durations reported in figure 2. Active procedure was the largest segment (33%), followed by postoperation (19%) and room setup (15%). The turnover phase (room cleanup and room setup) accounted for 24% (42 minutes) of cases. Total knee arthroplasty case deconstruction by median segment lengths with whiskers indicating the 25th and 75th percentile. Significant differences in unadjusted segment durations were observed for all providers, all with p<0.001 (table 2). Adjusted linear regression and F-tests revealed significant provider-level variation across most case segments (table 3). Variation was most pronounced during the active procedure among surgeons, with additional differences observed during anaesthesia induction, patient preparation, final preparation, postoperation and room setup. In contrast, patient exit times showed no significant variation across providers. The distribution of β for each provider is shown in online supplemental Figure 1. Table 2Median total knee arthroplasty case segment durations by providerSegmentAssigned providerNumber of providersMedian cases per provider (IQR)Median segment duration (IQR)P valueAnaesthesia inductionAnaesthesiologist1004.5 (2.0–17.8)10 (8–14)<0.001Patient preparationCirculating nurse1405.0 (2.0–21.0)11 (8–15)<0.001Final preparationSurgeon3822.5 (3.3–90.3)8 (6–12)<0.001Active procedureSurgeon3822.5 (3.3–90.3)59 (42–74)<0.001PostoperationScrub nurse12624.5 (2.0–18.5)34 (27–43)<0.001Patient exitCirculating nurse1405.0 (2.0–21.0)2 (1–3)<0.001Room setupCirculating nurseScrub nurse1401265.0 (2.0–21.0)24.5 (2.0–18.5)27 (20–40)27 (20–40)<0.001<0.001The ‘Assigned provider’ column reflects the staff member responsible for each workflow segment. Room cleanup was excluded from analysis due to the absence of environmental service staff data in the EHR. P values correspond with Kruskal-Wallis H tests assessing differences in segment durations across providers within each role. Median total knee arthroplasty case segment durations by provider The ‘Assigned provider’ column reflects the staff member responsible for each workflow segment. Room cleanup was excluded from analysis due to the absence of environmental service staff data in the EHR. P values correspond with Kruskal-Wallis H tests assessing differences in segment durations across providers within each role. Table 3Adjusted linear regression and F-test evaluating provider-level variation in total knee arthroplasty segment durationSegmentR2Assigned providerMedian β (IQR)FP valueAnaesthesia induction0.33Anaesthesiologist−0.87 (−3.7, 3.1)2.3<0.001Patient preparation0.35Circulating nurse0.16 (−2.3, 1.9)1.3<0.001Final preparation0.41Surgeon−0.61 (−2.2, 2.5)9.6<0.001Active procedure0.68Surgeon−1.04 (−20.9, 8.8)28.4<0.001Postoperation0.49Scrub nurse−0.06 (−2.9, 2.4)3.2<0.001Patient exit0.16Circulating nurse0.01 (−0.4, 0.3)0.70.98Room setup0.36Circulating nurseScrub nurse−1.17 (−5.2, 4.4)−0.21 (−3.7, 3.2)1.31.4<0.001<0.001The ‘Assigned provider’ column reflects the staff member responsible for each workflow segment. Room cleanup was excluded from analysis due to the absence of environmental service staff data in the EHR. β values represent adjusted provider effects using effect coding relative to the grand mean. P values correspond with F-tests. Adjusted linear regression and F-test evaluating provider-level variation in total knee arthroplasty segment duration The ‘Assigned provider’ column reflects the staff member responsible for each workflow segment. Room cleanup was excluded from analysis due to the absence of environmental service staff data in the EHR. β values represent adjusted provider effects using effect coding relative to the grand mean. P values correspond with F-tests. Discussion To our knowledge, this is the first study to use computer vision for real-time identification of perioperative events and segmentation of surgical case durations. By automatically measuring the duration and variability of individual case segments at the provider level, we can more effectively analyse the many components of a surgical case without the need for manual data entry. Automating perioperative event collection reduces the documentation burden on OR nurses, allowing them to focus on patient care, clinical decision-making and coordination tasks. In addition to enhancing workflow efficiency, automated timestamping improves data fidelity and consistency, reducing the risk of human error and enabling more reliable quality improvement efforts.23 Although most OR efficiency initiatives traditionally focus on shortening turnovers,3,5 our analysis reveals that turnovers (room cleanup and room setup segments) account for only 24% of total OR time in TKA cases. This finding suggests that the exclusive focus on turnover overlooks a substantial portion of the surgical workflow where delays and variability may also occur. By broadening the scope of efficiency efforts to include all perioperative segments such as anaesthesia induction, patient preparation, final setup, active procedure, post operation, patient exit, room cleanup and room setup processes, OR teams may uncover additional opportunities for time savings, standardisation and quality improvement. Significant provider-level variability was observed across multiple segments, including anaesthesia induction, patient preparation, final preparation, active procedure, postoperation and room setup, even after adjusting for case characteristics, daily surgical volume and team composition. The greatest variability occurred during the active procedure segment (IQR 29.7 min), which may reflect potential differences in surgical technique, intraoperative decision-making, workflow efficiency and unmeasured case complexity that may not have been fully captured by the model’s covariates.24 25 Room setup times among circulating nurses and scrub nurses also showed notable variation (IQRs 9.6 and 6.9 min), suggesting opportunities for standardisation of workflows and coordination practices. The adjusted β coefficients may offer a useful starting point for developing a provider-level benchmarking framework. Using effect coding, each β represents the difference in segment duration for a provider relative to the grand mean, enabling comparisons within provider roles. While these estimates should be interpreted cautiously and provider identities anonymised, consistently positive β values may flag opportunities for efficiency improvements, whereas negative β values may indicate more streamlined workflows. To explore the utility of this framework, we reviewed videos from the past 28 days of cases with long active procedures among anonymised surgeons with β in the 90th percentile. Delays were associated with atypical sequencing (eg, early patella resurfacing and unusually repetitive trialling), extended cement curing, prolonged closure, and limited staff support. Additionally, some vendor representatives were less familiar with the implant system and relied heavily on notes throughout the procedure. These cases had a median active procedure of 86 min compared with the cohort’s median of 59 min. While some individual segment-level time savings may appear modest, capturing small, consistent efficiency gains across routine workflows may result in substantial aggregate improvements that are clinically significant. In our study, the median total case duration was 178 min with an IQR of 63 min, reflecting significant variability across cases. Reducing the duration of the longest cases towards the median may enhance surgical throughput and optimise OR resource utilisation. Lean and Six Sigma methodologies in healthcare have demonstrated that incremental time savings can scale to meaningful system-wide efficiencies.26 A study at the Atlanta Veterans Affairs Health Care System applied these principles to reduce turnover times and saved US$2370 per turnover and US$663 000 over 11 months.4 In hospitals performing hundreds or thousands of procedures annually, these small efficiency gains can translate to hours of recovered OR time, allowing for the accommodation of additional cases and reduction in staff overtime.6 7 One limitation of this study is that the regression models do not adjust for patient-level factors such as age, body mass index or comorbidities. As a result, some of the observed variability in case segments may reflect unmeasured patient complexity rather than true differences in provider efficiency, particularly in the active procedure segment. Second, although computer vision-based systems offer advantages over sensor-based approaches, they remain sensitive to environmental variability such as room layout, lighting conditions and potential occlusion by equipment or personnel. To address these factors, our system uses four wall-mounted cameras in each OR to minimise blind spots. Model performance has remained consistently high across diverse operating environments, demonstrating strong robustness to these conditions. Ongoing integration of data from additional sites will further strengthen the system’s generalisability. Third, data for this study were derived exclusively from TKAs performed at a single regional academic health system which may limit the generalisability of findings. Workflow dynamics, provider roles and variability patterns observed here may differ in other surgical specialties or institutions with different staffing models, patient populations or perioperative workflows. Further validation across diverse surgical settings and with more granular clinical data would strengthen the robustness and applicability of these findings. This study demonstrates the feasibility and utility of using computer vision-based AI to segment surgical workflows and evaluate provider-level performance across distinct perioperative events. By leveraging automated event detection and effect-coded regression modelling, we identified meaningful variability in several segments of TKA cases that may serve as targets for operational improvement. Our findings support a shift away from the traditional focus on turnover times and towards a more comprehensive, data-driven benchmarking framework that accounts for the total surgical case. As healthcare systems seek to optimise OR performance and reduce inefficiencies, incorporating segment-level analytics powered by computer vision may enable more precise, scalable and sustainable quality improvement strategies. Future work should explore the integration of clinical variables and multi-institutional validation to enhance the generalisability and impact of this approach. Supplementary material Footnotes Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors. Provenance and peer review: Not commissioned; externally peer reviewed. Patient consent for publication: Not applicable. Ethics approval: Not applicable. Data availability statement No data are available. References Associated Data Supplementary Materials Data Availability Statement No data are available.