First-in-human evaluation of real-time pixel-level AI-assisted anatomical segmentation in neurosurgery: pituitary surgery as an exemplar Authors: Danyal Z Khan, Zhehua Mao, Anjana Wijekoon, Adrito Das, Simon C Williams, Ann Blandford, Abhiney Jain, Lauren Harris, Anouk Borg, Neil L Dorward, Matthew J Clarkson, Sophia Bano, Peter McCulloch, Danail Stoyanov, Hani J Marcus Published: 2026 ABSTRACT IntroductionPrecise anatomical navigation is fundamental to safe endoscopic pituitary surgery, a high-stakes procedure characterised by a challenging learning curve. While traditional navigation systems often rely on workflow-disrupting probes or static preoperative imaging, advancements in computer vision AI (CVAI) now enable dynamic, real-time pixel-level anatomical segmentation directly from live surgical video. Our group has previously conducted a series of preclinical human-computer interaction studies to refine the system’s design, alongside digital and high-fidelity physical simulations demonstrating the potential benefit of AI assistance in improving surgical performance, training, and safety. Building on this foundation, the current study represents a first-in-human evaluation of real-time pixel-level CVAI anatomical segmentation in the neurosurgical operating room - assessing feasibility, human factors and clinical outcomes, while iteratively improving the system.MethodGuided by the DECIDE-AI and IDEAL frameworks, this single-centre evaluation comprises an initial proof-of-concept of CVAI anatomical segmentation in endoscopic transsphenoidal pituitary surgery. The AI model utilised a DINOv3-derived vision transformer architecture, deployed via a high-performance edge computing unit to achieve low-latency real-time inference without reliance on cloud infrastructure. Feasibility and functionality were assessed via structured questionnaire, prospective observation, and blinded retrospective review of the recordings of the endoscopic surgical video feed and wider operating room environment. Continuous multi-stakeholder feedback through validated human factors surveys drove iterative technical refinements between cases. Routine clinical outcomes, aligning with the standard pituitary surgery core outcome set, were collected.ResultsEight patients with pituitary adenomas were enrolled. The CVAI system was successfully deployed in six cases, demonstrating acceptable real-time pixel-level sella segmentation accuracy. Deployment failed pre-operatively in two cases owing to a single platform-level boot-configuration issue. Iterative refinement between cases was driven by our experience and surgical team feedback. This resulted in the integration of additional anatomical structure segmentations (e.g., carotid arteries), enhanced model accuracy via training dataset expansion, and hardware firmware upgrades. Multi-stakeholder surveys demonstrated satisfactory system feasibility, usability, and acceptability among the surgical team. Both prospective observation and retrospective video review confirmed the absence of adverse events, including no significant distraction to the primary surgeon, and there were no AI-related clinical complications.ConclusionThis first-in-human early clinical evaluation (IDEAL Stage 1) of real-time pixel-level AI anatomical segmentation in live neurosurgery demonstrates feasibility, showcases iterative system evolution, and reports clinical and human factors outcomes. Future work will include a larger single-centre case series (IDEAL Stage 2a) with more surgical teams to further iterate the system and explore its impact on safety, training and workflow. As the underpinning AI models improve and integrate with other intra-operative navigational technologies, such tools will likely be the cornerstone of intra-operative surgical decision support systems.Supplementary InformationThe online version contains supplementary material available at https://doi.org/10.1007/s11102-026-01762-x. Background The mastery of anatomical knowledge and precise intra-operative navigation constitute the fundamental tenets of safe surgical practice. Inadequacy in anatomical orientation and navigational errors represent significant risks to patient safety, with the potential for serious morbidity and mortality. Consequently, a variety of surgical adjuncts have been developed to assist with orientation and navigation (e.g., image-guidance). When integrated correctly into the surgical workflow, these tools serve a dual purpose: providing real-time guidance and facilitating the consolidation of anatomical learning and reflection-in-practice. Recent advancements in artificial intelligence (AI) and computational power have catalysed the emergence of computer vision AI (CVAI) analysis of live surgical video as a candidate technology to assist with anatomical navigation. Unlike traditional navigational systems that often rely on workflow-disrupting specialised probes or static preoperative imaging, CVAI-based analysis of live surgical video offers a dynamic alternative that can adapt to changing surgical scenes. Pixel-wise segmentation, a subtype of CVAI, offers the potential for precise localisation of surgical structures, and therefore has more potential to assist with navigation, when compared to CVAI models which focus on simple detection or classification. While the proof of concept for real time CVAI segmentation of anatomical structures has recently been established in laparoscopic general surgery [1–3], its application in more complex, anatomically dense environments remains an active area of investigation. Endoscopic pituitary surgery represents a compelling candidate for the clinical translation of CVAI-based navigation. From a clinical perspective, the procedure is characterised by high stakes, a steep learning curve, and a surgical field crowded with critical neurovascular structures. From an AI engineering standpoint, the entirely endoscopic nature of the procedure provides a high-quality video feed and offers a balance between workflow homogeneity and scene heterogeneity. Leveraging this, CVAI models that are capable of accurate anatomical segmentation have been developed [4–10]. Preclinical data indicates these models improve anatomical orientation among surgeons and residents and enhance overall performance in high-fidelity simulations [11, 12]. The next stage in translation of CVAI-based anatomical segmentation for navigation assistance in endoscopic pituitary surgery is early clinical evaluation. However, prior to translation from the digital world to the operating room, the surgeon-AI interface must be carefully designed to maximise usability, reduce distraction and cognitive overload, and improve the surgeon’s ability to interpret AI recommendations appropriately [13]. To address this, emerging evidence has explored the optimisation of AI via augmented reality overlays via digital simulations and real-time physical simulations [11, 14, 15]. The resulting optimized user-interface not only retains the previously realised performance benefits with AI-assistance, but also may improve trust calibration, a key consideration in the DECIDE-AI guidelines for early clinical translation of AI systems [16, 17]. Thus, the goal of this clinical trial is to assess the feasibility of CV-based anatomical segmentation during live endoscopic pituitary surgery. Adhering to the DECIDE-AI and IDEAL frameworks for surgical innovation, we employ an iterative development process guided by real-world intra-operative experience and structured feedback from the multi-stakeholder surgical team [17, 18]. To our knowledge, this study is the first clinical case series of the application of real-time pixel-level CVAI anatomical segmentation in live neurosurgery with human factors and clinical outcomes reported alongside technical feasibility. Methods Overview The overarching goals of this early clinical evaluation are to iteratively improve the technology and assess its feasibility. Guided by the IDEAL Stage 1 and DECIDE-AI frameworks, a non-comparative, single-centre, proof-of-concept study was performed [17, 18]. The study was registered on a clinical trials database (NCT07568366; registered 5th May 2026), ethical approval was granted via regional ethics committee (IRAS 127474) and informed consent was received from each participant accordingly. System overview The CVAI model was developed using retrospectively collected surgical video data from 110 patients who underwent endoscopic transsphenoidal pituitary surgery at the National Hospital for Neurology and Neurosurgery (NHNN) [4–6]. From these procedures, 2157 image frames were selected from surgical stages where intraoperative anatomical guidance is considered most clinically relevant, including sellotomy, durotomy, and tumour resection, based on previously published international consensus recommendations for operative workflow and critical views for safe surgical progression [19, 20]. Selected images were annotated using semantic segmentation of clinically relevant anatomical structures, including the sella, carotid arteries, optic nerves, and clival recess. Annotations were performed by two neurosurgical residents (DZK and LH) with experience in pituitary surgery and subsequently spot-verified by an attending neurosurgeon (HJM). A DINOv3-derived vision transformer (VIT-B) architecture was chosen and the dataset was split at the patient level, with 97 patients used for model training and 13 held out for testing. On the held-out test set, sella segmentation achieved a precision of 84.7%, recall of 88.0%, and IoU of 76.0% (corresponding Dice score, 86.4%). Performance for the carotid arteries was lower, reflecting the greater difficulty of recognising these structures, with a precision of 54.3%, recall of 13.6%, and IoU of 12.2% (Dice, 21.8%). Following this preclinical evaluation, the same model configuration was retrained using data from all 110 patients and each subsequent patient included in the trial to maximise use of the available annotated data before each clinical deployment. No formal performance threshold was prespecified; however, informed by prior precedent, a DICE of approximately 70% for the primary target, the sella, was considered a reasonable minimum prior to first deployment [1, 21–27]. We aimed to improve this iteratively, along with performance for harder structures such as the carotid arteries, through dataset expansion and active learning during the trial. The CVAI system was hosted on an NVIDIA IGX Orin platform (NVIDIA, Santa Clara, CA, USA), comprising a 12-core Arm Cortex-A78AE CPU and 64 GB LPDDR5 system memory, and equipped with an NVIDIA RTX A6000 discrete GPU providing 48 GB GDDR6 GPU memory. The CVAI inference pipeline was executed on the RTX A6000 GPU using FP32 precision. The host system operated under Ubuntu 22.04 LTS with NVIDIA IGX-SW 1.1.2, while the CVAI application was deployed in a Docker environment using CUDA 13.0, TensorRT 10.13, and NVIDIA Holoscan SDK 3.6.1. Intraoperative video was acquired at a resolution of 1920 × 1080 pixels and 24 frames per second (fps), with the CVAI system operating in real time at the native acquisition rate. For intraoperative deployment, this system was integrated with existing operating theatre equipment, with the live endoscopic video feed from the surgical stack (Karl Storz, Tuttlingen, Germany) captured via an AJA KONA HDMI capture card, set up to use Remote Direct Memory Access to bypass the CPU and transfer video data directly into the GPU memory of the NVIDIA IGX Orin. This configuration is not dependent on external network connectivity or cloud infrastructure, which is often underdeveloped in hospital systems (Fig. 1). This results in high-throughput, low-latency video display of AI-generated anatomical overlays onto an existing secondary display monitor of the surgical stack, with an end-to-end processing latency of 50-100ms across pre-op testing. End-to-end processing latency was assessed at an application-level, using the Data Flow Tracking functionality of the NVIDIA Holoscan SDK. This tool tracks messages through the application graph and measures the elapsed time from video acquisition through preprocessing, neural network inference, post-processing, and visualisation. Importantly, the surgeon-AI interface was designed based on prior surgeon feedback during structured usability and interpretability digital simulations and real-time physical simulation studies [11, 12, 14, 15, 28]. The interface consisted of three key design features: (1) predicted pixel-level anatomical structures were rendered as outlines rather than filled masks, balancing informational salience with cognitive load; (2) activation of the CVAI overlay was placed under full surgeon control via a foot pedal, ensuring no unsolicited display of CVAI output during the procedure; and (3) a quantitative confidence metric, reflecting the model’s predictive certainty for structure segmentation, was displayed alongside the overlay to support informed intraoperative interpretation. This was calculated by the system by averaging the model-predicted probabilities across the region classified as the sella and the carotids separately. This was performed at 24fps (i.e., ~ 42ms intervals) and to avoid rapid fluctuations between consecutive frames, the displayed value was temporally smoothed. When activated using the second foot pedal, the confidence score was shown on the operative monitor as a percentage between 0 and 100 next to the anatomical label (e.g. “Sella: 92%”), while the segmentation outline remained overlaid on the endoscopic image (Fig. 2). The confidence display could be switched on or off independently. The score was intended as an indication of the model’s confidence in the displayed segmentation rather than a direct measure of segmentation accuracy. Fig. 1System set-up, with dataflow from the primary monitor into the NVIDIA IGX Orin computer, rather than from the video processing device, to avoid compromising the primary surgical feed. This is followed by real-time AI analysis by the deployed model in the IGX, with display of the IGX output on the in-built secondary monitor of the endoscope stack. This monitor is otherwise largely unused in our surgical practice (see IDEAL Stage 1 section) System set-up, with dataflow from the primary monitor into the NVIDIA IGX Orin computer, rather than from the video processing device, to avoid compromising the primary surgical feed. This is followed by real-time AI analysis by the deployed model in the IGX, with display of the IGX output on the in-built secondary monitor of the endoscope stack. This monitor is otherwise largely unused in our surgical practice (see IDEAL Stage 1 section) Proof-of-concept study procedure The aim of this study is demonstrating proof-of-concept, assessing feasibility, identifying safety concerns and gathering surgical team feedback to drive iterative development of the technology. In line with IDEAL Stage 1 recommendations, the sample was a small, pre-specified development cohort rather than a powered sample. Therefore, six adult patients undergoing elective endoscopic transsphenoidal surgery for pituitary adenomas were planned for enrolment. Exclusion criteria included: recent revision surgery (within 6 months), age < 18 years old, or surgeries requiring non-standard approaches (i.e. cavernous sinus exploration or extended endonasal surgery). All cases were performed at NHNN by a single attending surgeon (HJM) and their team. This team was orientated to the system using a high-fidelity simulator, with a standardised briefing on intended use, output interpretation, and the limitations of the system [11]. In the operating room, surgery was performed as per routine using the primary unchanged surgical display, with the AI-assistance system available on the secondary monitor. This AI-enabled display remained off by default until activated by the surgical team using the foot pedal during anatomical orientation steps [19, 29]. Immediately before each case, a standardised functionality check was undertaken to confirm correct video input, system start-up, overlay rendering, user control operation, and acceptable real-time responsiveness. If this functionality check failed, this was recorded and a root cause analysis performed. Data collection, analysis and system iteration Primary outcomes, measured per case, included:Demonstrating proof-of-concept: system functioned as intended, surgeon perceived latency, surgeon-AI alignment on anatomical structure location, system used by user as intended. These were measured via intra-operative observation and structured note taking (by a computer science postdoctoral research fellow); video recording of the surgical video feed, AI overlays and operating theatre (via the Medtronic TouchSurgery and LiveStream ecosystem). Perceived latency at the system-level between both monitors was assessed subjectively via questionnaire, and if noticeable, was explored objectively (e.g. via application-level end-to-end latency assessment as described above and root cause analysis).Assessing feasibility of deployment: measured via Feasibility of Intervention Measure questionnaire (FIM) and an open-ended question on how to improve it - judged by surgical team members who interacted with the system [30].Observation of safety: monitoring for primary surgeon distraction and surgical team workflow disruption (measured via event counting on review of operating room video recording) and deviation of clinical performance from the mean of a matched retrospective cohort (measured via blinded OSAT and operative time assessment of recorded surgical video feed and clinical outcome data) [31–33].Secondary outcomes were also assessed per case, by those who interacted with the system (surgeons & nurses), and included: (1) educational yield (measured via a structured questionnaire based on the Kirkpatrick model); (2) wider user feedback (via Acceptability of Intervention Measure, AIM; System Usability Scale, SUS; Short Form Trust in Automation Scale, STIAS, structured questionnaires) [30, 34–36]. Demonstrating proof-of-concept: system functioned as intended, surgeon perceived latency, surgeon-AI alignment on anatomical structure location, system used by user as intended. These were measured via intra-operative observation and structured note taking (by a computer science postdoctoral research fellow); video recording of the surgical video feed, AI overlays and operating theatre (via the Medtronic TouchSurgery and LiveStream ecosystem). Perceived latency at the system-level between both monitors was assessed subjectively via questionnaire, and if noticeable, was explored objectively (e.g. via application-level end-to-end latency assessment as described above and root cause analysis). Demonstrating proof-of-concept: system functioned as intended, surgeon perceived latency, surgeon-AI alignment on anatomical structure location, system used by user as intended. These were measured via intra-operative observation and structured note taking (by a computer science postdoctoral research fellow); video recording of the surgical video feed, AI overlays and operating theatre (via the Medtronic TouchSurgery and LiveStream ecosystem). Perceived latency at the system-level between both monitors was assessed subjectively via questionnaire, and if noticeable, was explored objectively (e.g. via application-level end-to-end latency assessment as described above and root cause analysis). Assessing feasibility of deployment: measured via Feasibility of Intervention Measure questionnaire (FIM) and an open-ended question on how to improve it - judged by surgical team members who interacted with the system [30]. Assessing feasibility of deployment: measured via Feasibility of Intervention Measure questionnaire (FIM) and an open-ended question on how to improve it - judged by surgical team members who interacted with the system [30]. Observation of safety: monitoring for primary surgeon distraction and surgical team workflow disruption (measured via event counting on review of operating room video recording) and deviation of clinical performance from the mean of a matched retrospective cohort (measured via blinded OSAT and operative time assessment of recorded surgical video feed and clinical outcome data) [31–33]. Observation of safety: monitoring for primary surgeon distraction and surgical team workflow disruption (measured via event counting on review of operating room video recording) and deviation of clinical performance from the mean of a matched retrospective cohort (measured via blinded OSAT and operative time assessment of recorded surgical video feed and clinical outcome data) [31–33]. Baseline characteristics of the patient (age, sex, tumour type and size) and of the surgical team members that interacted with the system (age, sex, level of training, AI literacy via the Meta AI Literacy scale short form; MAILS) were collected [37]. Clinical outcomes were collected via our existing internal prospectively maintained database, which aligns with a previous multicentre collaborative study on patient outcomes. It includes length of stay, dysnatraemia, pituitary hormone dysfunction, cerebrospinal fluid leak, visual loss, resection completeness, and biochemical remission (functioning tumours only) [38]. Data analysis included summary statistics of quantitative data (surveys and video analysis) and thematic analysis of qualitative data (observational notes and qualitative survey data) into actionable system iteration items. These items were organised into AI and software, hardware, user interface, and intra-operative implementation domains, forming a feedback loop to drive system improvements between operating sessions [39]. All changes made were logged and AI models were version controlled. Summary statistics were generated for baseline characteristics, wider user feedback and clinical outcomes. Results Overview Eight consecutive patients meeting the inclusion criteria were recruited to the study, across five operating lists. Successful proof-of-concept of the CVAI system was demonstrated in six cases (Fig. 2), with failure pre-operatively in two cases (case 2 and 3, on the same day, due to the same computer reboot bug). Four surgical team members interacted with the system across these cases – three surgeons (1 attending, 1 senior fellow, 1 resident) and one scrub nurse (with weekly pituitary surgery exposure). Baseline characteristics of the group were as follows: age (3/4 35-44 years old, 1/4 25-34 years), sex (3 males, 1 female), and AI literacy (MAILS median 6.1, IQR 5-7.1; max possible score of 10). Meaningful use of the system was limited to the sellar phase, as intended, during anatomical orientation just prior to the sellotomy and durotomy steps, when other navigational assisting tools (image guidance and doppler) are routinely used. Fig. 2Deployment of the system in real time during endoscopic pituitary surgery. A: Intra-operative deployment. B: First case with sella-only segmentation. C: With addition of carotid artery segmentation. D: Distortion of overlay by instrument Deployment of the system in real time during endoscopic pituitary surgery. A: Intra-operative deployment. B: First case with sella-only segmentation. C: With addition of carotid artery segmentation. D: Distortion of overlay by instrument Feasibility Feasibility was high across the cases series, with the median of mean participant FIM scores at 4.8 (IQR 4.3–4.8; max possible score of 5) (Table 1), with the only recommendation to improve feasibility being to deploy the model directly through existing surgical stacks rather than via a separate computer, which is currently not possible due to stack manufacturer configuration. Safety There were no events of significant primary surgeon distraction or surgical team workflow disruption in the current implementation (Table 1). Anecdotally, there was increased attention and curiosity toward the device in the first trial (likely due to novelty) and on the fourth trial (when the carotid segmentation was added). There was perceived latency between the primary and secondary monitor on subjective binary report by surgeons in one case, which had application-level latency speeds ~ 100ms, likely due to prolonged continuous use of the computer across two operations, the second of which was a revision case and longer than average. However, this did not appear to impact safety, feasibility, usability (case-specific SUS median 82 vs. median 78) or acceptability (case-specific AIM median 4.1 vs. case series median 4), and resolved on next use with resting of the system (Table 1). Surgical performance review via OSATS revealed a mean score of 27.2/30 ± 1 SD in the sellar phase, which did not meaningfully decline from the retrospective overall score and sellar phase score of 28.6/30 ± 0.8 (Supplementary Material 1) [32]. There were no intra-operative complications in this case series – including vascular or neural tissue injury. The median length of stay was 4 days. Three patients experienced anterior pituitary hormone deficiency requiring supplementation prior to discharge, and zero patients experienced posterior pituitary deficiency requiring supplementation. Two patients experienced hyponatraemia requiring treatment (case number two with no CVAI used, case number five with CVAI use). Both Cushing’s disease cases were cyclical Cushing’s and therefore assessment of remission will require longer term follow-up. Again, these were in line with retrospective cohort data and the wider literature (Supplementary Material 1) [32, 40]. Table 1Overview of proof of concept, feasibility and safety metrics. NFPA = non-functioning pituitary adenoma; GTR = gross total resection; STR = subtotal resection; SIADH = syndrome of inappropriate ADH. Colour coding as per operating session number. * changes between AI model versions are described in Fig. 3CaseOperating sessionClinical contextCVAI useAccuracy & surgeon-AI alignmentSurgeon Perceived LatencyMedianFeasibility (FIM) ScoreEvents of operating surgeon distractionEvents of workflow disruptionSurgical performance(mean sellar phase OSAT)Clinical Outcomes 1 1 48 MNFPAMacroadenomaYes; sellar phaseAI Model V1*: Sella only AI model, with false positive over carotidNo4.25(IQR 4.1–4.6)0029/30• No surgical complications.• GTR achieved. 2 2 26 MAcromegalyMacroadenoma with cavernous sinus invasionNo; reboot malfunctionN/AN/AN/AN/AN/A26/30• No surgical complications.• SIADH requiring treatment.• Cure achieved. 3 2 52 FNFPAMacroadenomaNo; reboot malfunctionN/AN/AN/AN/AN/A28/30• No surgical complications.• GTR achieved. 4 3 24 FCyclical Cushing’s diseaseMRI negative for lesionYes; sellar phaseAI Model V2: Sella segmentationaccurate; only centre of carotid prominence segmentedNo4.5(IQR 4.1–4.6)0028/30• No surgical complications.• Cure assessment requires follow-up. 5 4 77 FNFPAMacroadenoma with cavernous sinus invasionYes; sellar phaseAI Model V2.1: Sella & carotid segmentation satisfactoryNo4.9(IQR 4.8–4.9)0026/30• No surgical complications.• Optic decompression achieved via planned STR.• SIADH requiring treatment. 6 4 41 MRecurrent Cyclical Cushing’s disease with two previous transsphenoidal surgeriesMicroadenomaYes; sellar phaseAI Model V2.1: Sella & carotid segmentation only partially correctYes0027/30• No surgical complications.• Cure assessment requires follow-up. 7 5 51 FRecurrent NFPA Macroadenoma with cavernous sinus invasionYes; sellar phaseAI Model V2.2: Only sella was visible; segmentation centrally accurate despite distorted anatomyNo4.8(IQR 4.8–4.8)0026/30• Optic decompression achieved via planned STR.• No surgical complications 8 5 45 MNFPAGiant macroadenomaYes; sellar phaseAI Model V2.2: Sella segmentation accurate; carotid more accurate and consistent than last sessionNo0027/30• GTR achieved.• No surgical complications Table 1Overview of proof of concept, feasibility and safety metrics. NFPA = non-functioning pituitary adenoma; GTR = gross total resection; STR = subtotal resection; SIADH = syndrome of inappropriate ADH. Colour coding as per operating session number. * changes between AI model versions are described in Fig. 3CaseOperating sessionClinical contextCVAI useAccuracy & surgeon-AI alignmentSurgeon Perceived LatencyMedianFeasibility (FIM) ScoreEvents of operating surgeon distractionEvents of workflow disruptionSurgical performance(mean sellar phase OSAT)Clinical Outcomes 1 1 48 MNFPAMacroadenomaYes; sellar phaseAI Model V1*: Sella only AI model, with false positive over carotidNo4.25(IQR 4.1–4.6)0029/30• No surgical complications.• GTR achieved. 2 2 26 MAcromegalyMacroadenoma with cavernous sinus invasionNo; reboot malfunctionN/AN/AN/AN/AN/A26/30• No surgical complications.• SIADH requiring treatment.• Cure achieved. 3 2 52 FNFPAMacroadenomaNo; reboot malfunctionN/AN/AN/AN/AN/A28/30• No surgical complications.• GTR achieved. 4 3 24 FCyclical Cushing’s diseaseMRI negative for lesionYes; sellar phaseAI Model V2: Sella segmentationaccurate; only centre of carotid prominence segmentedNo4.5(IQR 4.1–4.6)0028/30• No surgical complications.• Cure assessment requires follow-up. 5 4 77 FNFPAMacroadenoma with cavernous sinus invasionYes; sellar phaseAI Model V2.1: Sella & carotid segmentation satisfactoryNo4.9(IQR 4.8–4.9)0026/30• No surgical complications.• Optic decompression achieved via planned STR.• SIADH requiring treatment. 6 4 41 MRecurrent Cyclical Cushing’s disease with two previous transsphenoidal surgeriesMicroadenomaYes; sellar phaseAI Model V2.1: Sella & carotid segmentation only partially correctYes0027/30• No surgical complications.• Cure assessment requires follow-up. 7 5 51 FRecurrent NFPA Macroadenoma with cavernous sinus invasionYes; sellar phaseAI Model V2.2: Only sella was visible; segmentation centrally accurate despite distorted anatomyNo4.8(IQR 4.8–4.8)0026/30• Optic decompression achieved via planned STR.• No surgical complications 8 5 45 MNFPAGiant macroadenomaYes; sellar phaseAI Model V2.2: Sella segmentation accurate; carotid more accurate and consistent than last sessionNo0027/30• GTR achieved.• No surgical complications Overview of proof of concept, feasibility and safety metrics. NFPA = non-functioning pituitary adenoma; GTR = gross total resection; STR = subtotal resection; SIADH = syndrome of inappropriate ADH. Colour coding as per operating session number. * changes between AI model versions are described in Fig. 3 48 M NFPA Macroadenoma 4.25 (IQR 4.1–4.6) • No surgical complications. • GTR achieved. 26 M Acromegaly Macroadenoma with cavernous sinus invasion • No surgical complications. • SIADH requiring treatment. • Cure achieved. 52 F NFPA Macroadenoma • No surgical complications. • GTR achieved. 24 F Cyclical Cushing’s disease MRI negative for lesion 4.5 (IQR 4.1–4.6) • No surgical complications. • Cure assessment requires follow-up. 77 F NFPA Macroadenoma with cavernous sinus invasion 4.9 (IQR 4.8–4.9) • No surgical complications. • Optic decompression achieved via planned STR. • SIADH requiring treatment. 41 M Recurrent Cyclical Cushing’s disease with two previous transsphenoidal surgeries Microadenoma • No surgical complications. • Cure assessment requires follow-up. 51 F Recurrent NFPA Macroadenoma with cavernous sinus invasion 4.8 (IQR 4.8–4.8) • Optic decompression achieved via planned STR. • No surgical complications 45 M NFPA Giant macroadenoma • GTR achieved. • No surgical complications Device iteration From surgical team feedback, intra-operative observation and surgical video review, problems were identified and addressed in a targeted fashion, as shown in Fig. 3. Solutions suggested after the last operating session (operating session 5) will be attempted prior to IDEAL Stage 2a. In two cases (cases 2 & 3), CVAI deployment was prevented by the same boot-configuration issue on the IGX Orin. Following a reboot, the system attempted multiple unavailable network boot options after failing to complete the normal Ubuntu boot process. Following discussion with the manufacturer and inspection of the UEFI configuration, the redundant network boot entries were removed from the active boot sequence, Ubuntu was retained as the primary boot option, and platform firmware was updated. The issue did not recur in subsequent cases. This was a platform-level fault rather than a failure of the CVAI application. Fig. 3Overview of iterative changes made between operating sessions to the system. Carotid segmentation inconsistency manifested as false positive errors, false negative errors, and resulting frame-by-frame volatility despite smoothing Overview of iterative changes made between operating sessions to the system. Carotid segmentation inconsistency manifested as false positive errors, false negative errors, and resulting frame-by-frame volatility despite smoothing Wider implementation feedback and human factors All participants saw potential as a valuable educational tool, particularly for surgeons and nurses in training. Acceptability was high across the cases series, with the median of mean participant AIM scores at 4 (IQR 3.8–4.2; max possible score of 5), with critique directed toward AI model accuracy at carotid arteries and at occlusion or distortion of AI overlay when instrument in field. Usability was also moderately high, with a SUS median score of 78 (IQR 65–79; max possible score of 100). Early trust in the system was moderate with a STIAS median of 4.3 (IQR 3.8–4.9; max possible score of 7). Discussion Principal findings This first-in-human IDEAL Stage 1 clinical evaluation demonstrates that deploying a real-time pixel-level CVAI anatomical segmentation system during endoscopic pituitary surgery is feasible, and early-stage implementation limitations were largely correctable. The system was successfully deployed in six of the eight planned cases, providing real-time sella and carotid pixel-level segmentation without introducing measurable distraction to the primary surgeon or disrupting operating room workflow. The translation of CVAI from digital and simulated environments to the live neurosurgical operating room presents significant logistical and technical challenges. Iterative system development across the full ecosystem (AI model, user interface, computing hardware and implementation) was driven by deployment experience and user feedback during this case series of standard and more challenging edge-cases. Most notably, the initial model generated false-positive sella (“go-zone”) segmentations over the carotid (“no-go”) regions and exhibited overlay distortion when surgical instruments occluded the operative field. Instrument occlusion and class imbalances are widely recognised as persistent challenges in real-time surgical segmentation, frequently compromising boundary-aware metrics during in vivo deployment [8, 21]. By moving beyond dataset-level AI accuracy metrics, and instead examining clinically significant and reproducible AI errors, we have employed a targeted and iterative strategy which progressively reduced these errors as the case series progressed. Findings in the context of the literature The current literature in surgical data science predominantly features retrospective AI models reporting high segmentation accuracies on curated, idealised datasets [21, 41]. Recent evidence suggests only 12% (n = 13) of surgical CVAI studies report the real-time capability necessary for live deployment in operating rooms [21]. Only a handful of studies have reached prospective early clinical stages (IDEAL Stage 1 & 2a), but mostly for general surgical applications - for example, laparoscopic cholecystectomy (anatomical segmentation of the “critical view of safety”) and during laparoscopic colorectal surgery (nerve recognition) [1–3]. CVAI has been deployed in the form of bounding box based detection during pituitary surgery [42], however, this technique does not precisely localise surgical structures at a pixel-level, and therefore likely has limited impact potential for navigation and training. Furthermore, this study primarily reported technical feasibility (e.g. latency) rather than a clinical evaluation of AI system use, human factors and outcomes. CVAI models that perform pixel-level segmentation are more difficult to train and are more computationally intensive, but offer the potential for precise surgical structure localisation, and thus harbour more potential for clinical impact. However, pixel-level anatomical segmentation of surgical video is yet to be translated in live neurosurgery, a specialty characterised by the potential for significant morbidity and mortality [21]. Translation in such high stakes environments requires a proportionally rigorous approach to development and evaluation [43]. In this work, we have performed preceding multifaceted preclinical evaluations, and now present a systematic iterative approach to clinical evaluation, which goes beyond crude AI model accuracy metrics and includes deliberate user-centred design and human factors analysis [11–16, 28]. This is reflected in the principles of the IDEAL and DECIDE-AI frameworks, and thus this present study demonstrates an exemplar for the systematic preclinical development and early clinical evaluation for higher stakes surgeries – such as most neurosurgery and cardiothoracic procedures [17, 18]. For such applications, it is unlikely that CVAI will operate in isolation, and future generations should integrate further with the wider operating room. For example through multimodal analysis of video, image-guidance and other adjuncts, when AI model and computing advances eventually allow this. Strengths and limitations The strengths of this study are the adherence to DECIDE-AI and IDEAL guidelines, with a multifaceted evaluation of system deployment, including human factors analysis, with the transparent reporting of iterative system failures and necessary improvements. Limitations include the early stage of this clinical evaluation (single centre, single surgical team, small case series). In order to drive iterative system improvement, AI-surgeon alignment focussed on assessments of AI errors (particularly false positives), with comparatively less attention given to instances where the model potentially outperformed humans or positively changed their decision. Studies of effectiveness when such systems are more mature, should encompass both. Furthermore, given the high-risk nature of pituitary surgery and the early stage of CVAI integration, the system was initially deployed as a training and educational tool rather than a primary decision-support system. Future iterations will provide direct intra-operative decision support, contingent upon the system meeting benchmarks for accuracy and consistency. Continual expansion of the training dataset (particularly for structures which are not always readily visible such as the carotid arteries; cases with challenging anatomy such as revision cases and invasive tumours; and across centres with varying techniques) and improvements to model architecture will be key development areas. Once addressed, future work will necessitate larger case series (IDEAL Stage 2a) and prospective multicentre evaluations (IDEAL Stage 2b) to further improve the technology and implementation, and explore its benefit and risk profile at scale. These studies of improved system versions should include exploration of trust and reliance calibration with AI confidence metrics; learning curves; and further understanding of surgeon-AI interaction after the initial curiosity-related use passes. Conclusion This first-in-human evaluation establishes that real-time, pixel-level, edge-computed CVAI anatomical segmentation can be feasibly deployed in the neurosurgical operating room without measurable distraction to the operating surgeon, disruption to team workflow, or AI-attributable complications across the deployed cases. The iterative design of this study identified correctable failure modes which were addressed between operating sessions through targeted software, hardware, interface, and implementation changes. The next steps are a larger single-centre case series with additional surgical teams (IDEAL Stage 2a) to confirm reproducibility and refine the system further, followed by multicentre evaluation (IDEAL Stage 2b). As the underpinning AI models improve and integrate with other intra-operative navigational technologies, such tools will likely be the cornerstone of future intra-operative surgical decision support systems. Electronic Supplementary Material Below is the link to the electronic supplementary material. Supplementary Material 1 Supplementary Material 1 Supplementary Material 2 Supplementary Material 2