Mixed reality applied to surgical planning and tailoring of carotid endarterectomies Authors: Elisa Colombo, Marcella Cavelli, Luca Regli, Jorn Fierstra, Martina Sebök, Menno R Germans, Tristan van Doormaal, Giuseppe Esposito Published: 2026 ABSTRACT AbstractPurposePerioperative three-dimensional (3D) anatomical visualization with mixed reality (MxR) may have a relevant impact on adequate carotid endarterectomy (CEA) planning compared to standard imaging studies, by highlighting patient specific anatomical features and allowing the rehearsal of the relevant surgical steps in 3D.MethodsAll consecutive CEAs planned with MxR between January 2022 and July 2024 were prospectively included. Using the preoperative head and neck CT-angiography (CTA) and/or MR-angiography (MRA), skin, skull, cervical spine and vascular anatomy including plaques were segmented. The resulting 3D interactive reconstructed object was matched on the patient with MxR glasses in the operating room after patient positioning and drawing initial skin incision. The impact of 3D visualization on surgical planning and anatomical understanding was documented and usability testing was performed.ResultsForty-nine CEAs were included. Mean 3D reconstruction preparation time was 17 ± 8 min. Mean duration of intraoperative 3D reconstruction use was 5 ± 2 min. The supervising surgeon significantly altered (shortening) the skin incision in 16 cases (33%). Higher scores were given to the domains Ease of Use and Satisfaction.ConclusionsCEA planning with MxR seems to have an impact on anatomical understanding for both less experienced neurosurgeons and highly experienced neurosurgeons performing more than 25 CEAs per year. Future research should focus on quantification of the benefits in surgical training and assess the impact on surgical outcomes.Keywords: Mixed reality, Carotid endarterectomy, Surgical revascularization, Cerebral ischemia, 3D visualization, Surgical planning Introduction Carotid endarterectomy (CEA) represents the first line of treatment for high-grade symptomatic and asymptomatic carotid artery stenosis worldwide [4, 8]. This surgical procedure has been introduced in the 1950 s and has been developed and extensively studied worldwide. The optimal performance of CEA requires experience and training as well as a precise knowledge and understanding of the patient neck anatomy [12]. The carotid bifurcation lies normally at the level of the vertebral bodies of C3-C4 [1]. However this is highly variable and cases of high or low bifurcation must be carefully examined to choose the best therapy (surgery or endovascular, in case of very high or very low carotid location of the plaque) [12, 14]. Also vessels’ tortuosity (i.e.: loop or kinking) or anomalies (i.e.: side-by-side carotid) and variations in its branching pattern can occur. Surgeons should take these variations into consideration for surgical planning. However, all these variations may not be adequately projected in three dimensions (3D) by viewing preoperative imaging on two-dimensional monitors and screens as compared to interactive and immersive 3D visualization [1]. Mixed Reality (MxR) could be a suitable technology for better 3D visualization and anatomical understanding and could lead to better surgical planning as well as intraoperative visualization of the patient-specific anatomy and its role has been established in cranial and spinal Neurosurgery [3]. MxR is a form of Augmented Reality (AR) featured by the overlay of a 3D stereoscopic interactive reconstruction of the patient anatomy over the real world’s anatomy using a head mounted display (HMD), thus without losing the perception of the reality around the 3D object. This connection substantially augments not only the anatomical appreciation, but also the spatial orientation and comprehension of the anatomical relationships among healthy and pathologic structures [9]. Therefore, MxR could optimize the surgical planning, increase intraoperative efficiency, decrease intraoperative adverse events and improve surgical outcomes. The authors present in this study the experience gathered on MxR application for carotid endarterectomies (CEAs), focusing on the impact on patient-specific anatomical understanding and surgical planning. Materials and methods The local review board approved the present study before its performance. All included patients signed an informed consent to give permission to the surgical procedure and to the publication of their clinical and radiological data under general KEK PB_2017_00093/NCT01628406 (Kanton of Zurich Ethical Committee). The present study has been performed in accordance to the principle of the Declaration of Helsinki. Patient clinical and radiological data Consecutive patients who underwent CEA surgery planned with MxR between January 2022 and July 2024 at our institution were included in the analysis. Surgical indication to CEA was given on the basis of ICA stenosis percentage, calculated according to the NASCET criteria, vessel angioanatomy and clinical presentation [5]. All the surgical indications were discussed interdisciplinary either at the cerebrovascular board of our Institution (neurologists, neurosurgeons and neurointerventionalists) or case by case. Preoperative and postoperative modified Rankin scale (mRS) and National Institute of Health Stroke Score (NIHSS) were collected for every patient. Each CEA was performed according to the state of art microsurgical technique [7, 13, 15, 18]. Using the preoperative imaging (head and neck CTA and/or MRA), data on plaque presence as well as its position and morphology, carotid bifurcation level (based on cervical vertebral bodies), anatomical variations (side-by-side carotid or abnormal ECA branch origin, i.e. anatomical variations of thyroid artery), carotid vessels’ tortuosity (kinking and loops) were recorded. Data on postoperative imaging control (CTA) performed within 48 h from the procedure and postoperative adverse events during the hospital stay were also gathered. Segmentation process and preparation of the 3D reconstruction Using the preoperative radiological studies as source imaging source in the form of Digital Images and Communication in Medicine (DICOM), the CCA, ECA and ICA, as well as the plaques presenting calcifications (when clearly evident) were semi-automatically and manually segmented. (3D Slicer image computing platform | 3D Slicer) Automatic segmentation of skull and cervical spine, as well as skin were performed in dedicated software (LumiNE, Augmedit bv, Netherlands), In this software a visualization scene was automatically deployed including source medical imaging and superimposed 3D segmentation [6, 16]. The software generates automatically a format viewable with a stereoscopic head up display (HoloLens 2, Microsoft, Richmond, USA). The total active time required for the 3D reconstruction preparation, i.e. the time demanded to perform manual and semi-automatic segmentations since the automatic segmentations did not require any active time, was recorded. All the stereoscopic reconstructions included in the present analysis were made by a neurosurgical resident with specific training on imaging segmentation and augmented reality. Intraoperative 3D reconstruction use Just before skin disinfection and draping, after patient positioning and a first planning of the skin incision, a manual matching between the 3D reconstruction, visualized through MxR glasses was performed (Fig. 1). Hereby anatomical landmarks (i.e. mandibular angle, sternal notch, external acoustic meatus and glabella) were used as anatomical landmarks for positioning. After the positioning, the 3D reconstruction was locked on the head of the patient thanks to a specific function provided by Lumi. Once the 3D reconstruction was locked on the head of the patient, it stabilized in that specific position and does not follow any change of the head, i.e. by modification of the head rotation or the height of the surgical bed, the 3D reconstruction had to be positioned again. Anatomical correspondence was verified by the supervising surgeon and eventually changed if deemed incorrect. The verification occurred by wearing the MxR glasses. Without wearing the HoloLens 2, the rest of the team could see how the surgical staff was maneuvering the 3D reconstructed object by means of a screen placed in the operating room and connected to the HoloLens 2. The screen was not placed in the room for all surgical procedures. The superimposition of the 3D reconstructed object on the surgical situs was utilized by the neurosurgeon to study the position of the carotid bifurcation and its relation to the incision made previously. Then each supervising surgeon was given the opportunity to shorten the surgical incision according to the 3D visualization and each modification was documented. After the usage of the 3D reconstruction, the operating neurosurgeon was questioned on the impact of the 3D visualization on the anatomical understanding and the use of the MxR system. The interview was performed on the following domains: Usefulness, Ease of Use, Ease of Learning, and Satisfaction. Table 1 describes in more details how the domains mentioned above were defined. Illustrative cases showing the identification of the carotid artery bifurcation and its level in correspondence to the cervical vertebra (a, b and c) and the extent of the plaque (a, b and c) or the stenosis (d) after intraoperative 3D reconstructed object-head matching. The bony structures (in white) may be visualized using different transparencies. The calcified parts (Hounsfield Units > 120) of the segmented plaques have been rendered in green, whereas the soft fatty component of the plaque, when visible in the preoperative CTA, were rendered in yellow (a) Table 1Definition of the evaluation domainsDomainDefinitionMeaning of a high scoreEase of UseHow naturally users can interact with the MxR systemThe interface feels straightforward and intuitiveEase of LearningHow quickly and comfortably users can get up to speed with the MR toolThe system can be mastered by those new to it with little effortSatisfactionHow users feel about their overall experience with the MR systemThe system allows the user to achieve the expected goals for its useUsefulnessHow valuable the MR system is, whether it adds meaningful supportThe technology is seen as genuinely helpful in surgical planning and anatomical understanding Definition of the evaluation domains Each domain was given a score from 1, lowest score, to 7, highest score; as score from 5 and above was deemed to be positive [11]. All included neurosurgeons had at least 6 months of exposure to the MxR system and their level of experience in CEA surgery was rated according to the annual number of surgeries performed as experienced (>/= 25 CEA/year as supervising surgeon) and inexperienced [2]. Statistical analysis and comparison with non-MxR CEAs A descriptive analysis was used to analyze the included prospective cohort of patients and characterize the application of MxR on surgical preparation. Data were expressed as percentages, means and standard deviations (SD). All statistical analyses have been performed with Excel 2016 (version 10.0.5491.1000) and R Studio (version 2023.09.0 + 463). Results Demographic and surgical data The study included forty-nine patients with a mean age of 75 ± 8 years. The cohort included 28 left (57%) and 21 right (43%) CEAs. Preoperatively, 38 patients (78%) presented symptomatic stenosis. All the patients but one, who demonstrated a NIHSS of 7 points, showed a NIHSS < 7, and 43 patients (88%) demonstrated a mRS ≤ 2. After surgery, all patients but one demonstrated no worsening of the scores. Forty-one patients (84%) received a preoperative CTA, 6 patients (12%) an MRA only and 2 patients (4%) underwent both imaging studies. Immediate postoperative imaging control with head and neck CTA was performed for 48 patients (98%), in one patient a doppler-ultrasound was performed instead. All patients showed a successful postoperative result with patent internal carotid artery. Preparation of the 3D reconstructions The mean time to achieve complete semi-automatic and manual segmentation was 17 ± 6 min. While for the majority of the included cases the arterial structures of interest were segmented semi-automatically (38 cases, 76%), the CCA, ICA and ECA were segmented manually in 12 cases (24%). The determination of the level of the bifurcation on the 3D reconstruction using the level of the vertebral bodies showed that the most represented bifurcation level was between C4 and C5 vertebral body (14 cases, 29%), followed by level C3-C4 (12 cases, 24%). In all cases included in the present cohort, the plaque’s localization included the level of the bifurcation. An overview of the steps of 3D reconstructed object preparation is illustrated in Fig. 2. Semiautomatic segmentation of the right CCA, ICA and ECA (green), as well as manual plaque segmentation (yellow) on 3D Slicer (a). 3D reconstruction rendering and visualization on LumiNe (b-c) Impact of intraoperative use Mean duration of intra-operative (i.e. during the non-sterile phase of the operation, after positioning and before disinfection and draping), 3D reconstruction use was 5 ± 2 min. Moderate kinking of the affected ICA was observed in 3 cases (6%): In one case (2%), a side-to-side anatomy was reported. Tailoring the skin incision, i.e. shortening it to focus on the position of the bifurcation, based on intraoperative 3D reconstruction visualization was performed in 16 cases (33%). After being questioned on their experience with MxR, by 3 of the 5 neurosurgeons the mean score of all the domains was above 5. The best scores were given to the domains Ease of Use and Satisfaction, while the worst score was given by an inexperienced surgeon in the domain Usefulness. More details on the results of the interviews are provided in Table 2. Table 2Summary of the interview’s results, and comparison between the results according to the experience of the users. Experienced surgeons are defined as those using the technology on a regular basis for longer than 6 months. Given the extremely small sample size and the limited resolution of non-parametric testing under these conditions, inferential statistics were deemed uninformative. Therefore, results are presented descriptively to more accurately reflect the observed data patternsParametersInexperienced Users Median (IQR)Experienced Users Median (IQR)Usefulness4.81 (4.75–4.88)3.69 (2.75–4.63)Ease of Use4.91 (4.91–4.91)3.73 (3.09–4.36)Ease of Learning6.63 (6.25–7.00)4.88 (4.50–5.25)Satisfaction5.50 (5.29–5.71)4.57 (4.14–5.00) Summary of the interview’s results, and comparison between the results according to the experience of the users. Experienced surgeons are defined as those using the technology on a regular basis for longer than 6 months. Given the extremely small sample size and the limited resolution of non-parametric testing under these conditions, inferential statistics were deemed uninformative. Therefore, results are presented descriptively to more accurately reflect the observed data patterns Illustrative case with ordinary anatomical features A 68-years-old male patient presenting with a transitory ischemic attack (TIA) showed a 50%-stenosis of the right ICA at a Doppler-Ultrasound. A preoperative head and neck CTA documented a carotid bifurcation at C3 vertebral body level and a partially calcified plaque localized at the level of the CCA bifurcation and at the origin of the ICA. Starting from the preoperative CTA as source imaging, a holographic rendering of the plaque and of the CCA, ICA and ECA, as well as the single thyroid artery was performed. After intraoperative patient positioning, the 3D reconstructed object was manually matched with the patient’s head and neck providing a 3D visualization of the carotid bifurcation and plaque, confirming the correct incision planning. The anatomical correctness of the matching of the 3D reconstructed object, the incision planning, the holographic plaque extension and the origin of the thyroid artery was then confirmed intraoperatively after reaching the carotid sheath and opening the CCA and ICA (Fig. 3). Intraoperative 3D visualization after patient-positioning, head-3D reconstructed object matching and surgical incision planning (a). Intraoperative visualization of the plaque after clamping of the ICA (white star), ECA and thyroid artery (black and white arrow respectively) and CCA (black star) (b) Illustrative case with complex anatomical features The case of a 78-years-old female patient was presented to the institutional interdisciplinary vascular board to discuss the indication of a left CEA. She presented a fluctuating paresis of the right hemibody as well as right hemodynamic watershed strokes on CT and an 80%-stenosis of the left ICA at Doppler-Ultrasound. The indication of a CEA over a stenting was given, driven by the presence of a floating thrombus at the level of the calcified plaque. The preoperative holographic reconstruction of the anatomy of the patient could demonstrate clearly plaque and thrombus, and it highlighted the presence of a severe kinking (110°) of the ICA distal of the stenosis (Fig. 4). Due to the relevant kinking of the ICA and with the aim to correct it surgically, the supervising surgeon decided to remove the plaque using the eversion technique. The eversion of the stenotic left ICA allowed indeed a complete removal of the plaque and a correction of the kinking. Preoperative (a to c) and postoperative (d) identification of the major pathoanatomical patient-specific features driving surgical planning and intraoperative decisions. Preoperative CTA (a) and its 3D reconstruction (b) showing the calcified plaque (red arrow), the thrombus (white triangle) and the kinking (yellow star). The same details visualized and augmented in the holographic reconstruction (c). The postoperative result (d) In this case, the interactive holographic reconstruction helped by increasing the anatomical appreciation of relevant patient-specific features and endorsed the intraoperative decision to perform the eversion technique. Discussion This study aimed to evaluate the applicability of MxR on preoperative surgical planning of CEA, anatomical understanding and surgical education. In our opinion, the added value of MxR lies in the direct transfer of 2D anatomical details in 3D and the overlay of the stereoscopic 3D reconstruction on the surgical field. This overlay has the potential to enhance the intraoperative identification of relevant morphological details, i.e. height of the bifurcation, loops or kinking, twisted anatomy, as well as the morphology of the plaque and its extension. The impact of an improved preoperative visualization may be relevant on the effectiveness of the surgical isolation of the carotid sheath and identification of the bifurcation and ICA. The improved anatomical appreciation may be beneficial for less experienced neurosurgeons and but also for expert ones. Less experienced surgeons could profit from the 3D visualization to increase spatial orientation improving their intraoperative understanding. On the other hand, experienced ones may benefit from the clearer depiction of details to further improve and tailor their surgical strategy. Furthermore, the relatively easy accessibility of the system could thereby increase its use for this indication. In fact, while HoloLens 2 and the yearly license to use LumiNe (which covers 5 users) have costs, these can be diluted with the regular use of the system. Moreover, the training of the neurosurgical staff requires a day of work with technicians and/or super users of the system, which also contributes to ease the logistics of its implementation. Indeed, mastering the use of LumiNe and the crafting of the 3D reconstructions does need more practice to be achieved. This study is a first exploration of the effects of the technology and cannot be used to study reduction of complications or improvement of outcome. Despite the wide diffusion of MxR in the neurosurgical field and the presence of multiple studies endorsing the positive impact of this technology on anatomical comprehension, surgical planning and training, to our knowledge no study specific on MxR for CEA planning has been published yet. Preliminary experience on the potential positive impact of this technology on CEA was previously published by our group in a more comprehensive cohort of neurosurgical cases [14]. Neuronavigation technology or ultrasound have not found its way to this procedure because the absence of head fixation and a high cost/benefit ratio. In spite of its merits, the present study has several limitations. First, this study is an initial exploration of the potential benefits of MxR on CEAs and cannot be used to study reduction of complications or improvement of outcome. Furthermore, the nature of the present cohort, including a small number of cases from a tertiary referral center and no control group, may fail to clearly demonstrate the benefits of the application of the technology. A major theoretical limitation of the technology thereby is the absence of rotation in the3D reconstruction. Head rotation during operative positioning could provide a change in the intraoperative anatomy compared to the anatomy during scanning which is performed in straight supine position. The same considerations can be made for the impact of soft tissue deformation that differentiates the imaging with the real-life settings. These aspects could be corrected by incorporating heavier post-processing into the making of the 3D reconstructions using programs able to alter the 3D structure and model it. The integration of further modeling of the 3D reconstructed object represents a necessary future upgrade to widen and improve the use of the technology for CEAs, unfortunately its application is for now conjectural and lacks a concrete framework that can represent a concrete solution. Lack of non-operator dependence in the 3D reconstructions to head matching could also undermine the value of one of the main results of the study, namely the reduction in the length of the surgical incision. If on one side this finding may be beneficial for a faster healing of the wound, the manual matching process may reduce the surgical exposure if the incision does not adequately correspond to the position of the carotid bifurcation. However, we did not find these of relevant impact for the purposes of this study, given the use of the technology as preoperative planning aid and not intraoperative guidance. Matching was still possible on the combination of anatomical landmarks and the intraoperatively found level of bifurcation was found adequate to the surgical exposure in all cases.. Formal objective measurement of the registration error is therefore a good step for future research. Other intrinsic limitations could also limit its wide adaptation. Its workflow is influenced by the quality of the internet connection, thus, it may not be easy to implement in operating rooms. Also preoperative imaging quality impacts heavily 3D reconstructions quality. A bad scan will result in an ugly reconstruction. Furthermore, HoloLens 2 bears some in-built limitation due to the lack of further updates of the hardware, the most relevant of which is spatial drift that may occur due to a loss of accuracy of the tracking device over time [10, 17]. In order to overcome hardware limitations, we are now transitioning from HoloLens 2 to more developed MxR glasses. Future developments should focus primarily on the improvement and automatization of the MxR workflow dedicated to CEA procedures: the head to 3D reconstructed object matching should be made automatic and not operator-dependent, and the 3D reconstruction should become adaptable to the specific head and neck positioning. Furthermore, in order to increase the reproducibility and applicability of the results, a more numerous and diverse cohort of cases should be examined and matched with a control group. Moreover, the integration of hemodynamic parameters into the 3D rendering, in order to combine patient-specific 3D interactive anatomical information to vessel-specific volume flow rates, flow velocity turbulences and shear stresses should be investigated. MxR could then be integrated in the diagnostic workup for carotid artery stenosis and influence the therapeutic decision making. Another relevant application is the training of surgeons to perform CEAs. MxR could represent a valuable simulation model for CEAs giving the possibility of training different surgical scenarios in a non-threatening environment and allowing less experienced surgeons to appreciate the nuances and variations of the anatomy of the carotid artery at the neck. This role of MxR should be further explored. Conclusion CEA planning with MxR seems to have an impact on incision planning and preoperative anatomical understanding. The technology in the form presented in this study represents an initial application and needs indeed further refinement. Future research should also focus on quantification of the benefits in surgical training. Authors contributions Elisa Colombo: conceptualization, methodology, data curation, writing (original draft preparation and revision); Marcella Cavelli: data curation, writing (original draft preparation and revision); Luca Regli: supervision, review and editing of the manuscript; Jorn Fierstra: data curation, review and editing of the manuscript; Martina Sebök: review and editing of the manuscript; Menno Germans: data curation, review and editing of the manuscript; Tristan van Doormaal: supervision, conceptualization, methodology, review and editing of the manuscript; Giuseppe Esposito: supervision, conceptualization, methodology, review and editing of the manuscript. Funding The present study was performed as part of the PhD of Dr. med. E. Colombo and received funding through the SURGENT Consortium, University of Zurich (UZH). https://www.hochschulmedizin.uzh.ch/en/projekte/surgent.html Data availability The medical and radiological data collected for the performance of the present study can be shared by the corresponding author upon sensible request. Declarations Ethics approval and consent to participate The local review board approved the present study before its performance. All included patients signed an informed consent to give permission to the surgical procedure and to the publication of their clinical and radiological data under general KEK PB_2017_00093/NCT01628406 (Kanton of Zurich Ethical Committee). The present study has been performed in accordance to the principle of the Declaration of Helsinki. Consent for publication By signing the informed consent mentioned above, each patient was made aware and consented to the use of personal medical data for the purpose of scientific research and thus to the divulgation of the data to the scientific community. Competing interests Tristan van Doormaal is the Chief Medical Officer of Augmedit, and received a grant from Hanarth foundation, paid to UMC Utrecht. Giuseppe Esposito is a member of the Board of Trustees for the Brain Disease Foundation, and is a consultant for Aesculap B. Braun. Luca Regli owns shares of Augmedit. The other authors have no personal, financial, or institutional interest in any of the drugs, materials, or devices described in this article. Footnotes Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Tristan van Doormaal and Giuseppe Esposito contributed equally to this work. References Associated Data Data Availability Statement The medical and radiological data collected for the performance of the present study can be shared by the corresponding author upon sensible request.