Anatomical recognition artificial intelligence for identifying the recurrent laryngeal nerve during endoscopic thyroid surgery: A single‐center feasibility study Authors: Yukio Nishiya, Kazuto Matsuura, Tateo Ogane, Kazuyuki Hayashi, Yumi Kinebuchi, Hirotaka Tanaka, Wataru Okano, Toshifumi Tomioka, Takeshi Shinozaki, Ryuichi Hayashi Published: 2024 ABSTRACT AbstractBackgroundWe investigate the feasibility of using artificial intelligence (AI) to identify the recurrent laryngeal nerve (RLN) during endoscopic thyroid surgery and evaluated its accuracy.MethodsIn this retrospective study, we develop an AI model using a dataset of endoscopic thyroid surgery videos, including hemithyroidectomy procedures performed between April 2019 and September 2023 at the National Cancer Center Hospital East, Chiba, Japan. Semantic segmentation deep learning methods were applied to analyze the endoscopic thyroid surgery videos.ResultsForty endoscopic thyroid surgery videos, all in high definition or better quality, were analyzed. The Dice values were 0.351, 0.568, and 0.746 for the inferior thyroid artery, RLN, and trachea, respectively. Data augmentation was performed by cropping, standardizing, and resizing to reduce false positives and improve accuracy.ConclusionsThe AI model showed high recognition accuracy of the RLN and trachea. This method holds potential for assisting in future cervical gasless endoscopic surgeries.Keywords: artificial intelligence, deep learning, endoscopic thyroid surgery, recurrent laryngeal nerve, thyroidectomy INTRODUCTION Conventional thyroid surgeries were performed through an open incision. However, in recent years, the use of remote‐access endoscopic and robotic surgeries has increased. 1 The chest‐breast approach, 2 bilateral axillo‐breast approach, 3 retroauricular approach, 4 and transoral approach 5 are remote‐access approaches that do not require a cervical incision. Thyroid disorders and tumors, which are prevalent among young women, have facilitated a high demand for aesthetic treatments. Remote‐access approaches offer high cosmetic satisfaction as they avoid surgical scars in the neck area. 6 In contrast to open surgeries, good surgical videos can be easily recorded in endoscopic surgeries. In contrast, endoscopic thyroid surgery takes longer than open surgery 7 , 8 , 9 and is reported to be associated with a higher frequency of recurrent laryngeal nerve (RLN) palsy, 10 , 11 , 12 particularly in facilities with limited experience. Furthermore, endoscopic thyroid surgeries have a long learning curve, 13 , 14 , 15 , 16 making them challenging to perform safely and accurately in less‐experienced facilities. The high frequency of complications and the extended recognition challenges may be attributed to anatomical features. Unlike thoracic and abdominal cavities, the neck region lacks a cavity structure. Thus, surgeons must first create a working space to perform an endoscopic procedure. 13 However, the space made by the surgeon is collapsible and unfamiliar, increasing the complexity of recognizing the dissection. In areas related to the thyroid gland, many artificial intelligence (AI) diagnostic tools have been reported. Among these tools, the ultrasound is reported to be particularly useful in the diagnosis of benign and malignant. 17 Furthermore, it yields higher diagnostic accuracy than radiologists and ultrasonographers 18 , 19 and aids the assessment of genetic risk. 20 It has also been reported that ultrasound is useful for the diagnosis of benign and malignant disease. However, only a few have directly catered to surgical applications. Although endoscopic thyroid surgery is a complex technique, it provides helpful surgical videos that are challenging to produce from open cervical incisional surgeries. This is particularly useful for developing new technologies and techniques. The rapid and accurate recognition of the RLN is an essential and challenging task because this nerve is the deepest and most critical anatomical feature in this field, particularly in gasless procedures. The trachea serves as a landmark for evaluating the estimated location of the RLN, and the inferior thyroid artery is a vessel that crosses the RLN; these are important structures for the surgical procedure. To determine whether AI can assist with endoscopic thyroid surgery safely without complications within a reasonable operating time at facilities with insufficient experience, we developed an AI model capable of anatomical recognition from endoscopic thyroid surgery videos. We further investigated the feasibility of this tool in assisting cervical endoscopic surgery. MATERIALS AND METHODS Study design and patient cohort This retrospective study adhered to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines. The AI model was developed using a dataset containing all cases for which endoscopic thyroid surgery videos were available, including those of hemithyroidectomy performed between April 2019 and September 2023 at the National Cancer Center Hospital East (https://www.ncc.go.jp/en/ncce/index.html). Exclusion criteria included cases wherein (1) usable high‐definition surgical videos or those of better quality were unavailable, (2) the RLN could not be preserved intraoperatively, and (3) the patient refused to participate. In this study, the RLN is considered the most important anatomy. Cases where the RLN could not be preserved were excluded because the nerve would not be in its original correct position, which may reduce learning accuracy. Surgical approach At our institution, we recommend endoscopic thyroid surgery when a tumor is less than 50 mm without invasion outside the thyroid capsule or metastasis to the lateral compartment. Further, we offer patients the option of endoscopic thyroid surgery for benign tumors smaller than 50 mm or thyroid cancers without invasion outside the thyroid capsule or metastasis to the lateral compartment. All endoscopic surgeries were performed using video‐assisted neck surgery, 21 which uses an endoscopic approach through a small subclavian incision on the affected side. After separating the cervical subcutis, the cervical skin was lifted with a retractor, and a single camera port was placed in the cervical field. Then, the thyroid gland was approached from the lateral side by separating the sternohyoid and omohyoid muscles on the affected side, pulling the omohyoid and sternocleidomastoid muscles laterally with muscle hooks, and fixing the operative field. The sternothyroid muscle was ligated and resected from the thyroid gland. The perithyroidal vessels, including the superior and inferior thyroid arteries, were ligated using surgical clips or energy devices. Generally, identifying the RLN is crucial during surgery. Thus, intraoperative neuromonitoring (IONM) was intermittently used to positively identify the RLN and trace its path to the larynx to ensure its preservation. A standard OLYMPUS rigid endoscope with a diameter of 4 mm was used in all cases. Although this study includes surgical videos of three surgeons, the approach is the same for all. Annotation The objects of recognition included the RLN, trachea, and inferior thyroid artery. The surgical videos were converted into MP4 format at a frame rate of 59.94 fps and annotated using the Computer Vision Annotation Tool (Ver 1.7.0, Intel Corporation). One of the endoscopic thyroid surgery surgeons performed all annotation tasks. The region of interest in each frame was precisely enclosed and individually labeled. Annotations were created from frames in which the target anatomical structures were clearly recognizable to new frames wherein they were only slightly recognizable retrospectively. Annotations were performed on a frame each time a forceps maneuver was performed around the recognition target, the camera position was changed, or the view of the surgical field was changed. The RLN and inferior thyroid arteries were annotated with the main trunk and branched only once each. The inferior thyroid artery was annotated before ligation. This is because identifying the vessel before ligation is beneficial, while its relevance decreases after ligation, making subsequent annotation more challenging. Ethics The study protocol was approved by the institutional review board of the National Cancer Center Hospital East (registration no. 2022‐358). Informed consent was obtained from patients in the form of opt‐out decisions via the study website. Data from those who rejected participation were excluded. This study conformed to the provisions of the Declaration of Helsinki of 1964 (revised in Brazil in 2013). Model optimization and evaluation Surgical videos were annotated for model learning and validation. Deep learning semantic segmentation tasks were performed using a convolutional neural network (CNN)‐based approach. DeepLabv3+ 22 was used as the CNN model for the semantic segmentation task, and ImageNet (https://image-net.org/download.php) was used for pretraining. Additionally, the original and cropped images were learned and validated in each case. As shown in Figure 1, only the central region without an unnecessary black frame was extracted as a cropped image, whose input sizes were increased, and the aspect ratio was improved to 1:1. The images were standardized using mean subtraction = [133.579, 44.421, 64.584] and standardization = [47.897, 47.964, 45.970]. The trained model was evaluated in terms of precision, recall, and Dice score (F‐measure) using the five‐fold cross‐validation method. As shown in Figure 2A, the data were divided into five groups, from CV1 to CV5. In particular, CV1 was used as the testing data in the first validation, and CV2–CV5 corresponded to the training data. This process was repeated five times, from CV1 to CV5. The averages of the results were used to determine the model accuracy. where true positive (TP) refers to the annotated regions correctly predicted by the model, false positive (FP) refers to incorrect positive predictions, and false negative (FN) refers to correct regions that the model does not predict as a target region. The pixel‐level results entail the computation of the TP, FP, and FN values for individual pixels within each image. The object‐level results require calculating these metrics for each discernible object in an image. For the object‐level results, TPs were defined with an intersection‐over‐union (IoU) equal to or greater than 0.1. As shown in Figure 2B, the IoU is an index that evaluates the amount of error between the object area predicted by the object detection model and the correct area. Specifically, the IoU is the overlapping area divided by the union area. Here, more than 10% overlap was present between the annotated correct region and the region predicted by the model. A confusion matrix 23 is a table frequently used to describe the performance of a model on a test dataset for which the true values are known. In our case, a confusion matrix was built using object‐level results as per‐pixel results possess considerably large values, thus increasing the complexity of visual interpretations. Cropped images with increased input size. Borders were identified, unnecessary areas were removed, sizes were enlarged, aspect was set to 1:1. Fivefold cross‐validation and intersection‐over‐union. (A) fivefold cross‐validation and (B) intersection‐over‐union. RESULTS Eighty‐two of the 134 patients who underwent hemithyroidectomy during the study period underwent open surgery, and 52 underwent endoscopic thyroid surgery. Of the 52 who underwent endoscopic thyroid surgery, 11 could not be considered owing to low image quality, and one case failed to preserve the RLN. Ultimately, as presented in Table 1, the dataset comprised 40 cases. None of the patients refused to participate. Half of the cases were right‐sided, and more cases of malignant tumors than benign tumors were present. A total of 3319 frames and 5533 labels were annotated for the surgical videos of the 40 cases for training and validation. Table 2 demonstrates the details of fivefold cross‐validation. Table 3 presents the validation results regarding the dice, precision, and recall scores. A higher accuracy was obtained with the cropped image at both pixel and object levels. Anatomically, the trachea exhibited the highest accuracy, followed by the RLN. Moreover, the inferior thyroid artery exhibited the lowest accuracy. Figure 3 shows the confusion matrix of the training results obtained using the cropped and original images. Each label in the ground truth is an annotated region, and the background is an unannotated structure other than the target of this recognition. The part predicted by the AI model to be the target area for the background part is FPs, which decreased when cropped images were used. The ground truth and FP area distributions are illustrated in Figure 4. The peak area distribution of the ground truth was from 5000 to 10,000 pixels, and the FPs were mainly observed in small areas, especially those with less than 1000 pixels. TABLE 1Training and validation dataset patient characteristic.Cases40Age61.6 (42–84)GenderMale16 (40%)Female24 (60%)BMI23.2 (17.6–31.6)Surgical sideRight20 (50%)Left20 (50%)Pathological diagnosisBenign9 (23%)Adenomatous nodule5Cyst1Follicular adenoma3Malignant31 (76%)Papillary carcinoma27Follicular carcinoma3Medullary carcinoma1Tumor size20.1 (6–45) Note: Average (range) or frequency (%). Training and validation dataset patient characteristic. Note: Average (range) or frequency (%). TABLE 2Details of 5‐fold cross validation.CV1CV2CV3CV4CV5TotalCases7997840Frames5767616476556803319LabelsITA143154149166143755RLN5705505695645822835Trachea3913913764013841943Abbreviations: ITA, inferior thyroid artery; RLN, recurrent laryngeal nerve. Details of 5‐fold cross validation. Abbreviations: ITA, inferior thyroid artery; RLN, recurrent laryngeal nerve. TABLE 3Model accuracy based on average cross validation results.Pixel levelObject levelOriginalCroppedOriginalCroppedDiceITA0.2490.3510.1610.232Recurrent N0.5330.5680.5210.594Trachea0.7150.7460.4230.479PrecisionITA0.2260.3700.1020.165Recurrent N0.5040.5560.3930.484Trachea0.6330.6680.2820.330RecallITA0.2780.3350.3840.393Recurrent N0.5660.5800.7750.769Trachea0.8210.8450.8400.873 Note: Original refers to the training results obtained using the original image of the surgery, and cropped refers to the training results obtained using the cropped image.Abbreviations: ITA, inferior thyroid artery; RLN, recurrent laryngeal nerve. Model accuracy based on average cross validation results. Note: Original refers to the training results obtained using the original image of the surgery, and cropped refers to the training results obtained using the cropped image. Abbreviations: ITA, inferior thyroid artery; RLN, recurrent laryngeal nerve. Confusion matrix. (A) Original and (B) cropped. A decrease in false positives was observed during cropped image learning. Area distribution of (A) ground truth (GT) and (B) false positives (FPs). GT area peaked at 5–10k pixels; FPs were observed in smaller areas. DISCUSSION We developed an AI model that recognizes anatomical images critical to surgical execution during endoscopic thyroid surgery. AI is increasingly being used for anatomical recognition in the abdominal and thoracic cavities, where minimally invasive surgeries are widely used. 24 , 25 , 26 However, only a few reports have focused on implementing AI models with gasless cervical endoscopic surgeries. Hence, this study is groundbreaking. The three anatomies identified in this study included the RLN (i.e., the most important anatomical item to preserve during thyroid surgery), inferior thyroid artery, and trachea. Both the artery and trachea are crucial when searching for the RLN. The results show that the trachea exhibited the highest recognition accuracy, followed by the RLN, whereas the inferior thyroid artery had the lowest accuracy. The inferior thyroid artery is a vessel that must be ligated when the thyroid gland is removed, and it also helps identify the RLN. However, because the inferior thyroid artery is ligated intraoperatively, fewer training images are available than for the trachea and recurrent laryngeal nerves. ITA's recognition accuracy is low, so more annotation data is needed to improve it in the future. Additionally, because the endoscopic cutting site is on the peripheral side of the inferior thyroid artery, many variations are present, and the area is small. 27 Therefore, achieving accurate results is challenging. Moreover, the trachea has no anatomical variation and serves as an orientation for surgery. Hence, AI models can yield highly accurate recognition. However, the accuracy of the RLN was intermediate in our results, and this may be because the RLN possesses a smaller area than the trachea. We posit that one of the reasons for the lower accuracy of the RLN compared to the trachea may be its deeper spatial location, which makes it more susceptible to even minor bleeding. Furthermore, this nerve is more variegated than the trachea but less variable and linear than the inferior thyroid artery. Dice results for the RLN and trachea differed between the per‐pixel and per‐object results possibly because the per‐pixel results tended to have improved structures with larger areas. In our study, the trachea and RLN were recognized with good accuracy. The results are adequate compared to the accuracy reported in other surgical fields, such as the Dice pixel of the prostate (0.66) 24 azygos vein and vena cava (0.79), aorta (0.74), 25 and the RLN in the thoracic cavity (0.58). 26 These findings suggest that AI‐based image recognition technologies can be integrated in cervical endoscopic surgery using gasless methods. In the cervical region, Wang et al. 28 developed an AI model able to recognize parathyroid glands with high precision, recall rate, and F1 scores of 88.7%, 92.3%, and 90.5%, respectively. They also reported that the AI model outperformed junior and senior surgeons. 29 We suggest that, alongside the results of this study, the technology supporting thyroid endoscopic surgery will aid the implementation of this procedure safely and appropriately at any facility, thus increasing its utility. Furthermore, the application of this accurate anatomical recognition technology from the thyroid to the entire cervical region will contribute to endoscopic cervical dissections and eventually to the automation of surgery. The primary surgical assistance device used in thyroid surgery is IONM. 30 IONM helps monitor and explore the RLN by inducing general anesthesia with an intubation tube using electrodes and stimulating the nerve with a probe or applying an automatic periodic stimulation electrode to the vagus nerve. Although the effectiveness of IONM for the early identification of the RLN has been emphasized, it does not significantly affect the rate of temporary or definitive RLN injury following thyroidectomy. However, its use is recommended in selected cases and for preventing bilateral palsy, as reported by Cozzi et al. 31 By contrast, image recognition AI, which does not require preparation for monitoring and visually recognizes anatomies from endoscopic images or in combination with IONM, is expected to improve surgical support. In contrast, image recognition AI, which requires no preparation for monitoring and visually recognizes anatomy from endoscopic images, differs from IONM in terms of its operational mechanism. This may be problematic for future studies as it is expected to improve surgical support by AI alone or in combination with IONM. Although this study utilized datasets from subclavian approach, different approaches (e.g., the bilateral axillo‐breast approach endoscopic thyroidectomy (BABA‐ET), 10 minimally invasive video‐assisted thyroidectomy (MIVAT), 32 trans‐oral endoscopic thyroidectomy through the vestibular approach (TOETVA), 5 , 33 and retroauricular approach endoscopic thyroidectomy (RAET) 4 ) may involve different techniques for identifying the RLN and vary in difficulty. A systematic review by de Vries et al. reported that the median transient RLN injury rates were 3.7%, 2.5%, 4.0%, and 1.1% for BABA‐ET, MIVAT, TOETVA, and RAET, respectively. However, these differences were not statistically significant. Therefore, the accuracy of RLN recognition across different techniques requires further study. In this study, model accuracy was improved by increasing the training data and using cropped images to standardize and increase the size of the input images. These novel methods improved the results of previous attempts mainly by suppressing the FPs. This development is expected to be effective in cervical endoscopic surgeries that use a narrow and rigid speculum, as a black frame is present when a narrow endoscope is used. Notably, several small‐area FPs were observed because anatomical regions with minimal areas cannot be easily recognized in surgical videos and are rarely annotated. Therefore, the dissection of small areas might not be sufficiently learned or correctly predicted, or the AI's prediction might be accurate but not annotated, leading to a misclassification as an FP. The accuracy may be further improved by annotating small important areas and postprocessing to remove unimportant ones. A limitation of this study is that our AI model was trained with single‐center surgical videos, all corresponding to subclavian approaches, using images acquired with a 4‐mm Olympus rigid speculum. Therefore, if a different scope is used, the accuracy of the results may vary owing to the different image quality obtained. Additionally, all annotations were performed by a single surgeon, addressing variability between annotators. However, there is a possibility of subjective bias. If multiple trained annotators could have performed the annotation, it may have provided a more robust ground truth. We lack a sufficient dataset to separately examine the difference in recognition accuracy for the left and right sides. However, the positional relationship between the inferior thyroid artery and RLN differs slightly between the left and right sides, thus requiring further investigation into the difference in recognition accuracy. Annotation of 40 cases may be a small number of cases for AI learning. In the future, whether the developed AI model can recognize other surgical environments and approaches with high accuracy should be determined. CONCLUSION In this study, an image recognition AI model was developed for endoscopic thyroid surgery using a gasless method. AI image recognition technology may now be applied to cervical endoscopic surgeries because the recurrent laryngeal nerves and trachea were accurately identified using this method. CONFLICT OF INTEREST STATEMENT Yukio Nishiya, Kazuto Matsuura, Tateo Ogane, Kazuyuki Hayashi, Yumi Kinebuchi, Hirotaka Tanaka, Wataru Okano, Toshifumi Tomioka, Takeshi Shinozaki, and Ryuichi Hayashi have no conflicts of interest. ACKNOWLEDGMENTS This work was supported by Center for the Promotion of Translational Research, National Cancer Center, Japan and JSPS KAKENHI Grant Number JP24K19779. Yukio Nishiya has full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Nishiya Y, Matsuura K, Ogane T, et al. Anatomical recognition artificial intelligence for identifying the recurrent laryngeal nerve during endoscopic thyroid surgery: A single‐center feasibility study. Laryngoscope Investigative Otolaryngology. 2024;9(6):e70049. doi: 10.1002/lio2.70049 REFERENCES