Model selection and real-time skill assessment for suturing in robotic surgery
Zhaoyang Jacopo Hu, Alex Ranne, Alaa Eldin Abdelaal, Kiran Bhattacharyya, Etienne Burdet, Allison M. Okamura, Ferdinando Rodriguez y Baena · 2026
Read Model selection and real-time skill assessment for suturing in robotic surgery by Zhaoyang Jacopo Hu, Alex Ranne, Alaa Eldin Abdelaal, Kiran Bhattacharyya, Etienne Burdet, Allison M. Okamura, Ferdinando Rodriguez y Baena (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12012.txt.
Introduction
Zhaoyang Jacopo Hu, Alex Ranne, Alaa Eldin Abdelaal, and their co-authors grade suturing skill in robot surgery in real time. Trained judges grade such skill with the OSATS scale. A live, fair grade tool could aid training and tests.
They study model choice for fast skill calls from surgery data. Fast calls let a trainee get notes while the task is fresh. Firm live grades could also flag unsafe moves at once. The work weighs which models keep speed and truth in balance. Live grades can shape drills for each trainee by need.
Catalog record
| CREATOR | Zhaoyang Jacopo Hu, Alex Ranne, Alaa Eldin Abdelaal, Kiran Bhattacharyya, Etienne Burdet, Allison M. Okamura, Ferdinando Rodriguez y Baena |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-12012
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Model selection and real-time skill assessment for suturing in robotic surgery Zhaoyang Jacopo Hu, Alex Ranne, Alaa Eldin Abdelaal, Kiran Bhattacharyya, Etienne Burdet, Allison M. Okamura, Ferdinando Rodriguez y Baena Abstract Automated feedback systems have the potential to provide objective skill assessment for training and evaluation in robot-assisted surgery. In this study, we examine methods to achieve real-time prediction of surgical skill level in real-time based on Objective Structured Assessment of Technical Skills (OSATS) scores. Using data acquired from the da Vinci Surgical System, we carry out three main analyses, focusing on model design, their real-time performance, and their skill-level-based cross-validation training. For the model design, we evaluate the effectiveness of multimodal deep learning models for predicting surgical skill levels using synchronized kinematic and vision data. Our models include separate unimodal baselines and fusion architectures that integrate features from both modalities and are evaluated using mean Spearman’s correlation coefficients, demonstrating that the fusion model consistently outperforms unimodal models for real-time predictions. For the real-time performance, we observe the prediction’s trend over time and highlight correlation with the surgeon’s gestures. For the skill-level-based cross-validation, we separately trained models on surgeons with different skill levels, which showed that high-skill demonstrations allow for better performance than those trained on low-skilled ones and generalize well to similarly skilled participants. Our findings show that multimodal learning allows more stable fine-grained evaluation of surgical performance and highlights the value of expert-level training data for model generalization. I Introduction Skill assessment in robotic surgery is a necessary component to provide feedback in surgical training and ensuring patient safety during procedures. With the increasing adoption of robotic systems like the da Vinci Surgical System, there is a growing demand for reliable, objective, and efficient methods to evaluate surgical performance. Traditional approaches to skill assessment [ 2 ] rely on post-procedure analysis, often involving subjective expert evaluations and manual review of recorded data. While these methods provide valuable insights, they lack immediacy and scalability, making it challenging to offer real-time feedback to surgeons during procedures. Figure 1: Suturing example in the JIGSAWS dataset. As the needle is pulled out, the tissue is visibly torn, which affects the skill assessment score “Respect for Tissue”. To address this limitation, researchers have developed machine learning models capable of automatically assessing surgical skill. A widely used resource for training such models is the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS) [ 5 ] . JIGSAWS includes multimodal data captured during robotic suturing tasks (Figure 1 ), combining synchronized vision and kinematic data streams. The dataset is widely used as a common standard for investigating surgeon movements during robotic surgery [ 17 , 18 , 16 ] . However, in the context of skill assessment, the annotations of technical skills in these types of datasets are limited, as they typically use only a single score per trial. This constraint has led most existing studies to focus on post-hoc evaluations, providing only trial-level assessments that overlook the specific gestures or events correlated to high or low skills [ 20 , 1 , 6 ] . To address this, our work introduces real-time, window-level skill prediction models that offer fine-grained, time-resolved feedback throughout the surgical task, enabling a shift toward continuous and adaptive evaluation. From the clinical perspective, having real-time skill assessment allows trainees to adjust their movements during a procedure, which is more effective in long term training [ 12 ] . A second limitation concerns the diversity and imbalance of participant expertise within such datasets. Recruiting expert surgeons is difficult, often resulting in datasets that contain a broad range of skill levels. This raises concerns about the reliability of labels and the influence of user skill level on model performance [ 8 ] . To investigate this, we conduct skill-level-based cross-validation, examining how models trained on participants of varying expertise perform and generalize across different user groups. Lastly, existing research often treats vision and kinematic data in isolation [ 1 , 13 , 4 , 3 ] , developing separate models for each modality and overlooking their complementary nature. To overcome this, we explore both unimodal and multimodal model architectures by evaluating seven deep learning models that process kinematic, visual, and combined input streams. The contributions of this work are as follows: (1) We present a comparative analysis of seven models across unimodal (kinematic or vision) and multimodal (fusion) approaches, establishing a foundation for understanding the effectiveness of different data modalities; (2) We extend these models to perform real-time, granular skill assessment, shifting away from post-hoc evaluations and aligning more closely with how surgical performance is assessed in live procedures; (3) We examine how varying levels of surgical expertise affect model performance, providing insight into the generalizability and robustness of skill assessment systems across different user skill levels. Together, these contributions form a framework for evaluating and enhancing real-time, multimodal surgical skill assessment, addressing both model effectiveness and practical deployment considerations. II Related Work Surgical skill assessment has commonly relied on an expert reviewing videos of the surgical task and manually scoring the performance of the surgeons. Zia et al. [ 20 ] highlighted the biased and time-consuming nature of manual assessments, emphasizing the need for automation. Several approaches have been explored to automate surgical skill assessment. Early methods leveraged Hidden Markov Models (HMMs) for modeling surgical gestures and classifying skill levels [ 15 ] . More recent studies have applied deep learning architectures, such as convolutional and recurrent neural networks (CNNs, RNNs), to extract spatiotemporal features from kinematic and video data. For instance, Zia et al. [ 20 ] introduced a feature fusion approach using sequential motion texture, discrete Fourier transform, and discrete cosine transform to classify skill levels in the JIGSAWS dataset. Several studies have also focused on categorical skill classification, grouping surgeons into novice, intermediate, or expert levels [ 5 , 19 ] . However, these methods often lack the granularity needed for real-time, continuous assessment. One of the most widely adopted skill assessment frameworks is the Objective Structured Assessment of Technical Skills (OSATS), which evaluates surgical performance based on predefined criteria such as respect for tissue, time and motion, and overall performance which allows a granular study of the surgical skills. Benmansour et al. [ 1 ] proposed a Convolutional Neural Network and Bidirectional Long Short-Term Memory (CNN+BiLSTM) model for predicting OSATS scores directly from kinematic data, achieving state-of-the-art performance. However, the use of BiLSTM limits the architecture to only perform post-hoc analysis on the recorded data. Additionally, many surgical skill datasets provide only a single label for an entire trial, typically assigned after task completion. While useful for global assessment, this labeling approach limits the ability to evaluate skill progression throughout the procedure. However, it is possible to leverage these trial-level labels to supervise in a finer-grained manner, enabling models to infer skill continuously over time despite the coarse annotation [ 10 ] . This work extends prior research in surgical skill assessment by introducing three primary contributions: • Performance analysis with different input modalities : We analyze the performances of models based on different data channels using Spearman’s correlation coefficient and Root Mean Square Error ( RSME ). • Real-time OSATS Score Prediction : Unlike studies that