Towards Markerless Intraoperative Tracking of Deformable Spine Tissue

Connor Daly (🖂), Elettra Marconi, Marco Riva, Jinendra Ekanayake, Daniel S. Elson, Ferdinando Rodriguez y Baena · 2025

Read Towards Markerless Intraoperative Tracking of Deformable Spine Tissue by Connor Daly (🖂), Elettra Marconi, Marco Riva, Jinendra Ekanayake, Daniel S. Elson, Ferdinando Rodriguez y Baena (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-23657.txt.

Introduction

Connor Daly and colleagues work on tracking spine tissue during surgery without markers, in a 2025 paper. Marker-based tracking adds time and hardware to an operation. Camera systems that sense color and depth could track tissue directly, but past work used cadavers only.

The team presents what they call the first real clinical dataset of this kind for spine surgery. They also built SpineAlign, a system that captures how the spine shape changes between scans before surgery and views during surgery. A further network finds key regions for aligning the two views. The work aims at safer, simpler guidance in the operating room.

Catalog record

CREATORConnor Daly (🖂), Elettra Marconi, Marco Riva, Jinendra Ekanayake, Daniel S. Elson, Ferdinando Rodriguez y Baena
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-23657

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Markerless Intraoperative Tracking of Deformable Spine Tissue Connor Daly (🖂), Elettra Marconi, Marco Riva, Jinendra Ekanayake, Daniel S. Elson, Ferdinando Rodriguez y Baena Abstract Consumer-grade RGB-D imaging for intraoperative orthopedic tissue tracking is a promising method with high translational potential. Unlike bone-mounted tracking devices, markerless tracking can reduce operating time and complexity. However, its use has been limited to cadaveric studies. This paper introduces the first real-world clinical RGB-D dataset for spine surgery and develops SpineAlign , a system for capturing deformation between preoperative and intraoperative spine states. We also present an intraoperative segmentation network trained on this data and introduce CorrespondNet , a multi-task framework for predicting key regions for registration in both intraoperative and preoperative scenes. 1 Introduction and Motivation Real-time intraoperative markerless tracking of orthopedic tissue using RGB depth sensors is still in its infancy but holds high translational potential [ 7 , 10 ] . Current Computer Aided Orthopedic Surgery (CAOS) tracking systems, like the StealthStation S8 (Medtronic, USA), use costly bone-anchored markers to align preoperative and intraoperative data [ 21 , 26 ] . These require additional surgical time for calibration and pose risks of complications [ 5 ] . Markerless tracking, instead, uses deep learning-based segmentation to identify relevant points in the surgical scene’s point cloud, aiding registration [ 7 , 10 ] . We present the first real-world clinical dataset for training a markerless tracking system, comprising 70,095 colored point clouds from 27 spine surgeries, paired with anonymized preoperative 3D imaging. As a first step toward automatic registration, we train a neural network to isolate key intraoperative points and use it as the backbone of CorrespondNet , a model that aligns preoperative and intraoperative data in a shared feature space for registration. Our contributions include: 1. The first real-world clinical RGB-D dataset for orthopedic markerless tissue registration. 2. A data labeling system for capturing articulated pose changes in the spine. 3. A markerless tracking architecture that simultaneously segments preoperative and intraoperative data to isolate key regions for registration. There have been several cadaver-based studies on knee and spine surgery [ 10 , 2 , 7 , 12 ] . Liu and Rodriguez y Baena [ 12 ] pioneered markerless tracking in knee surgery, leading to neural network-based segmentation work by Hu et al. [ 7 ] and Liebmann et al. [ 10 ] . Using a 3D preoperative mesh, conventional registration algorithms such as RANSAC with Iterative Closest Points mapped transformations between the preoperative and intraoperative scenes. Hu et al. improved segmentation in occluded scenes, with both Liu and Hu using a PointNet backbone [ 19 ] . For the spine, Chen et al. [ 2 ] registered CT-generated meshes of excised sheep spines against point cloud scans, while Liebmann et al. [ 10 ] applied a similar method to human spine cadavers. Their approach segmented relevant 2D RGB pixels to select depth map points for projection rather than segmenting the entire point cloud. Both studies registered each vertebra separately using ICP. Despite these advances, real-world surgery presents challenges due to limited tissue exposure. In an open lumbar posterior approach, dissection is typically restricted to the facet joints [ 22 ] , and key anatomical landmarks remain obscured by residual tissue (Fig. 1 (a)). Given the lack of shape cues for vertebral registration, here we first deform the preoperative mesh using a kinematic model before performing global rigid alignment to label our dataset. For segmentation, following [ 7 ] , we use a PointNet++ [ 20 ] backbone. Unlike previous studies however, our network simultaneously segments both the intraoperative scene and preoperative mesh, isolating key regions for registration despite the noisy surgical environment. This study has full ethics approval. Upon acceptance, the anonymized dataset, labeling code, and model code will be released. Figure 1: (a) RGB images of an exposition site showing L5, L4, and L3 spinous processes. (b) The surgeon indicates the L4 process. (c) Shows the fixed pose RGB-D intraoperative setup. (d) Rigid registration of a preoperative mesh to a surgery scene point cloud. (e) The registered mesh from (d) after deformation in SpineAlign . (f) Joint positions and axis vectors for deforming a preoperative CT spine mesh, with blue, red, and green arrows representing anteroposterior, longitudinal, and mediolateral axes. 2 Methods 2.1 Data Collection The dataset was collected over a series of 27 spine surgeries, all of which took an open posterior approach. The age of the patients at the time of surgery ranged between 42 and 78 years, with 11 male patients and 16 female patients. The pathologies presented varied; however, the vast majority suffered from spondylolisthesis. 23 patients had accompanying preoperative CT images while the rest had magnetic resonance images (MRI). 24 of the surgeries were lumbar surgeries, 1 was entirely thoracic, whilst the remaining 2 were cervical. In our analysis and inclusion for training our deep learning models, we used the 24 lumbar cases. Within the exposition sites of these surgeries, the number of vertebrae exposed ranges between 2 and 4. To generate the 3D preoperative state from the acquired CT and MRI data, we incorporated two existing vertebrae segmentation networks. To segment CT scans, we used the ‘anatomy-consistency’ model developed by Meng et al. [ 15 ] , which is particularly robust to variations in patient spine anatomy. To segment MRI scans, we incorporated the ‘TotalSpineSeg’ network developed by Warszawer et al. [ 24 ] , heavily based off ‘nnU-Net’ [ 8 ] . With the segmented scan, we then used the Marching Cubes algorithm [ 13 ] , in conjunction with a simple smoothing filter, to transform the CT representation to a 3D mesh. For the intraoperative video sequences (Fig. 1 (b)), the Time-of-Flight (TOF) RGB-D camera (Azure Kinect, Microsoft Corporation, USA) was fixed to a static tripod and held over the exposition site via a horizontal boom arm (Fig. 1 (c)). Upon completion of the initial exposure by the surgeon, the arm was lowered to a height between 30 - 40 cm from the incision, whereupon we recorded an RGB-D video of approximately 2 minutes length at a frame rate of 30 frames per second. Within the sequence, the operating surgeon pointed to at least three non co-linear ‘landmarks’ of specific vertebrae within the operating site, such as the spinous process and the facet joints. These in-sequence cues were used to provide the gross initial alignment between the preoperative mesh and the intraoperative point cloud scenes, estimating a coarse registration transform via Singular Value Decomposition. 2.2 The Spine as a Kinematic Chain Since both the camera and patient remain static during surgery, we assume that the most significant intervertebral movement occurs between the preoperative scan (when the patient is supine) and the intraoperative phase (when the patient is prone). Given our focus on the lumbar region, spine movement due to breathing is minimal [ 4 , 9 ] . Thus, in the same vein as [ 10 ] ’s system that performs registration with respect to a particular reference frame, for each sequence s , we labeled the articulation required to deform the spine from its preoperative state, Sp , to its intraoperative state, Si , with respect to the point cloud, P(s_{r})

Read the complete text