Markerless Tracking-Based Registration for Medical Image Motion Correction

Luisa Neubig, Deirdre Larsen, Takeshi Ikuma, Markus Kopp, Melda Kunduk, Andreas M. Kist · 2025

Read Markerless Tracking-Based Registration for Medical Image Motion Correction by Luisa Neubig, Deirdre Larsen, Takeshi Ikuma, Markus Kopp, Melda Kunduk, Andreas M. Kist (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10260.txt.

Introduction

X-ray videos of swallowing mix the act itself with stray head motion, and a 2025 paper by Luisa Neubig and co-authors builds a way to strip the stray motion out. Clean videos matter because doctors read them to judge how a patient's swallowing works.

Standard flow methods flicker and fail on these clips, so the team tested markerless trackers that follow points across frames. Even a sparse set of tracked points, spread into a smooth warp field, beat leading registration tools on standard error and likeness scores. Their pipeline removes the disruptive head motion while keeping the true motion of swallowing. That gives clinicians a steadier view of the throat in action.

Catalog record

CREATORLuisa Neubig, Deirdre Larsen, Takeshi Ikuma, Markus Kopp, Melda Kunduk, Andreas M. Kist
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-10260

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Markerless Tracking-Based Registration for Medical Image Motion Correction Luisa Neubig, Deirdre Larsen, Takeshi Ikuma, Markus Kopp, Melda Kunduk, Andreas M. Kist Abstract Our study focuses on isolating swallowing dynamics from interfering patient motion in videofluoroscopy, an X-ray technique that records patients swallowing a radiopaque bolus. These recordings capture multiple motion sources, including head movement, anatomical displacements, and bolus transit. To enable precise analysis of swallowing physiology, we aim to eliminate distracting motion, particularly head movement, while preserving essential swallowing-related dynamics. Optical flow methods fail due to artifacts like flickering and instability, making them unreliable for distinguishing different motion groups. We evaluated markerless tracking approaches (CoTracker, PIPs++, TAP-Net) and quantified tracking accuracy in key medical regions of interest. Our findings show that even sparse tracking points generate morphing displacement fields that outperform leading registration methods such as ANTs, LDDMM, and VoxelMorph. To compare all approaches, we assessed performance using MSE and SSIM metrics post-registration. We introduce a novel motion correction pipeline that effectively removes disruptive motion while preserving swallowing dynamics and surpassing competitive registration techniques. 1 Introduction Motion analysis plays a critical role in medical imaging, particularly in the evaluation of dynamic processes such as swallowing [ 7 ] . Videofluoroscopy Swallowing Studies (VFSS) capture high-temporal-resolution (up to 30 fps) X-ray recordings of patients swallowing a radiopaque bolus to assess swallowing function [ 12 ] . However, these recordings contain multiple overlapping sources of motion, including bolus flow, head movement, and anatomical structure shifts. This makes it difficult to focus and isolate clinically relevant swallowing dynamics. Accurately correcting for these confounding motions is crucial for precise physiological analysis. To achieve this, we must (i) assess global motion artifacts, (ii) identify distinct moving structures, and (iii) register relevant features to compensate for unwanted motion. In this study, we address these challenges by isolating unwanted motion, such as head movement, while preserving anatomical motions and accurate depiction of the bolus suitable for downstream quantitative analysis. By leveraging advanced tracking algorithms, we compute velocity and displacement fields to effectively suppress large, distracting shifts. We show that our proposed method works on rather still and more challenging data, as well as generalizes to data provided by other hospitals. 2 Related Works 2.1 Optical Flow-based Methods in Medical Imaging Optical flow methods are widely used in medical image analysis to estimate motion from pixel displacements and enhance computer vision tasks like segmentation and image registration. Xue et al. [ 20 ] propose a novel segmentation method for echocardiography that effectively utilizes motion information by accurately predicting the optical flow. This information and initial segmentation guesses result in a motion-enhanced segmentation module for final segmentation. Suji et al. analyzed how optical flow methods (i.e., Farneback, Horn-Schunck and Lucas-Kanade) can improve motion-based segmentation of lung nodules in thin-sliced CT scans [ 17 ] . In [ 21 ] , motion information is used to discriminate between healthy and affected patients. They use an optical flow-based method to quantitatively analyze the deformation of the right diaphragm in ultrasound imaging to track respiratory motion. 2.2 Image Registration in Medical Imaging Image registration enables the matching of medical images across time, modalities, or patients, as well as multimodal fusion to improve diagnosis and treatment. Deformable image registration for image-guided adaptive radiotherapy has issues in low-contrast regions. Meng et al. [ 13 ] address this problem by integrating a finite element method (FEM) with the ’demon’ algorithm and refining a tetrahedral mesh to improve displacement accuracy and reduce registration errors. A common issue in medical image registration is that labeling corresponding features is a very time-consuming task, which is why image registration is preferably performed unsupervised. [ 6 ] overcomes the labeling of corresponding areas by using foundation segmentation models, such as Segment Anything (SAM), to define ROI correspondences for registration. 2.3 Tracking in Videofluoroscopies Previous studies on VFSS have focused on the dynamics of anatomical landmarks, assuming a motionless patient. However, we are not aware of any study that investigated or corrected for patient motion during VFSS recordings. On rare occasions, the community acknowledges this issue: [ 10 ] draws attention to the importance of tracking the movement of the hyoid bone during VFSS. We believe that quantitative measurements such as in [ 10 ] , and [ 9 ] or tracking of cervical vertebrae such as in [ 14 ] can benefit from compensating for the patient’s head movement as we propose in our study. 3 Methods 3.1 Dataset We created two datasets termed motion and non-motion containing each of 10 manually selected VFSS recordings with a single swallowing event. The VFSS recordings were acquired at an American hospital (anonymized) and in accordance with the granted IRB (#IRBAM-anonymized). The videos have different lengths, ranging from approximately 30 to 170 frames. All videos have a temporal resolution of 30 fps. All frames were resized to a uniform resolution of 256\times 256 px. For validation purposes, we rely on example data from a European hospital (anonymized). This data is in many ways dissimilar to the American dataset, i.e., the temporal resolution is only 7.5 fps and the aperture is rectangular instead of circular. 3.2 Assessing Optical Flow using Markerless Tracking We first analyzed multiple optical flow methods including classical approaches (Lucas-Kanade Method, Farneback) and Deep Learning-based methods (RAFT, SpyNet, FlowFormer, PWCNet, UnFlow). Next to these methods, we tested three different markerless tracking algorithms (CoTracker, PIPs++ and TAP-Net) to determine correspondences. All experiments were performed on an NVIDIA GeForce RTX 4090 with 24GB RAM. Additionally, we experimented with different grid sampling strategies to assess their impact on tracking performance and adjusted grid sizes to meet the GPU RAM limitations. CoTracker utilizes a query-based mechanism, allowing for adaptive point selection and refinement across frames [ 8 ] . We employed CoTrackerPredictor3 in its default offline configuration with the provided pre-trained model [ 1 ] . PIPs++ uses a sliding window to update predictions iteratively and maintain trajectory consistency [ 22 ] . We utilized PIPs++ in its default configuration and leveraged the provided pre-trained model [ 5 ] . TAP-Net follows a two-stage approach to estimate trajectories accurately over time. In the matching stage, TAP-Net independently searches for a suitable candidate point match for the query point across all frames, leveraging local and global spatial information. We use the provided pre-trained BootsTAPIR model for our analysis [ 4 ] . 3.3 Motion Correction using Velocity Flow Fields We stabilize the patient during swallowing by correcting motion through image registration using the predicted velocity field. To evaluate our approach, we compare it with cross-domain state-of-the-art (SOTA) registration algorithms, as no existing methods are specifically designed for VFSS. The comparison includes Large Deformation Diffeomorphic Metric Mapping (LDDMM) [ 3 ] , Advanced Normalization Tools (ANTs) [ 18 ] , and VoxelMorph [ 2 ] . LDDMMs formulate registration as finding a smooth, time-varying velocity field \{v_{t}\}_{t\in[0,T]} whose flow integrates to a diffeomorphism. Specifically, we define the transformation \phi_{t}:\Omega\to\Omega by \frac{\partial\phi_{t}(x)}{\partial t}=v_{t}(\phi_{t}(x)),\quad\phi_{0}(x)=x, (1) so that \phi_{1} is the final deformation at t=1 . The objective function typically balances a regularization on the velocity field (encouraging smoothness) and an image similarity term: \min_{v_{t}}\int_{0}^{1}\mathcal{R}(v_{t})\,\mathrm{d}t\;+\;\mathcal{D}\bigl(I_{0}\circ\phi_{1}^{-1},I_{1}\bigr), (2) where \mathcal{R} denotes a regularization functional (e.g., \|\nabla v_{t}\|^{2} ) and \mathcal{D} is a

Read the complete text