PolyPose : Deformable 2D/3D Registration via Polyrigid Transformations

Vivek Gopalakrishnan, Neel Dey, Polina Golland · 2025

Read PolyPose : Deformable 2D/3D Registration via Polyrigid Transformations by Vivek Gopalakrishnan, Neel Dey, Polina Golland (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19256.txt.

Introduction

Vivek Gopalakrishnan, Neel Dey, and Polina Golland align a patient's 3D scan with as few as two live X-rays during surgery, in a 2025 paper. Surgeons have rich CT or MRI volumes before an operation but only flat X-rays during it. Their method, PolyPose, models the body's deformation as a chain of rigid moves, honoring the fact that bones do not bend. That built-in anatomy lesson replaces costly, fussy correction terms other methods need.

Across orthopedic and radiotherapy datasets, the approach succeeded in sparse, limited-angle setups where rivals failed outright. The anatomical prior does the heavy lifting: plausible motion is enforced by construction, not by tuning. The authors release code and tutorials. They pitch the method as practical 3D guidance for the operating room, where every extra image costs time and radiation.

Catalog record

CREATORVivek Gopalakrishnan, Neel Dey, Polina Golland
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-19256

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PolyPose : Deformable 2D/3D Registration via Polyrigid Transformations Vivek Gopalakrishnan, Neel Dey, Polina Golland Abstract Determining the 3D pose of a patient from a limited set of 2D X-ray images is a critical task in interventional settings. While preoperative volumetric imaging (e.g., CT and MRI) provides precise 3D localization and visualization of anatomical targets, these modalities cannot be acquired during procedures, where fast 2D imaging ( X-ray ) is used instead. To integrate volumetric guidance into intraoperative procedures, we present PolyPose , a simple and robust method for deformable 2D/3D registration. PolyPose parameterizes complex 3D deformation fields as a composition of rigid transforms, leveraging the biological constraint that individual bones do not bend in typical motion. Unlike existing methods that either assume no inter-joint movement or fail outright in this under-determined setting, our polyrigid formulation enforces anatomically plausible priors that respect the piecewise-rigid nature of human movement. This approach eliminates the need for expensive deformation regularizers that require patient- and procedure-specific hyperparameter optimization. Across extensive experiments on diverse datasets from orthopedic surgery and radiotherapy, we show that this strong inductive bias enables PolyPose to successfully align the patient’s preoperative volume to as few as two X-rays , thereby providing crucial 3D guidance in challenging sparse-view and limited-angle settings where current registration methods fail. Additional visualizations, tutorials, and code are available at https://polypose.csail.mit.edu . Figure 1: PolyPose is a locally-rigid framework for sparse-view deformable 2D/3D registration. (A) PolyPose can deformably align a high-resolution preoperative 3D volume to as few as two intraoperative 2D X-rays without the need of expensive regularizers or hyperparameter optimization. (B) To tackle this highly ill-posed problem, we estimate the poses ( ) of rigid bodies in the volume and smoothly interpolate them in space to produce a topologically consistent locally-rigid warp. (C) Using the estimated warps, PolyPose provides 3D volumetric guidance to procedures where only minimal supervision is available from intraoperative 2D X-rays . Figure 2: Illustration of polyrigid deformation fields. We visualize 2D slices of the rigid motion acting on every articulated structure. Linearly combining these transforms in the tangent space yields a smooth and invertible deformation field, which we color by the relative contribution from every structure. PolyPose enables the recovery of this 3D deformation field via differentiable rendering. 1 Introduction Estimating the 3D position of anatomical structures from 2D X-ray images is a critical task for clinical interventions that require millimeter-level precision, such as image-guided surgery [ 1 , 2 , 3 , 4 , 5 ] or the delivery of radiotherapy in cancer treatment [ 6 , 7 , 8 , 9 , 10 ] . The number of 2D X-rays available for 3D volumetric pose estimation is directly proportional to the radiation exposure to the patient and clinical team, as well as the time available for the procedure, thereby reducing the number of X-rays acquired [ 11 , 12 ] . Furthermore, the geometry of X-ray scanners limits the angular range of acquisitions, introducing spatial ambiguities along the projection direction and challenges for 3D localization [ 13 ] . While patients undergoing surgery and radiotherapy typically have previously acquired 3D volumes, such as computed tomography (CT) scans, their use is confounded by their misalignment with the intraoperative 2D X-rays as patients move between acquisitions (see the misaligned outlines in Figure 1 A). Several parameterizations of 2D/3D motion have been proposed to align these modalities. For example, rigid 2D/3D registration methods align global structure [ 14 , 15 , 16 , 17 ] , but do not account for the soft tissue deformation or articulated inter-joint motion that occurs during procedures and creates localization challenges. Other work estimates point-wise displacement fields using either deep learning [ 18 , 19 , 20 , 21 , 22 ] or optimization [ 23 , 24 , 25 ] . However, given the minimal supervision available for estimating 3D deformations in 2D sparse-view and limited-angle settings, deformable models require extensive application-specific regularization to generate anatomically faithful warps, thereby introducing new modeling decisions and hyperparameter tuning for every subject, procedure, and anatomical region. Our approach is instead motivated by a generic anatomical prior: bones are rigid bodies. We parameterize deformable 2D/3D registration using a low-dimensional polyrigid model with limited degrees of freedom ( Figure 2 ), where transformations are composed from individually estimated rototranslations of multiple articulated structures that are linearly combined in the tangent space \mathfrak{se}(3) [ 26 ] . This reduces the number of optimizable parameters from the order of voxels in the CT volume to the order of the number of rigid components. Furthermore, unlike other low-dimensional deformation models (e.g., splines [ 27 ] or linear bases [ 18 ] ), polyrigid transforms have several desirable properties by construction, such as smoothness, invertibility, and coordinate frame invariance [ 26 ] . Our method, PolyPose , enables the estimation of highly accurate non-rigid deformations that are anatomically plausible and topologically consistent. We do this via differentiable X-ray rendering, providing piece-wise 2D/3D registration targets from which to construct a polyrigid warp. Empirically, across diverse datasets, PolyPose is robust even for a small number of input views from limited angles. Furthermore, given its strong inductive priors, PolyPose does not require any regularization and has no tunable hyperparameters other than the step size of the optimizer. Our method outperforms both deep learning and optimization-based 2D/3D registration methods and enables the 3D localization of critical structures during medical interventions from intraoperative 2D images. Contributions. To summarize, PolyPose contributes: • A regularization-free framework for deformable 2D/3D registration that estimates polyrigid deformation fields using differentiable X-ray rendering. • A hyperparameter-free weighting function for linearly combining multiple rigid transformations, providing out-of-the-box generalization to new surgical and therapeutic procedures. • An anatomically motivated motion model that is robust in sparse-view and limited-angle settings and produces smooth, invertible, and accurate deformation fields by construction. 2 Related Work Rigid 2D/3D registration. Given a 2D X-ray and a 3D CT volume, rigid registration methods estimate a global rigid transformation in \mathbf{SE}(3) that optimally aligns the two images [ 28 , 29 ] . While state-of-the-art methods can now determine the pose of rigid bodies with less than a millimeter of error [ 15 , 16 ] (which, in a different reference frame, is equivalent to estimating the extrinsic matrix of the image), they fail to describe the motion of volumes subject to non-rigid deformable transformations. Deformable 2D/3D registration. Non-rigid deformable 2D/3D registration is crucial to radiation oncology, where a dense displacement field is needed to align a preoperative planning CT volume with multiple intraoperative X-ray images [ 20 , 23 ] . As deformably aligning a 3D volume to a set of sparse 2D X-rays is severely ill-posed, deformable 2D/3D methods rely on complex regularization schemes (e.g., diffusion [ 30 ] , total variation [ 31 ] , elastic penalties [ 32 ] ), introducing numerous hyperparameters that must be carefully tuned for every procedure, subject, and anatomical region. Deformable 3D/3D registration. Many methods exist to reconstruct 3D cone-beam computed tomography (CBCT) volumes from multiple 2D X-rays [ 33 ] . As such, one could reformulate multi-view 2D/3D registration as a 3D/3D registration task, an active research area, and use recent foundation models for multimodal 3D/3D registration

Read the complete text