Interpretable deformable image registration: A geometric deep learning perspective

Vasiliki Sideri-Lampretsa Technical University MunichMunich, Germany [email protected], Nil Stolt-Ansó Technical University MunichMunich, Germany [email protected] Huaqi Qiu Technical University MunichMunich, Germany [email protected] Julian McGinnis Technical University MunichMunich, · 2024

Read Interpretable deformable image registration: A geometric deep learning perspective by Vasiliki Sideri-Lampretsa Technical University MunichMunich, Germany [email protected], Nil Stolt-Ansó Technical University MunichMunich, Germany [email protected] Huaqi Qiu Technical University MunichMunich, Germany [email protected] Julian McGinnis Technical University MunichMunich, (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-13294.txt.

Introduction

Sideri-Lampretsa and six co-authors build a bendable scan-matching method that humans can follow, from the math of shapes, in work from 2024. Registration warps one scan to line up with a second, across two grid maps. Most nets treat that warp as a black box, shrug off the shape of the task, and need piles of data. The key is to see how a net matches a feature in one scan to a feature in the next.

Their base splits two jobs: pulling a feature out, and modeling the warp. From that split flow shifting fields of view and warp rules that know both grids. Warps are honed pass by pass, in smooth space, with no forced step back onto a plain grid. On brain and eye scans, the method beats the best scores. Its warp steps stay far clearer for a human to follow.

Catalog record

CREATORVasiliki Sideri-Lampretsa Technical University MunichMunich, Germany [email protected], Nil Stolt-Ansó Technical University MunichMunich, Germany [email protected] Huaqi Qiu Technical University MunichMunich, Germany [email protected] Julian McGinnis Technical University MunichMunich,
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-13294

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Interpretable deformable image registration: A geometric deep learning perspective Vasiliki Sideri-Lampretsa Technical University MunichMunich, Germany [email protected], Nil Stolt-Ansó Technical University MunichMunich, Germany [email protected] Huaqi Qiu Technical University MunichMunich, Germany [email protected] Julian McGinnis Technical University MunichMunich, Germany [email protected] Wenke KarboleTechnical University MunichMunich, Germany [email protected] Martin Menten Technical University MunichMunich, Germany [email protected] Daniel Rueckert Technical University MunichMunich, Germany [email protected] Abstract Deformable image registration poses a challenging problem where, unlike most deep learning tasks, a complex relationship between multiple coordinate systems has to be considered. Although data-driven methods have shown promising capabilities to model complex non-linear transformations, existing works employ standard deep learning architectures assuming they are general black-box solvers. We argue that understanding how learned operations perform pattern-matching between the features in the source and target domains is the key to building robust, data-efficient, and interpretable architectures. We present a theoretical foundation for designing an interpretable registration framework: separated feature extraction and deformation modeling, dynamic receptive fields, and a data-driven deformation functions awareness of the relationship between both spatial domains. Based on this foundation, we formulate an end-to-end process that refines transformations in a coarse-to-fine fashion. Our architecture employs spatially continuous deformation modeling functions that use geometric deep-learning principles, therefore avoiding the problematic approach of resampling to a regular grid between successive refinements of the transformation. We perform a qualitative investigation to highlight interesting interpretability properties of our architecture. We conclude by showing significant improvement in performance metrics over state-of-the-art approaches for both mono- and multi-modal inter-subject brain registration, as well as the challenging task of longitudinal retinal intra-subject registration. We make our code publicly available 1 1 1 https://anonymous.4open.science/status/GeoReg-1A1D . 1 Introduction Image registration is an indispensable tool in medical image analysis that aligns anatomically or functionally corresponding regions across images, often from different modalities and time points [ 35 ] . In particular, deformable registration aims to estimate a non-linear transformation that maps the source image to the coordinate system of the target image. Figure 1 : Our multi-resolution architecture begins by extracting features at increasingly coarse resolutions. In a coarse-to-fine fashion, the deformation function \tau_{\theta} refines the predicted deformation over N steps within the current resolution while the learned interpolation function \delta_{\theta} carries deformations onto the subsequent resolution. The architecture is supervised such that the majority of the deformation is modeled at the coarser (earliest) resolutions. Supervision of the finest (latest) resolutions provides learning signal to all deformations at the coarser levels. Since the advent of deep learning (DL), data-driven methods have been proposed [ 14 , 43 ] to leverage learned transformation priors over an image cohort, reducing the search space of plausible transformations. DL approaches excel at creating highly expressive, task-specific feature-extracting processes using end-to-end supervision signals. The underlying workhorse of these architectures, the convolution , is a pattern-matching tool that has been heavily optimized explicitly for grid-based inputs such as images. These convolutional layers are stacked sequentially to give rise to hierarchies of features, whereby early layers detect the presence of simple spatial building blocks (such as edges and corners), while later layers use these basic structures to capture larger-spanning spatial features of higher complexity. Unlike tasks such as image segmentation, where the feature extraction process is constrained to a single coordinate system, image registration poses an interesting challenge in deep learning due to having to simultaneously consider spatial relationships of multiple input coordinate systems. 1.1 Disentangling feature extraction and deformation modeling Many data-driven works have adopted a single-stream approach for deformation modeling, whereby source and target images are simply concatenated in the channel dimension as input to a convolutional network [ 3 , 7 , 28 , 29 , 44 , 5 ] . Furthermore, to allow for more flexibility in the feature extraction process or deformation modeling recent works incorporate transformer layers into the network [ 5 , 6 , 23 , 26 , 42 , 45 , 11 ] . Nonetheless, this comes at the cost of introducing high costs in terms of learnable parameters that may prove unrealistic for most real-world clinical applications outside of the registration benchmark datasets. We argue that early-fusion approaches make inefficient use of learnable parameters, as it causes the feature extraction to learn distinct representations within the network for each possible misalignment of target-source images, increasing the task’s complexity and generalizing poorly to unseen misalignments. The black-box nature of these architectures means the extraction of intensity-derived features is inseparable from the deformation modeling process. This poorly defined separation between the two sub-tasks of intra-domain feature extraction and inter-domain deformation modeling, leads to a general lack of interpretability. An alternative approach used in literature opts for dual-stream architectures, whereby feature extraction is performed separately from the deformation modeling process resulting in simpler and hence more interpretable inter-domain deformation modeling. In these late-fusion works, source and target images are encoded individually prior to the concatenation of both spatial domains. Moreover, this style of encoding produces feature hierarchies invariant to the alignment of the two spatial domains such that a change in one spatial domain has no effect on the feature extraction of the other. Additionally, mono-modality registration can benefit in terms of parameter efficiency by reusing the same encoder across both images. 1.2 Adaptive receptive fields and transformation refinement Receptive fields and function complexity: In the context of deformation functions, a convolution addresses the essence of a deformation modeling task: producing a deformation vector given a neighborhood of source and target features. When estimating the deformation at a point, the input neighborhood defines the receptive field of the operation in the source and target domain. Although using the largest possible neighborhood sizes increases the amount of potentially relevant input information, this also increases the modeling capacity required to process the input region. These convolutional operations can be stacked to for convolutional networks. Deep convolutional networks offer equivalent behavior to larger neighborhood sizes proportional to their receptive fields. At the extreme (such as with deep enough U-Nets or ViT architectures), the entirety of the spatial domain may be considered as an input argument to the deformation of any given grid point. Conversely, having fewer neighboring input points decreases the deformation function complexity. While simple functions offer easily interpretable relationships between input structures and output deformations, smaller neighborhoods limit the receptive field of available structures. A convolutional function with limited receptive field size may encounter situations where the available structural features are insufficient to capture accurate deformation. This becomes more prominent when the required deformations are larger than the available receptive field. Transformation refinement: One widely adopted technique across the literature involves progressively learning transformation through refinement steps. By splitting the space across multiple evaluations, this technique effectively extends the available receptive field of an architecture beyond what would otherwise be available in a single prediction. These architectures essentially implement a form of adaptive receptive fields, whereby the predicted transformation on a given source region determines the new neighborhood of target features available to the next step of the refinement process. Arguably the most common type of refinement technique is implemented under a cascading blueprint. These typically involve as a series of single-stream architectures [ 17 , 31 , 44 ] , where the output transformation of an earlier process warps

Read the complete text