Multi-Objective Deep-Learning-based Biomechanical Deformable Image Registration with MOREA
Georgios Andreadis, Eduard Ruiz Munné, Thomas H. W. Bäck, Peter A. N. Bosman, Tanja Alderliesten · 2025
Read Multi-Objective Deep-Learning-based Biomechanical Deformable Image Registration with MOREA by Georgios Andreadis, Eduard Ruiz Munné, Thomas H. W. Bäck, Peter A. N. Bosman, Tanja Alderliesten (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16525.txt.
Introduction
Georgios Andreadis and co-authors join two styles of deformable image registration into one hybrid, DL-MOREA. Deep learning methods predict a warp at once but can give less realistic warps on hard cases. Biomechanical finite element methods give more realistic warps yet take much longer to run.
The hybrid starts the evolutionary MOREA search from the output of a deep learning method called DL-MODIR, so the mesh model is tuned with a head start. Tests used CT pairs from fifteen cervical cancer patients with large bladder filling changes. Plain MOREA took a median of forty-five minutes, while the hybrid found good warps in five minutes, with far less folding than the deep learning output alone.
Catalog record
| CREATOR | Georgios Andreadis, Eduard Ruiz Munné, Thomas H. W. Bäck, Peter A. N. Bosman, Tanja Alderliesten |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-16525
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Multi-Objective Deep-Learning-based Biomechanical Deformable Image Registration with MOREA Georgios Andreadis, Eduard Ruiz Munné, Thomas H. W. Bäck, Peter A. N. Bosman, Tanja Alderliesten Abstract When choosing a deformable image registration (DIR) approach for images with large deformations and content mismatch, the realism of found transformations often needs to be traded off against the required runtime. DIR approaches using deep learning (DL) techniques have shown remarkable promise in instantly predicting a transformation. However, on difficult registration problems, the realism of these transformations can fall short. DIR approaches using biomechanical, finite element modeling (FEM) techniques can find more realistic transformations, but tend to require much longer runtimes. This work proposes the first hybrid approach to combine them, with the aim of getting the best of both worlds. This hybrid approach, called DL-MOREA, combines a recently introduced multi-objective DL-based DIR approach which leverages the VoxelMorph framework, called DL-MODIR, with MOREA, an evolutionary algorithm-based, multi-objective DIR approach in which a FEM-like biomechanical mesh transformation model is used. In our proposed hybrid approach, the DL results are used to smartly initialize MOREA, with the aim of more efficiently optimizing its mesh transformation model. We empirically compare DL-MOREA against its components, DL-MODIR and MOREA, on CT scan pairs capturing large bladder filling differences of 15 cervical cancer patients. While MOREA requires a median runtime of 45 minutes, DL-MOREA can already find high-quality transformations after 5 minutes. Compared to the DL-MODIR transformations, the transformations found by DL-MOREA exhibit far less folding and improve or preserve the bladder contour distance error. a]Dept. of Radiation Oncology, Leiden University Medical Center (LUMC), The Netherlands b]Leiden Institute of Advanced Computer Science (LIACS), Leiden University, The Netherlands c]Evolutionary Intelligence Group, Centrum Wiskunde & Informatica (CWI), The Netherlands keywords Deformable image registration, multi-objective optimization, deep learning, evolutionary algorithms † † authorinfo: Correspondence: Georgios Andreadis ([email protected]) and/or Tanja Alderliesten ([email protected]). 1 INTRODUCTION Many applications in image-guided radiation treatment could benefit from the transfer of information between multiple medical images of the same patient. However, local tissue deformations or content mismatch can hinder this transfer, requiring deformable image registration (DIR) to establish a spatial correspondence between the images. In this work, DIR entails registering a source image onto a target image. As this task has been shown to be inherently multi-objective [ 1 ] , a set of conflicting objectives should be optimized, simultaneously. The aim of multi-objective DIR is to find a so-called approximation set of transformations that covers the trade-off front of Pareto optimal transformations spanned by these objectives. To be of value, the transformations must be biomechanically plausible (i.e., realistic). Furthermore, to be useful in various clinical applications, they typically need to be calculated within a short time frame. These two requirements, however, can be in conflict. Two predominant types of DIR approaches have emerged, each excelling in one of these two requirements. On the one hand, DIR approaches using deep learning (DL) techniques [ 2 , 3 , 4 ] , such as the recently introduced multi-objective DL-MODIR approach [ 5 ] , can instantly predict a transformation for a given registration problem, after being trained on a set of prior problems. On problems with large deformations and content mismatch, however, this speed often comes at the expense of transformation realism, as current prediction models are not grounded in a realistic transformation model. On the other hand, DIR approaches which use biomechanical, finite element modeling (FEM) techniques [ 6 , 7 , 8 ] can find more realistic transformations, but often require a longer runtime. One example is the recently proposed MOREA [ 9 ] , an evolutionary algorithm-based, multi-objective DIR approach using a FEM-like biomechanical mesh transformation model. Until now, these two types of DIR approaches have always been considered separately. On the path to widespread clinical adoption of DIR, however, an approach is needed that overcomes the limitations of both approach types. In this work, we introduce a hybrid approach to combine the best of both worlds. The hybrid, called DL-MOREA and illustrated in Figure 1 , uses the transformations found by the DL-based DL-MODIR approach to smartly initialize the biomechanical FEM-like mesh model used by the MOREA approach, to more efficiently optimize the model. The MOREA approach then refines these transformations, reducing the required runtime for MOREA to reach high-quality solutions. Because of the multi-objective nature of both approaches, a comprehensive transfer of information is possible. We compare the hybrid to its components in experiments on pelvic CT scans of 15 patients by analyzing their transformations quantitatively and qualitatively. Figure 1: Illustration of the proposed hybrid DL-MOREA DIR approach and its main components. 2 MATERIALS AND METHODS Figure 2: Reference source and target renders of the three selected patients. Shown are annotated sagittal slices. 2.1 Dataset The registration problems in this study are derived from pelvic CT scans of 75 cervical cancer patients, acquired for radiation treatment planning purposes, using a Philips Brilliance Big Bore scanner. For each patient, we include a source image with a full bladder and a target image with an empty bladder, taken shortly thereafter. For better visibility, the empty bladder was typically imaged after a bladder fluid contrast agent was administered. This was the case in 64 of the 75 patients. On both source and target images, 7 organs are delineated by a radiation therapy technologist (RTT), as seen in Figure 2 . The image pair is resampled to a resolution of 3\times 3\times 3 mm and rigidly registered to align the bony anatomies, using the bone contours. Both images are then cropped to an axis-aligned bounding box around the bladder with a 30 mm margin, using the maximal bounds from either image. To comply with the dimensionality requirements of DL-MODIR, all images are padded to in-plane dimensions of 96\times 96 voxels and a minimum depth of 48 voxels. If images exceed a depth of 48, overlapping fragments of depth 48 are prepared. The total patient cohort is divided into a training and validation set of 60 patients, with no. 2–61, and a test set of 15 patients, with no. 62–76. From the test set, three representative patients are automatically chosen for illustration by clustering the patients on their relative bladder volume change V_{t}/V_{s} , where V_{s} is the source bladder volume and V_{t} is the target bladder volume. A V_{t}/V_{s} value of 1.0 therefore indicates no change between the bladder volumes, while a value lower than 1.0 represents a target bladder volume that is smaller than its corresponding source bladder volume. We employ this relative change as an indicator of their registration difficulty. Using K-Means clustering, we identify three clusters, with a small, medium, and large relative volume change, respectively. The relative bladder volume change of each patient and their corresponding cluster is depicted in Figure 3 . The derived clustering thresholds of the relative bladder volume change metric are: 0.5 (between the small change cluster and the medium change cluster), and 0.3 (between the medium change cluster and the large change cluster). Figure 3: Scatter plot of the source and target volumes of the bladder of each tested patient. Patients are clustered into three clusters by their relative volume change between source and target, indicated in color in the plot. The