Effective Feature Learning for 3D Medical Registration via Domain-Specialized DINO Pretraining
Eytan Kats, Mattias P. Heinrich · 2026
Read Effective Feature Learning for 3D Medical Registration via Domain-Specialized DINO Pretraining by Eytan Kats, Mattias P. Heinrich (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14086.txt.
Introduction
Kats and Heinrich pretrain feature models on 3D medical images to improve image registration. Aligning scans across patients and scanners is hard for intensity-based methods. Their DINO-style self-supervised training runs on medical volumes, not natural photos. The learned features target deformable registration of abdominal MRI and CT scans. The specialized model beats the big DINOv2 model while using far less compute at test time. It also tops established registration models on out-of-domain tests.
Catalog record
| CREATOR | Eytan Kats, Mattias P. Heinrich |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17400 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.14086
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Effective Feature Learning for 3D Medical Registration via Domain-Specialized DINO Pretraining Eytan Kats, Mattias P. Heinrich Abstract Medical image registration is a critical component of clinical imaging workflows, enabling accurate longitudinal assessment, multi-modal data fusion, and image-guided interventions. Intensity-based approaches often struggle with inter-scanner variability and complex anatomical deformations, whereas feature-based methods offer improved robustness by leveraging semantically informed representations. In this work, we investigate DINO-style self-supervised pretraining directly on 3D medical imaging data, aiming to learn dense volumetric features well suited for deformable registration. We assess the resulting representations on challenging inter-patient abdominal registration task across both MRI and CT modalities. Our domain-specialized pretraining outperforms DINOv2 model trained on large-scale collection of natural images, while requiring substantially lower computational resources at inference time. Moreover, it surpasses established registration models under out-of-domain evaluation, demonstrating the value of task-agnostic yet medical imaging–focused pretraining for robust and efficient 3D image registration. Our code and pre-trained models are publicly available at https://github.com/EytanKats/med_ssl_3d . † † address: Institute of Medical Informatics, University of Lübeck, Germany 1 Introduction Image registration is a fundamental task in medical image analysis, playing a critical role in a wide range of clinical and research applications. By spatially aligning anatomical structures across different images or modalities, registration enables accurate longitudinal assessment of disease progression, supports multi-modal image fusion, and facilitates image-guided interventions. Reliable registration is therefore essential for quantitative analysis and informed clinical decision-making. Despite its importance, medical image registration remains challenging due to several intrinsic factors. Variability in image intensities across scanners and modalities, as well as complex non-linear anatomical deformations, make direct intensity-based alignment unreliable. To address these challenges, modern approaches leverage robust feature representations that capture high-level semantic and structural information. Hand-crafted descriptors, such as the Modality Independent Neighborhood Descriptor (MIND) [ 6 ] , have long been employed to extract discriminative and semantically meaningful features. Once feature maps are extracted, registration can be formulated as an optimization problem in feature space. Algorithms such as ConvexAdam [ 11 ] estimate dense deformation fields that optimally align these features. These methods benefit from the semantic consistency of learned representations, enabling more robust registration compared to intensity-based optimization. Pretraining models on large-scale datasets has recently demonstrated significant success in learning rich and transferable feature representations for a wide range of downstream tasks [ 5 , 8 , 16 ] . Models from the DINO family [ 8 ] utilize self-distillation objectives to produce embeddings that capture both local and global semantic information. Recent studies [ 12 , 3 ] show that DINO features can be effectively applied to 3D volumetric registration without any fine-tuning. In these approaches, features are extracted from 2D slices and concatenated to form a 3D feature volume, which is subsequently projected to a lower-dimensional space using PCA [ 4 ] . The resulting features are then aligned using ConvexAdam [ 11 ] . While these methods achieve promising registration accuracy, they require substantial upsampling of 2D slices to compensate for the relatively large patch size (14×14) of pre-trained DINO models, resulting in high computational costs and limiting their clinical applicability. In this work, we investigate pretraining DINO directly on 3D medical imaging data from scratch, with the goal of improving zero-shot registration performance. Our main contributions are summarized as follows: • We pretrain a transformer-based architecture using the self-distillation DINO framework on 3D medical data, incorporating design choices that enable computationally efficient inference for registration tasks. • We demonstrate that employing a smaller patch size is critical for achieving strong registration performance, as it preserves fine-grained structural details essential for accurate alignment. • We show that our pretrained model significantly outperforms DINOv2 and established registration models on challenging inter-patient abdominal registration tasks. Figure 1: Zero-shot registration framework. Features from the moving and fixed images are first extracted using the pretrained transformer and then projected into a low-dimensional space using PCA. ConvexAdam leverages these semantically rich representations to estimate an accurate displacement field. 2 Method Our approach consists of two stages. In the first stage, we pre-train a transformer-based model specifically designed for 3D medical images to learn dense, semantically rich feature representations in a self-supervised manner (Sec. 2.1 ). In the second stage, the pre-trained feature extractor is employed in a zero-shot registration framework, where the learned representations guide the alignment of volumetric images without additional fine-tuning (Sec. 2.2 ). 2.1 Pretraining on 3D medical data As the feature extractor, we employ the PRIMUS architecture [ 14 ] , an adaptation of the Vision Transformer (ViT) [ 2 ] specifically designed for 3D medical imaging tasks. For pretraining, the convolutional upscaling and decoding layers originally present in PRIMUS are removed, as our goal is to learn task-agnostic feature representations. The features are extracted from the output of the final transformer block, providing rich volumetric embeddings that capture spatial context. Following the DINOv2 self-distillation framework [ 8 ] , token embeddings are projected through a multi-layer perceptron (MLP) head to obtain high-dimensional feature vectors, upon which three complementary loss functions are jointly optimized to promote the emergence of semantically meaningful representations. The self-distillation loss aligns the projected class-token outputs between a student and a momentum-updated teacher network under different data augmentations, encouraging invariance to appearance and viewpoint changes. The iBOT loss [ 16 ] enforces local consistency between masked and unmasked patch tokens, enhancing spatial sensitivity. Finally, the KoLeo loss [ 10 ] acts as a diversity regularizer, maximizing the spread of feature representations on the hypersphere to prevent collapse and ensure uniform coverage of the embedding space. Together, these objectives guide the model toward learning volumetric representations specialized for the medical domain that capture both global semantic context and fine-grained anatomical details. These properties support accurate and robust dense correspondence estimation in downstream medical image registration task. 2.2 Zero-shot registration For registration, the features extracted from the final transformer block of the pretrained model are first projected into a low-dimensional space using PCA [ 4 ] , preserving the dominant components. These semantic feature volumes serve as input to the ConvexAdam algorithm [ 11 ] , which estimates the dense displacement field between fixed and moving images. An overview of the zero-shot registration pipeline is illustrated in Fig. 1 . ConvexAdam operates in two complementary stages, both of which depend critically on the quality of the feature representation. In the coupled convex stage, an initial deformation field is obtained by maximizing local feature correlations, yielding a robust coarse alignment. Building upon this initialization, the continuous refinement stage performs iterative optimization using Adam to minimize a sum-of-squared-differences (SSD) objective. This refinement progressively improves alignment while enforcing deformation smoothness through regularization. As both stages rely on the distinctiveness and semantic consistency of the feature embeddings, the overall effectiveness of ConvexAdam is tightly coupled to the quality of the learned representations. High-quality semantic features are therefore essential for achieving accurate and reliable registration. Figure 2: Comparison of registration performance across different feature extractors. The results highlight the advantage of domain-specific DINO-style pretraining over DINOv2 and MIND features. 3 Experiments We pretrain the feature extractor on a diverse collection of volumetric medical images, enabling the model to learn semantically rich feature representations. The learned representations are assessed in a