nnLandmark: A Self-Configuring Method for 3D Medical Landmark Detection
Alexandra Ertl, Shuhan Xiao, Stefan Denner, Robin Peretzke, David Zimmerer, Peter Neher, Fabian Isensee, Klaus H. Maier-Hein · 2025
Read nnLandmark: A Self-Configuring Method for 3D Medical Landmark Detection by Alexandra Ertl, Shuhan Xiao, Stefan Denner, Robin Peretzke, David Zimmerer, Peter Neher, Fabian Isensee, Klaus H. Maier-Hein (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06742.txt.
Introduction
Alexandra Ertl and co-authors present nnLandmark, a self-configuring system that finds anatomical landmarks in three-dimensional medical scans. Landmarks anchor treatment planning and body measurements, but marking them by hand is slow and expert work. Deep learning can automate the task, yet scattered benchmarks and inconsistent baselines have made methods hard to compare fairly.
Built on the engine of a well-known segmentation framework, nnLandmark tunes its own heatmap generation, loss design, and inference settings for each new dataset. It reached state-of-the-art results across five public datasets and one private one, against three recent rival methods. The authors pitch it both as a ready tool for labs without tuning expertise and as a common baseline for testing new ideas.
Catalog record
| CREATOR | Alexandra Ertl, Shuhan Xiao, Stefan Denner, Robin Peretzke, David Zimmerer, Peter Neher, Fabian Isensee, Klaus H. Maier-Hein |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-06742
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
nnLandmark: A Self-Configuring Method for 3D Medical Landmark Detection Alexandra Ertl, Shuhan Xiao, Stefan Denner, Robin Peretzke, David Zimmerer, Peter Neher, Fabian Isensee, Klaus H. Maier-Hein Abstract Landmark detection plays a crucial role in medical imaging tasks that rely on precise spatial localization, including specific applications in diagnosis, treatment planning, image registration, and surgical navigation. However, manual annotation is labor-intensive and requires expert knowledge. While deep learning shows promise in automating this task, progress is hindered by limited public datasets, inconsistent benchmarks, and non-standardized baselines, restricting reproducibility, fair comparisons, and model generalizability. This work introduces nnLandmark, a self-configuring deep learning framework for 3D medical landmark detection, adapting nnU-Net to perform heatmap-based regression. By leveraging nnU-Net’s automated configuration, nnLandmark eliminates the need for manual parameter tuning, offering out-of-the-box usability. It achieves state-of-the-art accuracy across two public datasets, with a mean radial error (MRE) of 1.5 mm 1.5\text{\,}\mathrm{mm} on the Mandibular Molar Landmark (MML) dental CT dataset and 1.2 mm 1.2\text{\,}\mathrm{mm} for anatomical fiducials on a brain MRI dataset (AFIDs), where nnLandmark aligns with the inter-rater variability of 1.5 mm 1.5\text{\,}\mathrm{mm} . With its strong generalization, reproducibility, and ease of deployment, nnLandmark establishes a reliable baseline for 3D landmark detection, supporting research in anatomical localization and clinical workflows that depend on precise landmark identification. The code will be available soon. 1 1 footnotetext: Equal contribution 1 Motivation Accurate detection of anatomical landmarks is critical for several medical imaging applications, including diagnosis, treatment planning, image registration and surgical navigation [ 1 , 18 , 17 , 19 , 20 ] . Annotations typically involve between 10 and up to 50 landmarks in a single image, making the process both time-consuming and highly dependent on expert anatomical knowledge, especially in 3D imaging data. Deep-learning-based methods already demonstrated great potential in automating this task. A common approach to detecting landmarks is regressing heatmaps, whereby each landmark is represented by a Gaussian blob in a dedicated channel [ 13 , 14 ] . In the prediction, the target voxel is then identified by finding the channel-wise maximum. This approach has shown to outperform direct coordinate regression due to more effective processing of spatial information. However, despite active methodological research in 3D medical landmark detection, the thorough validation of new methods is compromised by an absence of standardized baselines and public benchmarks. As a result, existing models are often designed only on a single private dataset, hindering transparent, fair comparisons and raising the question of dataset-overfitting and generalization to other datasets [ 4 , 3 , 6 , 7 , 8 , 15 , 17 , 18 , 21 ] . Given the complexity of medical imaging, including variations in imaging modalities and anatomical structures, the development of robust and generic methods is critical [ 17 , 18 ] . And while many publications compare their methods to a 3D U-Net [ 5 ] , variations in hyperparameters and implementation details can significantly affect the performance, even when using the same architecture [ 6 , 7 , 8 ] . Additionally, model development and parameter tuning are time-consuming and computationally expensive, especially in 3D medical imaging due to the increased number of parameters and higher memory demands compared to 2D methods. In segmentation, this has been addressed by nnU-Net [ 8 ] , a self-configuring framework for medical image segmentation. Key components of the nnU-Net encompass a set of fixed hyperparameters, which have shown to be robust across datasets as well as a set of rule-based parameters, which automatically adapt to new datasets. Its out-of-the-box usability thereby mitigates the need for manual hyperparameter tuning. nnU-Net achieves state-of-the-art performance on various datasets, outperforming most recent developments and specialized methods [ 8 , 9 ] . In this work, we present nnLandmark, an adaptation of nnU-Net for 3D medical landmark detection. We are the first to: • Establish a robust, self-configuring method for 3D medical landmark detection by adapting nnU-Net for Gaussian blob heatmap regression. • Achieve state-of-the-art accuracy across diverse public datasets (CT and MRI), surpassing prior methods and on par with inter-rater variabilities. • Provide a strong, standardized baseline for landmark detection research, supporting fair comparisons and helping to determine true methodological advances. • Offer an open-source, out-of-the-box solution to promote transparency, reproducibility, and real-world clinical adoption. Figure 1: Overview of the proposed nnLandmark approach, leveraging key characteristics of the nnU-Net for heatmap-based 3D medical landmark detection by adjusting the respective fixed parameters. Figure 2: The landmark segmentations are transformed to heatmaps at the end of the data-augmentation pipeline. Thereby each landmark is represented by a Gaussian blob in a dedicated channel. In the postprocessing, the exact positions of the landmarks are then identified by taking the channel-wise maximum. 2 Method For our method nnLandmark, we build upon the well-established nnU-Net framework to enable heatmap-based landmark detection. nnU-Net is widely adopted in the medical image segmentation community, consistently demonstrating state-of-the-art performance across various datasets and outperforming most other methods, including recent developments [ 8 , 9 ] . A key concept of nnU-Net is its automatic configuration of a set of rule-based parameters, such as image normalization or network topology, depending on dataset-specific characteristics. Our aim is to retain as much of the original nnU-Net and only make selective changes for landmark detection to benefit from its proven design and keep its self-configuring approach. Our changes concern predominantly fixed parameters, such as adding a heatmap transform to the data loader and adapting training parameters to handle the challenges of landmark detection. An overview of our modifications is shown in Figure 1 , while Figure 2 outlines the handling of landmark labels. The following section explains key implementation details of our method. To ensure compatibility with nnU-Net’s experiment planning and preprocessing, the landmark labels are initially in the format of a multi-class segmentation. During training, these segmentation labels are transformed into heatmaps, with each landmark assigned its own channel. Each landmark is represented as a Gaussian blob—with a standard deviation of \sigma=4 and normalized between 0 and 1—whose exact position is determined by computing the center of mass of the corresponding segmentation. This conversion is applied as the final transformation in the data augmentation pipeline. In the final layer of the network, a sigmoid activation function is added to constrain the predicted voxel values between 0 and 1, thus stabilizing the training for heatmap regression. The number of network output channels is set to the number of landmarks. The model is trained using the Mean Squared Error (MSE) loss function to regress the continuous heatmap values. We chose the Adam optimizer to better handle the strong class imbalance in multi-channel heatmap regression, where each channel contains an entire volume with only a relatively small Gaussian blob foreground. The adaptive learning rate mechanism of Adam, designed to adjust updates based on first and second moment estimates, is particularly effective in managing the relatively sparse foreground signals and mitigating the influence of overwhelming background noise [ 10 ] . This choice provided increased learning stability compared to nnU-Net’s original use of Stochastic Gradient Descent (SGD). In the postprocessing, landmark coordinates are extracted from the predicted heatmaps by identifying the channel-wise maximum. To address potential left-right confusion, a post-processing step compares