Auto-nnU-Net: Towards Automated Medical Image Segmentation
Jannis Becktepe †, Leona Hennig, Steffen Oeltze-Jafra, Marius Lindauer · 2025
Read Auto-nnU-Net: Towards Automated Medical Image Segmentation by Jannis Becktepe †, Leona Hennig, Steffen Oeltze-Jafra, Marius Lindauer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16561.txt.
Introduction
Jannis Becktepe, Leona Hennig, Steffen Oeltze-Jafra, and Marius Lindauer auto-tune a top scan sort frame in a 2025 paper on care AI. Scan sorts span bone to organ tasks, each with its own snags for the best fit in the ward. The base frame sets much by fixed knobs and craft rules, which caps its reach in real use.
Their full-auto frame adds knob hunts and form hunts in tiers for each task at hand. A cost-wise hunt rule weighs grade gain next to train burn for real wards with tight means. Tests span a wide set of scan tasks and sets from a few care fields.
Catalog record
| CREATOR | Jannis Becktepe †, Leona Hennig, Steffen Oeltze-Jafra, Marius Lindauer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-16561
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Auto-nnU-Net: Towards Automated Medical Image Segmentation Jannis Becktepe †, Leona Hennig, Steffen Oeltze-Jafra, Marius Lindauer Abstract Medical Image Segmentation (MIS) includes diverse tasks, from bone to organ segmentation, each with its own challenges in finding the best segmentation model. The state-of-the-art AutoML-related MIS-framework nnU-Net automates many aspects of model configuration but remains constrained by fixed hyperparameters and heuristic design choices. As a full-AutoML framework for MIS, we propose Auto-nnU-Net, a novel nnU-Net variant enabling hyperparameter optimization (HPO), neural architecture search (NAS), and hierarchical NAS (HNAS). Additionally, we propose Regularized PriorBand to balance model accuracy with the computational resources required for training, addressing the resource constraints often faced in real-world medical settings that limit the feasibility of extensive training procedures. We evaluate our approach across diverse MIS datasets from the well-established Medical Segmentation Decathlon, analyzing the impact of AutoML techniques on segmentation performance, computational efficiency, and model design choices. The results demonstrate that our AutoML approach substantially improves the segmentation performance of nnU-Net on 6 out of 10 datasets and is on par on the other datasets while maintaining practical resource requirements. Our code is available at https://github.com/automl/AutoNNUnet . † † footnotetext: Work was conducted at Institute of AI, Leibniz University Hannover. Ap Ap [] 1 Introduction Machine learning (ML) plays a key role in modern healthcare, enabling accurate diagnoses ( Fauw et al., 2018 ; Bernard et al., 2018 ; Khan et al., 2023 ; Wang et al., 2024 ) , early cancer detection ( Cao et al., 2023 ) , and scientific discovery ( Falk et al., 2018 ) . Medical image segmentation (MIS) aims to identify anatomical structures in medical scans but is challenging due to datasets variations, class imbalances, and task-specific constraints ( Litjens et al., 2017 ; Isensee et al., 2020 ; Ali et al., 2024 ) . Self-configuring methods reduce the need for manual tuning by adapting models for a given dataset ( Ali et al., 2024 ) . nnU-Net ( Isensee et al., 2020 ) has emerged as a state-of-the-art framework that automatically configures U-Net-based architectures to achieve strong segmentation performance. However, nnU-Net surprisingly relies on some fixed hyperparameters and manually designed heuristics, which limit flexibility and may not always yield optimal results across datasets ( Bergstra & Bengio, 2012 ; Quinton et al., 2024 ) . In this work, we leverage automated machine learning (AutoML) ( 1 ) to address these challenges and perform a large-scale study on the impact of AutoML on MIS. We introduce Auto-nnU-Net, a novel variant of nnU-Net that integrates AutoML to enable hyperparameter optimization (HPO) and neural architecture search (NAS) for nnU-Net. By combining PriorBand ( Mallik et al., 2023 ) with multi-objective optimization ( Karl et al., 2023 ) , we introduce Regularized PriorBand for Joint Architecture and Hyperparameter Search (JAHS) ( Awad et al., 2023 ) , addressing the growing interest in resource efficiency in MIS as highlighted by recent work ( Rayed et al., 2024 ) . Our study evaluates Auto-nnU-Net on the Medical Segmentation Decathlon (MSD) datasets ( Simpson et al., 2019 ; Antonelli et al., 2022 ) , providing insights into the impact of optimization strategies, hyperparameter importance, and dataset characteristics. Notably, unlike most studies on AutoML for MIS ( Ali et al., 2024 ) , we report results for all ten MSD datasets, providing a more thorough assessment of generalizability and robustness across diverse medical segmentation challenges. In this work, we make the following contributions: 1. Auto-nnU-Net for AutoML-driven MIS. We propose a novel framework that automates key design decisions in nnU-Net for flexible and structured HPO and NAS. 2. Efficient optimization with Regularized PriorBand. We introduce Regularized PriorBand, which incorporates training runtime as an optimization objective to reflect real-world constraints, where limited resources and frequent retraining make efficient training essential. It selects slower models only if they improve accuracy, and inherently yields trade-off solutions. 3. Extensive evaluation across all ten MSD datasets. We analyze the impact of AutoML on segmentation accuracy, including hyperparameter importance and dataset transferability, enabling a deeper understanding of generalization behavior and guiding the design of more robust, efficient models across diverse medical imaging tasks. 2 Background on Image Segmentation Following Szeliski (2022) , semantic segmentation refers to partitioning an image into regions associated with specific classes. We use the term image segmentation interchangeably to describe this task, where each pixel is labeled to enable structured analysis of visual data. Medical image segmentation (MIS) involves partitioning medical images, e.g., magnetic resonance imaging (MRI) or computer tomography (CT) scans, to identify areas of interest, incl. organs or potentially malicious structures such as tumors ( Antonelli et al., 2022 ) . In practice, automated segmentation assists clinicians by accurately identifying critical areas for patient treatment ( Liang et al., 2019 ) . Recent MIS datasets focus on foreground classes, treating the background as a single, excluded class ( Menze et al., 2015 ; Heller et al., 2019 ; Simpson et al., 2019 ; Antonelli et al., 2022 ) . Unlike natural image segmentation, MIS faces challenges like limited availability of training data, class imbalances, small or branching anatomical structures, weak boundaries, and variable intensity distributions. The segmentation of 3D images consisting of voxels (volume pixels) from MRI and CT scans further increases the segmentation complexity and computational demands of MIS ( Isensee et al., 2020 ; Ali et al., 2024 ) . 3 Related Work In this section, we review previous work on core components of Auto-nnU-Net: self-configuring segmentation frameworks, HPO, NAS, and multi-objective optimization for MIS. Our work aims to unify them into a comprehensive AutoML framework tailored to the task at hand. nnU-Net Self-configuring frameworks address the challenge of designing and tuning MIS models for a given task and dataset. Building on U-Net’s success ( Ronneberger et al., 2015 ) , Isensee et al. (2020) introduced nnU-Net, which optimizes an U-Net for a given task. Like CASH ( Thornton et al., 2013 ) , it jointly selects training hyperparameters and the final model or ensemble for inference. nnU-Net achieves this by leveraging dataset meta-features common in AutoML ( Vanschoren, 2019 ) . We focus on the self-configuration mechanism of nnU-Net, omitting pre- and post-processing steps. The pipeline consists of three phases: (i) Experiment planning , where rule-based hyperparameter selection leverages dataset properties; (ii) Training , where 2D U-Net, 3D U-Net, and, if needed, a 3D U-Net cascade are trained using 5-fold cross-validation; (iii) Inference , where the best-performing model or ensemble is selected based on validation scores. nnU-Net relies on three types of hyperparameters: (i) Fixed (e.g., learning rate, optimizer, loss function); (ii) Rule-baesd (e.g., preprocessing, network topology); (iii) Heuristic (e.g., ensemble selection, post-processing). Hyperparameter Optimization (HPO) for MIS In general, there is little work on HPO for MIS. Yang et al. (2019) propose reinforcement learning for optimizing the data augmentation and learning rate of a 3D segmentation model. Quinton et al. (2024) apply HPO to different models, including nnU-Net, by subsequently performing Bayesian optimization (BO) for three groups of hyperparameters: (i) patch size, (ii) data pre-processing and augmentation, and (iii) loss function and optimizer. Neural Architecture Search (NAS) for MIS Various NAS methods have been adapted for MIS. Several approaches build on DARTS ( Liu et al.,