Semantic-aware Random Convolution and Source Matching for Domain Generalization in Medical Image Segmentation Journal: Computer Methods and Programs in Biomedicine (currently under revision)
Franz Thaler, Martin Urschler, Mateusz Kozinski, Matthias AF Gsell, Gernot Plank, Darko Štern · 2025
Read Semantic-aware Random Convolution and Source Matching for Domain Generalization in Medical Image Segmentation Journal: Computer Methods and Programs in Biomedicine (currently under revision) by Franz Thaler, Martin Urschler, Mateusz Kozinski, Matthias AF Gsell, Gernot Plank, Darko Štern (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01510.txt.
Introduction
Franz Thaler and co-authors teach scan models to work across scan types. A model trained on CT scans often fails on MR scans. In real care, the new scan type comes with no labels. Retraining is not an option at test time.
Their method trains on one source type alone. Random convolution with semantic care varies the training images. Source matching aligns traits across styles. The net then segments the new type cold. Cross-type skill would save huge labeling costs when clinics switch scanners or protocols. No target-domain images are used at all in training, which matches clinics that cannot share scans across sites for legal reasons.
Catalog record
| CREATOR | Franz Thaler, Martin Urschler, Mateusz Kozinski, Matthias AF Gsell, Gernot Plank, Darko Štern |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-01510
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Semantic-aware Random Convolution and Source Matching for Domain Generalization in Medical Image Segmentation Journal: Computer Methods and Programs in Biomedicine (currently under revision) Franz Thaler, Martin Urschler, Mateusz Kozinski, Matthias AF Gsell, Gernot Plank, Darko Štern Abstract Background and Objective: We tackle the challenging problem of single-source domain generalization (DG) for medical image segmentation. To this end, we aim for training a network on one domain (e.g., CT) and directly apply it to a different domain (e.g., MR) without adapting the model and without requiring images or annotations from the new domain during training. Methods: We propose a novel method for promoting DG when training deep segmentation networks, which we call SRCSM. During training, our method diversifies the source domain through semantic-aware random convolution, where different regions of a source image are augmented differently, based on their annotation labels. At test-time, we complement the randomization of the training domain via mapping the intensity of target domain images, making them similar to source domain data. Results: We perform a comprehensive evaluation on a variety of cross-modality and cross-center generalization settings for abdominal, whole-heart and prostate segmentation, where we outperform previous DG techniques in a vast majority of experiments. Additionally, we also investigate our method when training on whole-heart CT or MR data and testing on the diastolic and systolic phase of cine MR data captured with different scanner hardware, where we make a step towards closing the domain gap in this even more challenging setting. Conclusions: Overall, our evaluation shows that SRCSM can be considered a new state-of-the-art in DG for medical image segmentation and, moreover, even achieves a segmentation performance that matches the performance of the in-domain baseline in several settings. The source code of SRCSM will be made publicly available upon acceptance. 1 Introduction Deep neural networks are known to underperform in the presence of domain shift, especially when test data has been generated from a vastly different distribution than the data used to train the network [ 5 , 55 ] . Domain shift has a strong impact in the context of medical imaging tasks, where the same anatomy can be captured with different imaging modalities, but models trained in one modality, e.g., Magnetic Resonance (MR) , may even fail completely when applied to another modality, e.g., Computed Tomography (CT) [ 1 , 46 ] . Consequently, to achieve state-of-the-art performance, medical imaging applications require annotated data from all target domains, which is often prohibitively costly due to the expertise required to produce the annotations. Single-source Domain Generalization (DG) methods [ 68 , 57 ] seek to address this problem by training-time techniques that promote generalization of the network to data from unseen domains. Many existing approaches rely on strong, randomized source data augmentation [ 65 , 60 , 10 , 43 , 51 ] or adversarial strategies [ 47 , 59 , 61 , 8 ] to synthetically diversify the training data. In doing so, these strategies aim to expand the model’s coverage towards unseen test data distributions by making the model more robust towards synthetically introduced differences. However, ensuring coverage of unseen test domains is inherently difficult. As shown in our evaluation in Section 5.2 , existing methods are still far away from closing the performance gap between the source and target domains. Figure 1 : We propose SRCSM, a single-source cross-modality domain generalization approach that (1) during training, expands the source data distribution ( \mathcal{S} ) through our Semantic-aware Random Convolution (SRC), and (2) at test-time, shifts images from unseen target domains ( \mathcal{T}_{1} and \mathcal{T}_{2} ) towards the source domain ( \mathcal{S} ) via our Source Matching (SM) strategy. To reduce this gap, we propose a single-source DG method for semantic segmentation based on two key contributions: First, we note that appearance differences between images acquired with different modalities are largely determined by the tissue type. Consequently, augmentation strategies that are oblivious to tissue types are unlikely to produce appearance changes that are expected between source and target domain data. We address this by introducing Semantic-aware Random Convolution (SRC) , where a distinct, highly non-linear random augmentation is applied to each region of source domain data that corresponds to a different ground truth label. Second, we argue that it is much easier to shift test data towards the known distribution of the training data than to sufficiently augment the training data to ensure coverage of any unknown target domain. Driven by this observation, we propose to remap the intensity of test images at test-time in order to make them more similar to those of the training domain, which we call Source Matching (SM) . We combine these two contributions with conventional augmentation strategies and propose a single-source DG method named Semantic-aware Random Convolution and Source Matching (SRCSM) . An overview of our method is provided in Fig. 1 . To assess the performance of SRCSM , we perform an extensive evaluation including cross- modality and cross-center generalization on four abdominal, eight cardiac and six prostate domains in multiple settings, resulting in a large variety of source-target combinations. Notably, we extend the common abdominal organ segmentation setting by using two additional, significantly larger domains: AMOS CT and AMOS MR [ 24 ] . Further, besides the widely used four label setting for whole-heart cardiac segmentation employed in literature, we also evaluate a more detailed setting that uses seven labels. Lastly, to the best of our knowledge, we are the first to benchmark generalization in the challenging setting, where a model is trained on whole-heart CT or whole-heart MR data and evaluated on cardiac cine MR data. This setting results in a large and multifaceted domain gap, dramatically exceeding domain shifts evaluated by prior work. In this massive set of experiments, SRCSM outperforms a wide range of existing methods in a vast majority of cases, setting a new state-of-the-art for cross-modality and cross-site single-source DG . Additionally, SRCSM even outperforms several Domain Adaptation (DA) and Test-time Adaptation (TTA) approaches that have an advantage over single-source DG methods as they adapt the model on target domain data. Most notably, SRCSM significantly alleviates the impact of domain shift by achieving results that are close to or even in-line with the in-domain performance. We summarize our contributions as follows: 1. We introduce Semantic-aware Random Convolution (SRC) , a highly nonlinear augmentation strategy that accounts for semantic labels. 2. We propose to use an intensity quantile mapping called Source Matching (SM) to align target domain images to the source domain at test-time. 3. We extend the popular abdominal organ segmentation task for single-source DG by adding a second CT and a second MR domain, leading to the most comprehensive evaluation on abdominal DG reported so far. 4. To the best of our knowledge, we are the first to evaluate generalization from whole-heart MR and CT data to diastolic and systolic cardiac cine MR data. 5. With our SRCSM approach, we establish a new state-of-the-art in abdominal organ, whole-heart and prostate segmentation, and achieve results that close in on or even match the in-domain baseline. 2 Related Work In section 2.1 , we review related techniques that adhere to the DG scenario. To put our method in a broader context, in