Learning to segment anatomy and lesions from disparately labeled sources in brain MRI

Meva Himmetoglu, Ilja Ciernik, Ender Konukoglu, for the Alzheimer’s Disease Neuroimaging Initiative · 2025

Read Learning to segment anatomy and lesions from disparately labeled sources in brain MRI by Meva Himmetoglu, Ilja Ciernik, Ender Konukoglu, for the Alzheimer’s Disease Neuroimaging Initiative (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18840.txt.

Introduction

Himmetoglu and co-authors present a 2025 method to outline both healthy tissue and lesions in brain MRI scans, trained on data sets that never label both at once. Lesions warp anatomy, which trips standard tools, and jointly labeled training sets are rare. Their method splits the two jobs into two paths.

One path maps healthy tissue, the other finds lesions, and an attention step merges their reads across scan types. At test time, the model adapts to each image to blunt the lesion's pull on tissue calls. Meta learning bakes that adaption into training. Glioblastoma scans showed gains over top rivals.

Catalog record

CREATORMeva Himmetoglu, Ilja Ciernik, Ender Konukoglu, for the Alzheimer’s Disease Neuroimaging Initiative
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-18840

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning to segment anatomy and lesions from disparately labeled sources in brain MRI Meva Himmetoglu, Ilja Ciernik, Ender Konukoglu, for the Alzheimer’s Disease Neuroimaging Initiative Abstract Segmenting healthy tissue structures alongside lesions in brain Magnetic Resonance Images (MRI) remains a challenge for today’s algorithms due to lesion-caused disruption of the anatomy and lack of jointly labeled training datasets, where both healthy tissues and lesions are labeled on the same images. In this paper, we propose a method that is robust to lesion-caused disruptions and can be trained from disparately labeled training sets, i.e., without requiring jointly labeled samples, to automatically segment both. In contrast to prior work, we decouple healthy tissue and lesion segmentation in two paths to leverage multi-sequence acquisitions and merge information with an attention mechanism. During inference, an image-specific adaptation reduces adverse influences of lesion regions on healthy tissue predictions. During training, the adaptation is taken into account through meta-learning and co-training is used to learn from disparately labeled training images. Our model shows an improved performance on several anatomical structures and lesions on a publicly available brain glioblastoma dataset compared to the state-of-the-art segmentation methods. 1 Introduction Segmentation of structural brain MRI of patients suffering from brain lesions plays an important role in scientific research, quantitative analysis, follow-ups and treatment planning. So far, most of the research focused on accurate lesion segmentation [ 21 , 22 ] , leaving aside the segmentation of the healthy structures in the same brain. Meanwhile, it has been reported that quantification of many structures along the lineation of the lesion as well as farther areas may play an important role in diagnosis, assessing progression and treatment effects, by providing additional indicators from healthy structures [ 16 , 24 , 33 ] . For example, atrophy is seen as a clinically relevant component of disease progression in multiple sclerosis [ 4 ] and may be related to cognitive impairment observed post-radiotherapy in brain tumor patients [ 33 ] . Despite the remarkable performance of advanced algorithms for healthy tissue segmentation in lesion-free brain MRI, their performance degrades when applied on patient images with lesions naïvely because lesions can change the appearance and introduce a domain shift for trained models [ 26 ] . A straightforward approach of tackling segmentation of lesions and structures simultaneously is with supervised learning [ 28 ] . However, this approach requires labeling healthy structures of interest in patient images, and for different types of lesions. This is clearly expensive and time consuming. It is also most likely the reason why publicly available datasets containing both healthy structure and lesion annotations on the same images are very rare. Furthermore, the supervised approach is limited with the structure set used during labeling. Adding new healthy structures would require relabeling the lesion images. An alternative, and arguably more resourceful approach is to use the available datasets focusing on each of the tasks separately: learning healthy tissue segmentation from the publicly available volunteer-based datasets, where healthy tissue labels in large numbers can be obtained with available models designed for this task ( [ 39 , 26 ] ), and lesion segmentation from patient data sets, where only lesion labels are available ( [ 3 , 2 , 27 ] ). Models that can jointly segment lesion and healthy tissue on the same brain and that could be trained from such disparately labeled task-specific datasets can be tremendously useful. Moreover, for new lesion types, generating only lesion labels for a set of images would suffice to train such a joint segmentation model. A few recent attempts aimed to solve the joint segmentation problem using disparately labeled datasets. The proposed methods leveraged: (i) the availability of multi-sequence acquisitions, dominantly employed in patient imaging, and (ii) the presence of shared sequences in most volunteer and patient datasets, at the least T1w. In [ 8 ] , authors define a Bayesian model and solve the resulting optimization problem to segment both healthy structures and lesions in multiple contrasts and resolutions. In [ 6 , 5 ] , authors achieve robustness for wild variations in images through extensive domain randomization in training. Authors in [ 12 ] , use shared T1w sequences in healthy and patient images, and assume lesions would not affect the segmentation predictions done based on the T1w sequence. Evidence given in their articles, as well as our initial experiments, suggest that this is indeed the case for lesions that do not grossly disrupt the brain anatomy, such as white matter hyper-intensity and multiple sclerosis lesions. However, lesions that cause gross disruptions, such as brain tumors, cannot be handled by this approach because the assumption that the lesion does not affect the segmentation prediction on the T1w sequence no longer holds. In this work, we develop a new algorithm to exploit multi-sequence datasets; we use multiple available sequences to extract lesion information and the T1w sequence to extract healthy structure information. There are two main differences compared to previous work, and in particular to the closer previous attempt in [ 12 ] . First, we propose to learn to fuse features extracted from different sequences through an attention mechanism, inspired from [ 43 ] that proposed a multi-sequence segmentation approach for liver tumor segmentation from multi modal computer tomography images. This minimizes adverse effects of concatenating different sequences during feature extraction process. For instance, when using patient and volunteer datasets together, where the former often has more sequences than the latter, our approach removes the need to synthesize lesion-specific sequences for volunteer samples. Moreover, this strategy allows developing separate paths for tissue and lesion segmentation, each pre-trained for the respective tasks, and help attain higher segmentation accuracy for lesions when fused. Second, we employ image-specific adaptation at inference time to reduce the influence of lesion areas on predictions of healthy tissue segmentation. The adaptation takes into account a preliminary lesion segmentation into account to improve robustness of healthy tissue segmentation to lesion-caused disruptions on the anatomy. The adaptation process is taken into account during training through a meta-learning strategy. 2 Related Work Multisequence datasets offer rich information that can be harnessed for more accurate segmentation. Effective integration of information from these different sequences, such as T1w, FLAIR, contrast enhanced T1w and T2w, is important for the segmentation task, since each sequence can offer complementary information about the anatomy and the pathology. There are two particularly relevant points we would like to highlight. First, imaging studies for brain lesions often include multiple MRI sequences to display the complementary information. Today there are publicly available multi-sequence MRI datasets for a number of lesions including lesion segmentation labels. These datasets enabled development of accurate lesion segmentation models using multiple sequences [ 18 , 40 ] and even when some sequences are missing [ 42 ] , [ 11 ] . Second, healthy structure segmentation in brain MRI today is mostly based on the T1w sequence, in which the contrast between gray and white matter is strong. There are publicly available volunteer and neurodegeneration-focused datasets, which have large number of T1w images, cover a wide age-span and are acquired from individuals without gross lesions. Well established analysis toolboxes, such as Freesurfer [ 14 ] , SPM

Read the complete text