Unified 3D MRI Representations via Sequence-Invariant Contrastive Learning
Liam Chalcroft 1, Jenny Crinion 2, Cathy J. Price 1, John Ashburner 1 · 2025
Read Unified 3D MRI Representations via Sequence-Invariant Contrastive Learning by Liam Chalcroft 1, Jenny Crinion 2, Cathy J. Price 1, John Ashburner 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-12057.txt.
Introduction
Liam Chalcroft, Jenny Crinion, Cathy Price, and John Ashburner teach one neural network to understand brain MRI scans no matter which settings produced them, in this 2025 paper. Three-dimensional MRI analysis lags behind ordinary image analysis because scans are scarce and networks trained on flat pictures miss depth. Worse, the same brain looks different under different scan sequences.
Their framework learns without labels from quantitative MRI, which measures tissue properties directly. From one such scan they simulate many contrast versions and force the network to give the same internal description for all of them, so it learns anatomy rather than sequence style. The single resulting encoder was tested on healthy brain segmentation, stroke lesion segmentation, and denoising. It beat rival self-supervised methods, with the largest gains when little labeled data was available, and it carried over to hospitals it had never seen. Code and trained models are public.
Catalog record
| CREATOR | Liam Chalcroft 1, Jenny Crinion 2, Cathy J. Price 1, John Ashburner 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-12057
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Unified 3D MRI Representations via Sequence-Invariant Contrastive Learning Liam Chalcroft 1, Jenny Crinion 2, Cathy J. Price 1, John Ashburner 1 Abstract Self-supervised deep learning has accelerated 2D natural image analysis but remains difficult to translate into 3D MRI, where data are scarce and pre-trained 2D backbones cannot capture volumetric context. We present a sequence-invariant self-supervised framework leveraging quantitative MRI (qMRI). By simulating multiple MRI contrasts from a single 3D qMRI scan and enforcing consistent representations across these contrasts, we learn anatomy-centric rather than sequence-specific features. This yields a robust 3D encoder that performs strongly across varied tasks and protocols. Experiments on healthy brain segmentation (IXI), stroke lesion segmentation (ARC), and MRI denoising show significant gains over baseline SSL approaches, especially in low-data settings (up to +8.3% Dice, +4.2 dB PSNR). Our model also generalises effectively to unseen sites, demonstrating potential for more scalable and clinically reliable volumetric analysis. All code and trained models are publicly available. \midlauthor \orcid 0000-0003-3363-6454 1 Department of Imaging Neuroscience, University College London, UK \orcid 0000-0001-8080-6562 2 Institute of Cognitive Neuroscience, University College London, UK \orcid 0000-0001-7605-2518 † † volume: – Under Review † † year: 2025 † † workshop: Full Paper – MIDL 2025 submission † † editors: Under Review for MIDL 2025 1 Introduction Deep learning has advanced medical image analysis across a range of tasks, from image registration Balakrishnan et al. 2019 to semantic segmentation Chalcroft et al. 2023 and visual question-answering Narayanan et al. 2024 . However, unique challenges arise when working with 3D MRI data, including increased computational demands and the difficulty of applying 2D pre-trained models to volumetric contexts Ma et al. 2024 . Although large-scale 3D datasets Li et al. 2024 and models Wu et al. 2024 have recently emerged, fine-tuning them for specific clinical tasks remains non-trivial due to inevitable domain shifts Yang et al. 2024 . Self-supervised learning (SSL) offers a promising means of learning robust representations without the need for large labelled datasets Chaitanya et al. 2020 ; Huang et al. 2023 . Yet, existing SSL methods often treat each MRI sequence as a separate domain, neglecting the shared anatomical information across contrast variations. In contrast, we leverage the observation that different MRI sequences, despite their unique contrast properties, encode the same underlying anatomy. By incorporating physics-based knowledge of MRI signal generation into an SSL framework, we can learn representations that are robust to sequence changes while preserving anatomical fidelity. In this paper, we make the following contributions: • We propose a sequence-invariant self-supervised learning framework based on physics-driven contrast simulation, enabling robust 3D MRI representations across multiple sequences. • We show significant performance gains in low-data regimes, with up to 8.3% Dice improvement in segmentation and a 4.2 dB PSNR increase for denoising when using just 1% of the training data. • We demonstrate enhanced cross-sequence generalisation and site adaptation from a single pre-trained encoder, particularly in out-of-domain scenarios. • We provide comprehensive evaluations on three diverse tasks - healthy brain segmentation, stroke lesion segmentation, and image denoising - highlighting the clinical utility of our approach. Our method addresses key problems in medical imaging by enabling robust feature learning across different sites and sequences, even with limited annotated data. This work takes a step towards developing more generalisable and clinically applicable models. We release all code and backbone weights at github.com/liamchalcroft/contrast-squared . 2 Related Work We briefly review three core areas that underpin this work: contrastive learning, robust representations in 3D medical imaging, and quantitative MRI (qMRI). 2.1 Contrastive Learning Self-supervised learning (SSL) can leverage unlabelled data by creating proxy tasks that encourage useful invariances in learned representations. Techniques include predictive coding ( van den Oord et al. 2019 ; Hénaff et al. 2020 ) , masked image modelling ( He et al. 2021 ; Xie et al. 2022 ) , and contrastive learning ( Chen et al. 2020 ; He et al. 2020 ) . Recent contrastive methods such as SimCLR ( Chen et al. 2020 ) and MoCo ( He et al. 2020 ) learn representations by aligning features from different augmented views, while BYOL ( Grill et al. 2020 ) and Barlow Twins ( Zbontar et al. 2021 ) reduce the reliance on explicit negative samples or introduce redundancy reduction. 2.2 Robust Representations in Medical Imaging Clinical MRI segmentation tasks face challenges when transferring models to new hospitals or protocols. Public benchmarks often involve a small set of consistent sequences, limiting models to scenarios where training and testing domains match (e.g. T1w-only). Real-world deployment must handle diverse sequences and acquisition conditions. Existing domain adaptation methods typically require multiple unlabelled images ( Wang et al. 2021 ) or prior knowledge of the target domain ( Dorent et al. 2023 ) , which is not always feasible. SynthSeg ( Billot et al. 2023 ) addresses this by randomising tissue contrast with synthetic data, with subsequent work showing the transferrability of the learned representations to new tasks ( Liu et al. 2024a ; Liu et al. 2024b ) . The performance of these methods is expected to be constrained by the quality of the synthetic data, which may fail to capture specific anatomical details desired in downstream tasks. Similarly, Meyer et al. 2021 adjust contrast on specific regions in real images, but this approach is restricted to modest in-domain variations rather than full sequence simulation. Meanwhile, Roschewitz et al. 2024 demonstrate that generating counterfactual views can boost domain robustness for 2D chest X-ray encoders. We extend these insights to 3D MRI for sequence-invariant representations. 2.3 Quantitative MRI Quantitative MRI (qMRI) acquires per-voxel parameter maps ( e.g. , R_{1} , R_{2}^{*} ) that govern the signal formation in conventional scans ( Weiskopf et al. 2021 ) . These maps facilitate the simulation of numerous synthetic MRI sequences from a single qMRI acquisition ( Tanenbaum et al. 2017 ) , improving model robustness under domain shift. For example, synthesised multi-contrast data has led to enhanced results in healthy brain parcellation ( Borges et al. 2021 ) and pathology segmentation ( Chalcroft et al. 2024 ) . Other methods rely on MR fingerprinting ( Ma et al. 2013 ) to derive similar quantitative maps ( Adams et al. 2024 ) , further expanding opportunities for sequence-invariant learning. 3 Methods We propose a sequence-invariant self-supervised learning (SSL) approach for robust 3D MRI representations. Figure 1 gives an overview of the method, which comprises: 1. A backbone encoder trained via contrastive learning on augmented volume pairs; 2. A decoder to reconstruct the original images, promoting semantic consistency for dense tasks; 3. A physics-based generative model to simulate multiple MRI sequences from quantitative MRI (qMRI) parameter maps, ensuring sequence invariance. \subfigure \subfigure Figure 1: Overview of the proposed SSL approach . ( 1 ) An MPRAGE volume is augmented into two random views. We extract a feature vector h via the backbone encoder, project it to z for a contrastive loss \mathcal{L}_{\text{contrastive}} , and use a decoder to optimise a reconstruction/inpainting loss \mathcal{L}_{\text{recon}} . ( 1 ) We generalise this by simulating multiple scanner sequences from qMRI parameter maps, enabling sequence-invariant learning. 3.1 Self-Supervised Learning We adopt SimCLR ( Chen et al. 2020 ) as our core contrastive framework, though