SSPFormer: Self-Supervised Pretrained Transformer for MRI Images

Jingkai Li, Xiaoze Tian, Yuhang Shen, Jia Wang, Dianjie Lu, Guijuan Zhang, Zhuoran Zheng · 2026

Read SSPFormer: Self-Supervised Pretrained Transformer for MRI Images by Jingkai Li, Xiaoze Tian, Yuhang Shen, Jia Wang, Dianjie Lu, Guijuan Zhang, Zhuoran Zheng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-12747.txt.

Introduction

Jingkai Li and six co authors adapt a pretrained transformer for MRI scans. These models do very well on photos of the world at large. MRI scans are a harder fit for two clear reasons. The shapes inside a body follow rules that photos do not teach. Scans are also scarce, since they are private health data.

Their model learns from scans with no labels at all, a trick called self supervision. That route needs no hand labels, which are slow and dear to make. A strong base model for MRI could lift many clinic tasks at once. Cuts, spots, and scans could all start from the same base. The work aims at a base model for health imaging as a whole.

Catalog record

CREATORJingkai Li, Xiaoze Tian, Yuhang Shen, Jia Wang, Dianjie Lu, Guijuan Zhang, Zhuoran Zheng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-12747

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SSPFormer: Self-Supervised Pretrained Transformer for MRI Images Jingkai Li, Xiaoze Tian, Yuhang Shen, Jia Wang, Dianjie Lu, Guijuan Zhang, Zhuoran Zheng Abstract The pre-trained transformer demonstrates remarkable generalization ability in natural image processing. However, directly transferring it to magnetic resonance images faces two key challenges: the inability to adapt to the specificity of medical anatomical structures and the limitations brought about by the privacy and scarcity of medical data. To address these issues, this paper proposes a Self-Supervised Pretrained Transformer (SSPFormer) for MRI images, which effectively learns domain-specific feature representations of medical images by leveraging unlabeled raw imaging data. To tackle the domain gap and data scarcity, we introduce inverse frequency projection masking, which prioritizes the reconstruction of high-frequency anatomical regions to enforce structure-aware representation learning. Simultaneously, to enhance robustness against real-world MRI artifacts, we employ frequency-weighted FFT noise enhancement that injects physiologically realistic noise into the Fourier domain. Together, these strategies enable the model to learn domain-invariant and artifact-robust features directly from raw scans. Through extensive experiments on segmentation, super-resolution, and denoising tasks, the proposed SSPFormer achieves state-of-the-art performance, fully verifying its ability to capture fine-grained MRI image fidelity and adapt to clinical application requirements. Figure 1: Schematic diagram of the proposed self-supervised pretrained Transformer framework tailored for MRI images. The framework integrates core components, including frequency-aware hierarchical masking and Fourier domain noise augmentation, aiming to learn domain-specific feature representations of MRI images efficiently. 1 Introduction Magnetic Resonance Imaging (MRI) has become a cornerstone of clinical diagnosis, disease monitoring, and medical research, thanks to its non-ionizing radiation and superior soft tissue contrast. It plays an irreplaceable role in tumor screening, neurological disorder assessment, and treatment plan formulation, where the precision of lesion localization (clinical requirement \leq 2 mm) and pathological grading directly depend on MRI image quality and analytical accuracy. With the deep integration of deep learning in medical imaging, downstream tasks such as 7 , super-resolution reconstruction 37 , and segmentation 20 have imposed higher requirements on the robustness, generality, and data efficiency of feature extraction, driving the exploration of more efficient image processing models. In Natural Language Processing (NLP) and Computer Vision (CV), Transformer-based large-scale pretraining paradigms (e.g., BERT 5 in NLP, ViT and MAE 21 ; 44 ; 29 in CV) have achieved revolutionary progress by learning universal features from large datasets. The emergence of such methods provides high-quality initial parameters for solving visual problems, but they still face certain limitations in the field of MRI: 1) Domain adaptation gap, where models pretrained on natural images fail to adapt to MRI’s unique frequency-domain characteristics 18 ; 15 —MRI high-frequency components (spatial frequency \geq 0.5 cycle/pixel) overlap with key anatomical structures by 89.7%, which is fundamentally different from natural images. 2 ) data scarcity constraint, as supervised learning relies heavily on large-scale annotated data 34 , while ethical regulations and scanning costs restrict MRI data acquisition, and annotations require professional physicians, resulting in extreme scarcity of high-quality labeled data (only 5% of clinical MRI data is annotated). 3) Artifact robustness deficit, since MRI images are prone to field inhomogeneity, motion artifacts, and frequency-dependent noise, which existing models trained on synthetic noise fail to handle effectively; finally, cross-task/cross-modality limitation, where task-specific models suffer from parameter redundancy and poor transferability, while multi-sequence MRI fusion often ignores frequency-domain correlations between modalities. To address these intertwined challenges, we propose SSPFormer ( S elf-supervised P re-trained T ransformer), a framework dedicated to MRI data. It comprises three core components. First, we curate a large-scale, privacy-compliant dataset ( MRI-110k ) covering major organs and multiple MRI sequences, enabling the learning of organ-agnostic, domain-specific representations. Second, SSPFormer employs three complementary, frequency-aware self-supervised objectives (see Figure 1 ): (i) inverse frequency-aware hierarchical masking; (ii) frequency-weighted FFT noise augmentation; and (iii) cross-modality frequency attention. Third, the architecture follows a “frozen shared Transformer backbone + lightweight task-specific heads/tails” design with asymmetric fine-tuning: the encoder is frozen to preserve universal anatomical priors 27 ; 9 , while the decoder and task heads are updated for fast clinical adaptation. Our main contributions are threefold: • We propose a frequency-aware self-supervised learning paradigm for MRI, explicitly integrating morphological frequency priors and real acquisition noise into pretraining. • We design three complementary core modules (inverse frequency masking, frequency-weighted noise augmentation, cross-modality frequency attention) to address MRI’s unique challenges. • Construct the largest unlabeled full-body MRI dataset with strict privacy compliance, supporting universal pretraining for multi-organ, multi-task MRI processing. Validate SSPFormer on five downstream tasks across neuro, body, and musculoskeletal imaging, showing superior performance with only 20% labeled data, meeting clinical requirements for precision and efficiency. Figure 2: Pipeline of the proposed self-supervised pretrained Transformer framework for MRI images (SSPFormer). The framework integrates core components, including inverse frequency-aware masking and frequency-weighted Fourier noise augmentation, enabling customized learning of domain-specific feature representations for MRI images. 2 Related Works 2.1 MRI Image Processing Medical image processing, encompassing tasks like super-resolution, denoising, and segmentation, has evolved through several paradigms. Early deep learning approaches predominantly relied on task-specific small models . For instance, SRCNN 13 ; 12 pioneered end-to-end CNN-based super-resolution, while subsequent works like VDSR 40 , EDSR 31 , and RCAN 1 improved performance via deeper architectures and attention mechanisms. Similarly, denoising methods such as FFDNet 47 and CBDNet 17 employed CNNs to predict noise residuals. Despite their success, these models face two intrinsic limitations: limited capacity (typically \leq 10 M parameters), hindering the learning of cross-task priors, and low data efficiency , as they are trained from scratch on small, paired datasets prone to overfitting. Inspired by the “pre-training–fine-tuning” paradigm from NLP, recent large-scale vision models have sought to unify low-level tasks. IPT 8 first unified multiple image restoration tasks within a Transformer framework, pre-training on 1M synthetic image pairs. SwinIR 32 and Restormer 46 further advanced this line by incorporating shifted windows and channel-wise attention to expand receptive fields. However, these methods still depend on synthetic paired data with fixed degradation kernels, which introduces a domain gap when applied to real-world, data-scarce modalities like MRI. To bridge this gap, we propose a Self-Supervised Pre-trained Transformer (SSPFormer) for MRI. Unlike prior works, SSPFormer is pre-trained entirely on unpaired and unlabeled MRI scans via a frequency-aware self-supervised objective that combines masked image modeling with FFT-based noise augmentation 48 . This forces a 97 Mparameter Swin-Transformer to learn robust, anatomically consistent representations. After pre-training, SSPFormer requires only 1/20 of the labeled data to outperform fully supervised task-specific models on super-resolution, denoising, and segmentation. Our core contribution is extending the “large model + large unlabeled data” paradigm from the natural image 21 domain to the medical frequency domain , offering a universal, data-efficient pre-training framework for privacy-sensitive 30 and data-scarce medical vision tasks 49 . 2.2 Pre-trained Transformer The Transformer and its variants have demonstrated strong potential in the field of unsupervised pre-training of natural images. For instance, ViT 14 models images by cutting them into blocks and performing pure attention modeling, MAE reconstructs original pixels through a high-proportion random masking, and BEiT completes visual BERT-style pre-training by leveraging discrete VAE tokens. Subsequent work further expanded the masking strategy from random blocks to multi-scale, edge-aware, or frequency-domain constraints to strengthen structural priors. However, these paradigms are

Read the complete text