Learning Robust and Task-Invariant Functional Representation from fMRI through Siamese Self-Supervised Learning Journal: Medical Imaging Analysis
Jiyao Wang, Peiyu Duan, Nicha C. Dvornek, Lawrence H. Staib, Denis Sukhodolsky, Pamela Ventola, James S. Duncan · 2026
Read Learning Robust and Task-Invariant Functional Representation from fMRI through Siamese Self-Supervised Learning Journal: Medical Imaging Analysis by Jiyao Wang, Peiyu Duan, Nicha C. Dvornek, Lawrence H. Staib, Denis Sukhodolsky, Pamela Ventola, James S. Duncan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28990.txt.
Introduction
Jiyao Wang and colleagues build a light self-supervised framework for brain imaging that works when data are scarce and labels are noisy. Their BrainSimSiam learns from positive-only pairs, producing features that transfer across downstream classification and regression tasks. The features beat fully supervised baselines and approach large-scale models, without the heavy pretraining and fine-tuning costs of foundation models. The work targets functional MRI, where small samples, variable label quality, and high dimensionality raise overfitting risk.
Catalog record
| CREATOR | Jiyao Wang, Peiyu Duan, Nicha C. Dvornek, Lawrence H. Staib, Denis Sukhodolsky, Pamela Ventola, James S. Duncan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35148 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.28990
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning Robust and Task-Invariant Functional Representation from fMRI through Siamese Self-Supervised Learning Journal: Medical Imaging Analysis Jiyao Wang, Peiyu Duan, Nicha C. Dvornek, Lawrence H. Staib, Denis Sukhodolsky, Pamela Ventola, James S. Duncan Abstract Functional magnetic resonance imaging (fMRI) is a powerful tool for investigating human brain function. However, the high cost of data acquisition and the inherent subjectivity of psychiatric rating scales often lead to datasets with small sample sizes and variable label quality, especially when targeting a specific neurological condition. Combined with the inherently high dimensionality of fMRI data, these limitations substantially increase the risk of model overfitting. Recent years have seen growing interest in developing fMRI foundation models by combining multiple datasets; however, the computational resources needed for pretraining and fine-tuning are often prohibitive. We show that a lightweight self-supervised framework yields representations that generalize across diverse downstream tasks, outperforming fully supervised baselines and approaching the performance of large-scale models. We introduce BrainSimSiam, a data-efficient self-supervised representation learning framework that leverages positive-only data pairs to learn robust and generalizable features. We demonstrate that the learned representations achieve strong performance across multiple downstream classification and regression tasks, highlighting the potential of BrainSimSiam for data-limited neuroimaging applications. 1 Introduction Through the integration of deep learning and functional magnetic resonance imaging (fMRI), significant advances have been made in understanding neurological and psychiatric disorders such as Alzheimer’s disease [ 1 ] , attention deficit hyperactivity disorder (ADHD) [ 3 ] , and autism spectrum disorder (ASD) [ 24 ] . Although these disorders are inherently heterogeneous [ 11 , 13 , 7 ] , most supervised learning frameworks often rely on binary classification labels. Collapsing diverse disease subtypes and healthy controls (HC) into binary categories can obscure meaningful subtype differences, potentially leading to model overfitting and reduced interpretability. In addition, labels for neurological disorders are inherently noisy due to variability in disease severity and human factors in assessment. Although machine learning has driven substantial gains in medical imaging, models usually remain vulnerable to noise in labels [ 30 ] . In supervised learning settings, even with strong augmentation and regularization techniques [ 10 , 24 ] , overfitting can produce distorted decision boundaries, limiting the generalization capability of models. Label-independent approaches, especially unsupervised and self-supervised learning, offer a promising path to more robust representation learning and downstream performance. A wide range of machine learning architectures have been proposed for fMRI analysis, covering supervised [ 10 , 24 , 36 ] and self-supervised [ 28 , 38 , 21 , 37 , 45 ] paradigms. Despite this architectural diversity, most methods consider fMRI as a homogeneous sequence during acquisition for both resting-state and task-based data. In recent years, growing evidence indicates that task-based fMRI provides richer, behaviorally grounded information than resting-state fMRI data [ 17 , 44 ] . Moreover, the functional variation induced by diverse fMRI task paradigms yields informative contrasts that can be leveraged to learn robust, task-invariant functional representations. For data-limited applications, truncating task-based fMRI sequences into task-homogeneous subsequences also increases the effective sample size. In this work, we introduce Brain Simple Siamese representation learning (BrainSimSiam), a lightweight and data-efficient self-supervised framework for representation learning of fMRI. In a two-step training scheme, we show that the pretrained BrainSimSiam encoder produces more robust discriminative models in both multilayer perceptron (MLP) probing and end-to-end finetuning settings for downstream applications. Experiments are performed on classification and regression tasks including predictions of both biological and behavioral phenotypes. Our contributions are: 1. We develop a self-supervised representation learning framework for task-based fMRI that outperforms supervised and self-supervised baselines on both datasets examined. 2. We unify voxel-wise fMRI and graph-based functional views via a joint region of interest (ROI) masking scheme applied during training and post hoc interpretation. The same mechanism readily supports multimodal fusion by incorporating registered structural T1-weighted or diffusion MRI with the functional brain graph. 3. We leverage contrasts across diverse task stimuli as natural augmentations to learn task-invariant representations, disentangling evoked responses from spontaneous brain dynamics. 2 Related Work 2.1 Graph Neural Network (GNN) GNNs are a family of neural networks for graph-structured data [ 20 , 34 , 15 ] . In ROI-based brain graphs, nodes correspond to regions of interest and edges encode functional or structural connectivity [ 24 , 36 , 12 ] . GNNs learn by iteratively aggregating information from each node’s neighbors. Let h_{i}^{k} denote the feature of node i at layer k and \mathcal{N}(i) denote its neighbor set. A generic message-passing update of a GNN can be described as h_{i}^{k+1}=\phi\left(h_{i}^{k},m_{ij,j\in\mathcal{N}(i)}|m_{ij}=\psi\big(h_{i}^{k},h_{j}^{k},e_{ij}\big)\right) (1) where \psi computes messages m , e_{ij} are edge attributes, and \phi produces the updated node representation. 2.2 Simple Siamese (SimSiam) Contrastive frameworks such as MoCo v2 [ 5 ] and SimCLR [ 4 ] learn invariances by pulling together augmented views of the same sample while pushing apart others. They have shown greater robustness to label noise and spurious correlations than supervised pretraining, but typically rely on large numbers of negatives via large batches or memory queues. SimSiam [ 6 ] removes the need for negatives by using a stop-gradient Siamese architecture with a prediction head, achieving competitive performance in natural image applications with substantially simpler training. This property is also advantageous in neuroimaging settings characterized by modest batch sizes, where contrastive methods that depend on large negative sets are operationally fragile. Although negative examples can be informative, defining generalizable negatives in fMRI is challenging. Treating different subjects as negatives presumes inter-subject dissimilarity and discards shared functional patterns. Constructing negatives from task labels entangles features with the label space, leading to oversimplified and task-bound representations, undermining cross-task generalization. We therefore adopt SimSiam [ 6 ] as our framework for fMRI representation learning, a more suitable choice for learning generalizable representations across downstream tasks. 3 Data and Preprocessing 3.1 Human Connectome Project (HCP) Dataset We use task-based fMRI scans from the WU–Minn Human Connectome Project (HCP) S1200 young-adult release [ 32 ] with seven task paradigms: emotion, gambling, language, motor, relational, social, and working memory. All images were spatially normalized to the MNI152 standard space and resampled to 2 mm isotropic voxels. After excluding subjects with missing task runs or behavioral measures, the final cohort comprises 1,005 participants (Table 1 ). Evaluation is conducted with 5-fold cross-validation with equal numbers per fold. Number Percentage All Subjects 1005 100% Biological Sex Female 535 53% Male 470 47% Age (years) 22-25 216 21% 26-30 438 44% 31-35 341 34% 36+ 10 1% Table 1: Statistics of subjects in the HCP dataset 3.2 Biopoint Dataset ASD HC All Subjects 75 43 Biological Sex Female 12 20 Male 63 23 Age (years) Mean 9.50 10.92 Std 3.97 3.07 Table 2: Statistics of ASD and HC subjects in the Biopoint dataset Figure 1: Example of biological and scrambled motion tasks in the Biopoint dataset Figure 2: Model architectures. (a) General SimSiam framework. (b) BrainSimSiam encoder f . The Biopoint dataset [ 18 ] comprises task-based fMRI acquired in an ASD study using 12 point-light display videos as task stimuli. Each 24-second video falls into one of two categories: biological motion depicting socially meaningful human actions and scrambled motion with random point trajectories (Fig. 1 ). Videos from two tasks