ReMAP-PET: Beyond Visual Understanding - Learning Re gion-Guided M etabolic A lignment Semantics from Brain P ET
Dasen Dai, Yanteng Zhang, Shuoqi Li, Yuxiang Wei, Hongjie Yu, Qingxin Zhang, Qizhen Lan, Jagath C. Rajapakse, Vince D. Calhoun · 2026
Read ReMAP-PET: Beyond Visual Understanding - Learning Re gion-Guided M etabolic A lignment Semantics from Brain P ET by Dasen Dai, Yanteng Zhang, Shuoqi Li, Yuxiang Wei, Hongjie Yu, Qingxin Zhang, Qizhen Lan, Jagath C. Rajapakse, Vince D. Calhoun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29577.txt.
Introduction
Dai and eight co-authors build ReMAP-PET, a framework that teaches brain PET models the meaning behind their images. Positron Emission Tomography, or PET, shows how active each brain region is and guides disease checks. Existing three-dimensional brain foundation models treat PET scans as generic volumetric data. That misses the regional patterns of activity that make PET distinctive.
ReMAP-PET supervises a MedicalNet 3D ResNet-50 encoder with regional SUVR profiles, which summarize activity region by region. Two training signals combined push the encoder to learn what the scans mean. On more than a thousand paired samples, the model predicts SUVR values closely and retrieves matching profiles far better than five frozen baselines. Alignment with BioClinicalBERT links the embeddings to medical language, enabling report writing, and simple probes confirm they retain useful medical information.
Catalog record
| CREATOR | Dasen Dai, Yanteng Zhang, Shuoqi Li, Yuxiang Wei, Hongjie Yu, Qingxin Zhang, Qizhen Lan, Jagath C. Rajapakse, Vince D. Calhoun |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.29577
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ReMAP-PET: Beyond Visual Understanding - Learning Re gion-Guided M etabolic A lignment Semantics from Brain P ET Dasen Dai, Yanteng Zhang, Shuoqi Li, Yuxiang Wei, Hongjie Yu, Qingxin Zhang, Qizhen Lan, Jagath C. Rajapakse, Vince D. Calhoun Abstract Positron Emission Tomography (PET) reveals brain metabolism and is clinically central to neurodegenerative disease assessment, yet existing 3D brain foundation models treat PET as generic volumetric data, missing the structured regional metabolic information that distinguishes it from structural neuroimaging. To address these limitations, we propose ReMAP-PET, a framework that moves beyond visual encoding by supervising a partially-tuned MedicalNet 3D ResNet-50 with brain regional standardized uptake value ratio (SUVR) profiles through joint regression and contrastive objectives, enabling the encoder to learn the metabolic semantics underlying PET modality. On 1015 paired PET–SUVR samples, ReMAP-PET achieves 0.070 SUVR MAE and 77.8% PET → \rightarrow SUVR Recall@1, substantially outperforming five frozen pretrained baselines. We further connect the metabolic embedding to clinical language via contrastive alignment with frozen BioClinicalBERT and demonstrate end-to-end PET-to-report generation through SUVR-constrained verbalization. Linear probing on diagnostic classification and cognitive regression tasks confirms that the embeddings retain clinically relevant information without task-specific fine-tuning. Our results show that grounding PET encoders in regional metabolic semantics—rather than treating PET as generic volumetric data—yields representations that are structured, interpretable, and language-compatible, pointing to a new direction for metabolic-aware PET understanding. 1 Introduction Accurately modeling brain metabolic patterns is a core challenge in functional neuroimage analysis and neurodegenerative disease research, with important implications for understanding disease progression and supporting clinical diagnosis 19 . As one of the clinical gold-standard functional modalities for dementia diagnosis, fluorodeoxyglucose (FDG)-PET reflects neuronal activity through glucose metabolism. Unlike structural MRI, which captures anatomical changes, FDG-PET reveals early functional abnormalities and cross-regional metabolic degeneration patterns, and has been widely shown to be closely associated with early screening, disease progression, and cognitive decline in Alzheimer’s disease (AD) 27 . More importantly, PET possesses explicit quantitative medical properties: standardized uptake value ratios (SUVR) measured across brain regions provide physiologically meaningful descriptions of regional metabolism 23 . For example, AD related hypometabolism typically first appears in the posterior cingulate cortex, precuneus, and temporoparietal regions. Compared with raw voxel intensities, these regional metabolic measurements exhibit substantially stronger structural organization and clinical interpretability 2 ; 11 . Learning clinically interpretable metabolic representations from PET therefore matters directly for downstream cognitive disease analysis. Figure 1: Existing methods treat PET as generic 3D volumes, extracting visual features that lack metabolic semantics (left). ReMAP-PET introduces region-level metabolic supervision to teach the encoder the physiological meaning behind PET imaging (right), producing representations that encode both visual and metabolic structure and enable cross-modal retrieval, clinical interpretation, and report generation. In recent years, the development of pretrained 3D medical image encoders has made brain representation learning increasingly feasible, enabling effective transfer to downstream neuroimaging tasks under limited annotation settings 29 ; 10 . However, nearly all existing frameworks are still built around structural MRI, with objectives primarily focused on visual texture modeling, global image representation learning, or anatomical structure reconstruction, while lacking explicit modeling of brain metabolic patterns themselves 33 ; 9 . As a result, when directly applied to PET scans, these methods still tend to treat PET as an ordinary 3D visual image without leveraging the metabolic information that PET inherently provides. This limits the model’s ability to capture the spatial organization of brain metabolism and to align representations with the structured clinical signals available in PET. A natural question is whether pretrained encoders can retain their general visual capability while also learning to encode the regional metabolic structure specific to PET. Rather than relying solely on image-level supervision, we aim to map PET into an interpretable metabolic space constrained by region-level measurements, so that the resulting representations capture neurodegenerative functional patterns and can support PET retrieval, automated report generation, and vision-language model (VLM) integration. To address these challenges, we propose ReMAP-PET ( Re gion-guided M etabolic A lignment with P artial-tuned PET Encoders), a method that grounds PET representation learning in regional metabolic activity. As illustrated in Figure 1 , ReMAP-PET uses 120-region SUVR profiles as structured supervision to reshape the embedding space of a pretrained MedicalNet 3D ResNet-50 4 , updating only the final residual stage ( layer4 ) while preserving generic anatomical knowledge in the earlier layers. A joint objective combining SUVR regression and bidirectional PET–SUVR contrastive alignment produces embeddings that are both numerically predictive and semantically structured. We further connect this metabolic representation to clinical language: lightweight projection heads align the frozen PET embedding with region-text summaries encoded by frozen BioClinicalBERT 1 , and an end-to-end report generation pipeline verbalizes the predicted SUVR profile into clinical-style metabolic summaries whose factual content is fully constrained by the underlying measurements. Our experiments show three main results. First, regression alone produces accurate SUVR predictions but a degenerate embedding space; the contrastive term is what creates the structure needed for retrieval and downstream transfer. Second, partial tuning is architecture-dependent: the same last-block recipe that produces large gains on MedicalNet’s ResNet yields negligible or negative effects on ViT-based and U-Net-based encoders, suggesting that ResNet’s feature hierarchy provides a uniquely suitable bottleneck for metabolic adaptation. Third, linear probing on seven clinical tasks confirms that the metabolic embeddings carry clinically relevant information without task-specific fine-tuning, and the gains in metabolic alignment do not come at the expense of clinical signal. Our contributions are as follows: • We propose a metabolic-aware representation learning paradigm for FDG-PET: instead of treating PET as generic volumetric data, we use 120-region SUVR profiles as structured metabolic supervision with joint regression and contrastive objectives, enabling the encoder to learn regional metabolic semantics beyond visual features. • We show that partial tuning of the final ResNet stage is uniquely effective for PET metabolic adaptation—the same recipe does not transfer to ViT or U-Net architectures—providing an empirical finding about when and why partial tuning works. • We connect the learned metabolic embedding to clinical language through PET–text contrastive alignment and SUVR-constrained report generation, completing an image-to-representation-to-language pipeline for FDG-PET. 2 Related Work 3D brain foundation models. Pretrained 3D medical and brain encoders have become common in neuroimaging representation learning. MedicalNet provides a general 3D transfer-learning backbone, while BrainIAC ( 22 ) , BrainFM ( 17 ) , AnatCL ( 3 ) , and BrainSegFounder ( 6 ) explore different forms of brain-specific pretraining. Other volumetric backbones such as SAM-Med3D ( 24 ) and SwinUNETR ( 12 ) are also widely used, with recent surveys summarizing this line of work. However, these models are largely designed for structural MRI, segmentation, or generic 3D representation learning, and do not explicitly model the regional metabolic patterns that are central to FDG-PET. Medical vision-language alignment. Image–text contrastive learning has been widely used to align visual features with language ( 20 ) . Medical variants such as ConVIRT ( 32 ) , MedCLIP ( 25 ) , PubMedCLIP ( 8 ) , and BiomedCLIP ( 31 ) extend this idea to reports, captions, or biomedical text. Most of them focus on 2D modalities and use report- or label-level supervision. In the neuroimaging domain, recent efforts explore ROI-level multimodal alignment and cross-modality translation using structured regional features ( 26