Precision Synthesis of Multi-Tracer PET via VLM-Modulated Rectified Flow for Stratifying Mild Cognitive Impairment

Tuo Liu, Shuijin Lin, Shaozhen Yan, Haifeng Wang, Jie Lu, Jianhua Ma, Chunfeng Lian · 2026

Read Precision Synthesis of Multi-Tracer PET via VLM-Modulated Rectified Flow for Stratifying Mild Cognitive Impairment by Tuo Liu, Shuijin Lin, Shaozhen Yan, Haifeng Wang, Jie Lu, Jianhua Ma, Chunfeng Lian (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11176.txt.

Introduction

Liu and six colleagues introduce DIReCT, a model synthesizing several PET tracers from MRI plus clinical information. Alzheimer's testing relies on neuroimaging. PET use is limited by cost and dose, blocking screening. New models can make PET from MRI, but personal precision remains a challenge. The approach joins a 3D architecture linking tracer relationships. A tuned vision model, BiomedCLIP, gives text output. Tests on several datasets show quality and accurate disease patterns. With MRI, made PET allows stratification of cognitive impairment.

Catalog record

CREATORTuo Liu, Shuijin Lin, Shaozhen Yan, Haifeng Wang, Jie Lu, Jianhua Ma, Chunfeng Lian
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45308
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11176

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Precision Synthesis of Multi-Tracer PET via VLM-Modulated Rectified Flow for Stratifying Mild Cognitive Impairment Tuo Liu, Shuijin Lin, Shaozhen Yan, Haifeng Wang, Jie Lu, Jianhua Ma, Chunfeng Lian Abstract The biological definition of Alzheimer’s disease (AD) relies on multi-modal neuroimaging, yet the clinical utility of positron emission tomography (PET) is limited by cost and radiation exposure, hindering early screening at preclinical or prodromal stages. While generative models offer a promising alternative by synthesizing PET from magnetic resonance imaging (MRI), achieving subject-specific precision remains a primary challenge. Here, we introduce DIReCT + ⁣ + ++ , a Domain-Informed ReCTified flow model for synthesizing multi-tracer PET from MRI combined with fundamental clinical information. Our approach integrates a 3D rectified flow architecture to capture complex cross-modal and cross-tracer relationships with a domain-adapted vision-language model (BiomedCLIP) that provides text-guided, personalized generation using clinical scores and imaging knowledge. Extensive evaluations on multi-center datasets demonstrate that DIReCT + ⁣ + ++ not only produces synthetic PET images ( 18 F-AV-45 and 18 F-FDG) of superior fidelity and generalizability but also accurately recapitulates disease-specific patterns. Crucially, combining these synthesized PET images with MRI enables precise personalized stratification of mild cognitive impairment (MCI), advancing a scalable, data-efficient tool for the early diagnosis and prognostic prediction of AD. The source code will be released on https://github.com/ladderlab-xjtu/DIReCT-PLUS . keywords Controllable Cross-Modal Synthesis ,Domain-Knowledge Encoding ,PET Imaging ,Alzheimer’s Disease † † These authors contributed equally to this work. † † corresponding: Corresponding authors 1 Introduction Alzheimer’s disease (AD), the most prevalent neurodegenerative disorder, constitutes a mounting global health crisis. Its pathological processes begin years before clinical dementia manifests, making the prodromal stage of mild cognitive impairment (MCI) a critical window for intervention [ 1 ] . However, the heterogeneous nature of MCI, where only a subset of individuals progress to AD, poses a significant prognostic challenge [ 2 ] . This challenge is underscored by the modern biological definition of AD, which is based on in vivo evidence of core pathologies, specifically amyloid- \beta (A \beta ) deposition (e.g., on 18 F-AV45 PET) and neuronal injury (e.g., 18 F-FDG PET hypometabolism), with structural MRI quantifying associated atrophy [ 3 , 4 , 2 ] . In clinical practice, however, the routine acquisition of PET is hampered by high cost, limited access, and radiation exposure, restricting this multi-modal biomarker profiling to research settings and hindering timely, widespread screening [ 5 ] . Generative artificial intelligence (AI) offers a transformative pathway to bridge this clinical gap by synthesizing PET biomarkers from routinely acquired MRI [ 6 , 7 ] . Initial studies, primarily using generative adversarial networks (GANs) [ 8 , 9 , 10 ] and, more recently, diffusion models [ 11 , 12 , 13 ] , have established the feasibility of producing visually realistic PET images. Subsequent advances have sought to improve fidelity by incorporating domain knowledge, for instance, through guidance from large vision-language models (VLMs) [ 14 , 15 ] . Despite these advances, a critical limitation remains: the subject-specific precision necessary for personalized diagnosis and prognostic stratification is still lacking, which is particularly crucial for a heterogeneous condition like AD [ 3 ] . This challenges stems from the inherent nature of MRI-to-PET synthesis as an ill-posed inverse problem; neurodegenration visible on MRI is typically a downstream effect of prior molecular pathologies captured by PET [ 16 ] . Consequently, individuals with similar structural MRI presentations at early disease stages can exhibit vastly different pathological burdens, leading to ambiguous mapping. Overcoming this ambiguity requires an AI synthesis framework that can comprehensively model complex cross-modal and cross-tracer associations to provide strong, patient-specific guidance. To this end, we propose DIReCT ++ (Domain-Informed ReCTified flow), a VLM-modulated generative model for the high-fidelity synthesis of patient-specific, multi-tracer PET images from baseline MRI and textual guidance. The DIReCT ++ framework is built on two core innovations. First, it employs an efficient 3D rectified flow architecture to directly capture the complex, high-dimensional relationships between MRI and multi-tracer PET (specifically 18 F-AV-45 and 18 F-FDG), enabling concurrent and rapid generation. Second, to achieve subject-specific precision, the model is conditioned via a VLM (BiomedCLIP [ 17 ] ) with application-dedicated adaptations. This domain-adapted VLM integrates fundamental patient knowledge (encoded as patient-specific clinical assessments and tracer-specific imaging domain knowledge) to guide the synthesis process, ensuring the generated PET images reflect individual pathological states rather than population averages. By reconciling an efficient generative process with precise, text-based patient conditioning, our approach directly addresses the ill-posed nature of cross-modal synthesis in heterogeneous disease populations. We rigorously evaluate DIReCT ++ on large-scale, multi-center datasets, including the Alzheimer’s Disease Neuroimaging Initiative (ADNI) [ 18 ] and Open Access Series of Imaging Studies (OASIS) [ 19 ] . Comprehensive experiments demonstrate that our model generates multi-tracer PET images with superior fidelity and generalizability compared to state-of-the-art methods. Crucially, the synthesized images accurately recapitulate known disease-specific patterns of amyloid deposition and hypometabolism. When combined with the original MRI scans to train deep learning classifiers [ 20 , 21 ] , the synthesized PET data enable accurate personalized diagnosis and, most importantly, the stratification of MCI patients based on their risk of progression to AD. Our work establishes a translational paradigm for precision biomarker synthesis, offering a scalable and data-efficient tool for early intervention in dementia. 2 Materials and methods 2.1 DIReCT ++ DIReCT ++ is a VLM-modulated rectified flow (RF) framework for synthesizing 18 F-AV-45 and 18 F-FDG PET images from a T1-weighted MRI scan, conditioned on text prompts derived from patient information and tracer knowledge (Fig. 1 ). The framework integrates a pre-trained VLM, fine-tuned from BiomedCLIP via prompt learning, with a conditional 3D rectified flow model. The VLM encodes text guidance, which is incorporated into the flow model via cross-attention to achieve subject-wise precision. The prompt learning strategy is designed to maximize intra-tracer text-image alignment while preserving inter-tracer distinctions, reflecting the complementary nature of the biomarkers. The following subsections detail the architecture and training. 2.1.1 Domain-Adapted VLM To incorporate domain-specific knowledge into the generation process, we develop an adaptive guidance mechanism that leverages a frozen, pre-trained BiomedCLIP model. Specifically, we learn two affine transformations, Adapt_{f} and Adapt_{a} , to align the textual embeddings of FDG and AV-45 PET modalities with their respective image feature spaces. Let c_{f},c_{a} denote the BiomedCLIP textual embeddings for FDG and AV-45, respectively. For simplicity, we adopt lightweight affine transformations parameterized by a scalar scale and a vector bias: Adapt_{f}(c_{f})=\alpha_{1}\,c_{f}+\alpha_{2},\quad Adapt_{a}(c_{a})=\beta_{1}\,c_{a}+\beta_{2}, where \alpha_{1},\beta_{1} are learnable scalar weights and \alpha_{2},\beta_{2} are learnable bias vectors. We choose scalar scaling rather than full linear projection to minimize trainable parameters and avoid overfitting, given the limited size of medical imaging datasets. The optimal parameters are obtained by solving a constrained optimization problem: \displaystyle\underset{Adapt_{f},\,Adapt_{a}}{\arg\min}\quad\mathcal{L}_{\text{align}}=\big(1-\operatorname{sim}(Adapt_{f}(c_{f}),\mathbb{E}[f_{f}])\big)+\big(1-\operatorname{sim}(Adapt_{a}(c_{a}),\mathbb{E}[f_{a}])\big) (1) \displaystyle\text{subject to}\quad\operatorname{sim}\big(Adapt_{f}(c_{f}),Adapt_{a}(c_{a})\big)\geq\tau, where \operatorname{sim}(\cdot,\cdot) denotes cosine similarity, and \tau\in[0,1] is a predefined threshold. The constraint ensures the cross-modality guidance vectors maintain a certain level of semantic consistency. By solving ( 1 ), we obtain the modality-specific guidance vectors Adapt_{f}(c_{f}) and Adapt_{a}(c_{a}) , which are subsequently used to steer the rectified flow translation process in a domain-informed manner. Figure 1: Overview of the DIReCT ++ framework for multimodal PET synthesis and downstream analysis. (a) Schematic of the rectified flow

Read the complete text