Benchmarking the Reproducibility of Brain MRI Segmentation Across Scanners and Time

\firstname Ekaterina \surname Kondrateva \firstname Sandzhi \surname Barg \firstname Mikhail \surname Vasiliev · 2025

Read Benchmarking the Reproducibility of Brain MRI Segmentation Across Scanners and Time by \firstname Ekaterina \surname Kondrateva \firstname Sandzhi \surname Barg \firstname Mikhail \surname Vasiliev (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-15931.txt.

Introduction

Ekaterina Kondrateva, Sandzhi Barg, and Mikhail Vasiliev benchmark how reproducibly two brain MRI segmentation tools perform across scanners and time. Their 2025 study uses FastSurfer and SynthSeg, both housed in the widely used FreeSurfer suite. Accurate brain measurement matters for tracking disease, but scanner differences and repeat-scan noise can swamp the subtle changes clinicians hope to detect.

Two data sets anchor the test: a seventeen-year longitudinal cohort and a nine-site test-retest cohort. Four agreement measures were computed for each brain region. Small deep structures such as the amygdala varied by seven to eight percent between scans even under controlled conditions. That noise rivals the five-to-ten-percent changes longitudinal studies chase. The authors propose surface-based quality filtering to firm up the measurements.

Catalog record

CREATOR\firstname Ekaterina \surname Kondrateva \firstname Sandzhi \surname Barg \firstname Mikhail \surname Vasiliev
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-15931

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Benchmarking the Reproducibility of Brain MRI Segmentation Across Scanners and Time \firstname Ekaterina \surname Kondrateva \firstname Sandzhi \surname Barg \firstname Mikhail \surname Vasiliev Abstract Accurate and reproducible brain morphometry from structural MRI is critical for monitoring neuroanatomical changes across time and imaging domains. Although deep learning has accelerated segmentation workflows, scanner-induced variability and reproducibility limitations remain—particularly in longitudinal and multi-site settings. In this study, we benchmark two state-of-the-art pipelines— FastSurfer and SynthSeg —both integrated into FreeSurfer , one of the most widely adopted tools in neuroimaging. Using two complementary datasets—a 17-year single-subject longitudinal cohort (SIMON) and a 9-site test-retest cohort (SRPBS)—we quantify inter-scan segmentation variability using Dice, Surface Dice, Hausdorff Distance (HD95), and Mean Absolute Percentage Error (MAPE). Our results reveal up to 7–8% volume variation in small subcortical structures such as the amygdala and ventral diencephalon, even under controlled test-retest conditions. This raises a critical question: is it feasible to detect subtle longitudinal changes—on the order of 5–10%—in pea-sized brain regions, given the magnitude of domain-induced morphometric noise? We further analyze the effects of registration choices and interpolation modes, and propose surface-based quality filtering to improve reliability. This work provides a reproducible benchmark and calls for harmonization strategies to enable robust morphometry in real-world neuroimaging studies. Our code is available at https://github.com/kondratevakate/brain-mri-segmentation . \melbaid YYYY:NNN \melbaauthors Kondrateva, Barg and Vasiliev \firstpageno 1337 \melbayear 2025 \datesubmitted yyyy-m1-d1 \datepublished yyyy-m2-d2 \melbaspecialissue Medical Imaging with Deep Learning (MIDL) 2025 \melbaspecialissueeditors Marleen de Bruijne, Tal Arbel, Ismail Ben Ayed, Hervé Lombaert \ShortHeadings Brain MRI Segmentation Across ScannersKondrateva, Barg and Vasiliev \affiliations DOI: 10.59275/j.melba.2024-AAAA Volume: 2 keywords Machine Learning, Brain Morphometry, MRI, Multi-Scanner Variability, Dice, FreeSurfer, SynthSeg, Segmentation, Statistics, Test Retes, Domain Shift 1 Introduction \enluminure Advances in AI-driven medical imaging have revolutionized pathology detection, yet reproducible morphometric analysis of healthy brains—especially across scanners and over time—remains a challenge. This gap limits our ability to monitor individual brain health trajectories and detect early pathological changes. While artificial intelligence (AI) has significantly advanced medical imaging—particularly in pathology segmentation tasks such as tumor identification in the BraTS challenge—there remains a notable gap in applying these advancements to morphometric analyses of healthy brains across varied domains. This underexplored area presents opportunities for developing robust, generalizable AI models that can accurately capture subtle anatomical variations, thereby deepening insight into brain aging and development. Traditional tools like FreeSurfer ( Fischl, 2012 ) have been instrumental in providing detailed morphometric analyses. Recent integrations, such as SynthSeg ( Billot et al., 2023 ) , offer contrast-agnostic segmentation capabilities trained on synthetic data, aiming to improve generalizability across different imaging protocols. Despite these advancements, challenges persist in ensuring reproducibility of volumetric estimates under real-world conditions, particularly when dealing with data from multiple scanners and protocols. This study aims to assess the consistency of brain volume measurements using FastSurefer and FreeSurfer 8 with integrated SynthSeg across longitudinal MRI scans from a single individual. By quantifying inter-scan variability using metrics like absolute volume difference, Dice, and Surface Dice, we seek to highlight the limitations of current segmentation pipelines in personalized brain health monitoring and early detection of neurodegenerative conditions. 2 Related Works Related Work 2.1 Segmentation Pipelines for Morphometry Extraction Deep learning has significantly advanced individual-level brain morphometry from structural MRI. Traditional pipelines such as FreeSurfer ( Fischl, 2012 ) have long served as a gold standard, producing cortical and subcortical morphometric features (e.g., thickness, volume, surface area). However, these methods are computationally intensive and sensitive to scanner variability, limiting their scalability in large-scale or multisite studies. Recent versions of FreeSurfer integrate SynthSeg ( Billot et al., 2023 ) , a contrast-agnostic segmentation model trained on synthetic data. SynthSeg+ provides robust volumetric estimates across diverse contrasts, resolutions, and scanners. Its compatibility with standard atlases (e.g., Desikan-Killiany, MUSE) makes it suitable for harmonized morphometry across heterogeneous datasets. To address runtime bottlenecks, FastSurferVINN ( Henschel et al., 2023 ) replaces FreeSurfer’s anatomical stream with a vision transformer-based model, enabling accurate surface-based cortical thickness estimation within minutes. Tools such as Brainchop prioritize clinical scalability, though often at the cost of generalization to unseen protocols. Other high-performing segmentation models include nnU-Net ( Isensee et al., 2021 ) and nnFormer ( ( Zhou et al., 2023 ) ), which yield excellent accuracy in controlled benchmarks but often require dataset-specific finetuning to generalize effectively in clinical or real-world settings. Recent segmentation advances also include multi-atlas deep learning pipelines ( Wang et al., 2025 ) , which integrate lifespan-spanning templates to enhance anatomical precision, particularly in pediatric and geriatric cohorts. 2.2 Longitudinal Modeling and Individualized Morphometry Beyond segmentation, recent work has focused on modeling spatiotemporal brain changes at the individual level. Latent diffusion-based progression modeling , such as Brain Latent Progression (BLP) ( Puglisi et al., 2025 ) , uses temporally conditioned diffusion models to infer personalized disease trajectories from serial MRI scans. Learning-based Inference of Brain Change (LIBC) ( Kim et al., 2025 ) models smooth morphometric changes over time using neural timeline embeddings, capturing subtle age- and disease-related progression in cortical and subcortical structures. Normative modeling frameworks ( Allen et al., 2024 ) enable the estimation of z-score deviations from large-scale population references. This approach is particularly effective in identifying early deviations in psychiatric populations and supports both clinical and subclinical applications. Another widely adopted line of work focuses on brain age prediction. BrainAGE ( Franke and Gaser, 2012 ) models estimate biological aging based on MRI-derived morphometric features, frequently using FreeSurfer outputs. These models have demonstrated strong longitudinal reliability and clinical interpretability. Emerging tools like Neurofind ( Vieira et al., 2025 ) offer user-friendly platforms that integrate normative modeling and brain age estimation, providing individualized reports based on high-resolution structural MRI images. Despite these advances, challenges remain in achieving sulcal-level surface precision, quantifying uncertainty, and ensuring reproducibility in real-world multisite studies. Although morphometry has clear clinical applications (e.g., in epilepsy and dementia 1 1 1 https://icometrix.com/expertise#mri ), rigorous longitudinal reproducibility benchmarks remain scarce. 2.3 Brain morpometry as a biomarker Longitudinal MRI studies have greatly expanded our understanding of how brain morphometry changes over time, particularly in response to aging, disease, and stress. A growing body of work highlights structural biomarkers in specific brain regions—especially the hippocampus, anterior cingulate, and prefrontal cortex—that reflect vulnerability or resilience to neuropsychiatric conditions. In healthy populations Papagni et al. (2011) demonstrated gray matter volume (GMV) reductions in the anterior cingulate cortex (ACC), hippocampus, and medial prefrontal cortex (mPFC) in individuals exposed to stress. Similar findings were confirmed in large-scale aging studies, including Schaefer et al. (2018) , who reported consistent hippocampal atrophy associated with aging. MacDonald and Pike (2021) provide a broader review of region-specific atrophy across the lifespan. Structural biomarkers also inform psychiatric research. Cardoner et al. (2024) review evidence of stress-induced degeneration in the ACC and dorsolateral prefrontal cortex (dlPFC), while Carnevali and Sgoifo (2018) identify preserved amygdala volumes as potential resilience markers. UK Biobank analyses further support longitudinal volume reductions in fronto-limbic circuits among individuals with high stress exposure ( Statsenko et al., 2022 ) . Importantly, several studies have examined structural changes within individuals undergoing therapy. Gryglewski et al. Gryglewski et al. (2019) found hippocampal and amygdalar volume increases after electroconvulsive therapy (ECT) in treatment-resistant depression. Furtado et al.

Read the complete text