Flatten Wisely: How Patch Order Shapes Mamba-Powered Vision for MRI Segmentation

Osama Hardan ∗, Omar Elshenhabi ∗, Tamer Khattab, Mohamed Mabrok · 2025

Read Flatten Wisely: How Patch Order Shapes Mamba-Powered Vision for MRI Segmentation by Osama Hardan ∗, Omar Elshenhabi ∗, Tamer Khattab, Mohamed Mabrok (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-13384.txt.

Introduction

Osama Hardan and co-authors show that the order in which an image is fed to a Mamba vision model can swing its medical scan results. Their 2025 paper studies models that read images as one long sequence of patches, gaining speed but forcing a choice of scan path. For brain MRI, rich in anatomical pattern, that choice should matter, yet no one had tested it at scale.

The team built a module that tries many scan paths at no extra cost, then benchmarked twenty-one strategies over seventy thousand slices from three public data sets. Scan order proved a real factor, with accuracy swinging widely between paths. Their result turns a hidden default into a design lever for medical imaging.

Catalog record

CREATOROsama Hardan ∗, Omar Elshenhabi ∗, Tamer Khattab, Mohamed Mabrok
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-13384

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Flatten Wisely: How Patch Order Shapes Mamba-Powered Vision for MRI Segmentation Thanks: This work was supported by Qatar Research, Development, and Innovation (QRDI) Council under UREP Grant No. UREP31-077-2-025. Thanks: ∗ Equal contribution Osama Hardan ∗, Omar Elshenhabi ∗, Tamer Khattab, Mohamed Mabrok Abstract Vision Mamba models promise transformer-level performance at linear computational cost, but their reliance on serializing 2D images into 1D sequences introduces a critical, yet overlooked, design choice: the patch scan order. In medical imaging, where modalities like brain MRI contain strong anatomical priors, this choice is non-trivial. This paper presents the first systematic study of how scan order impacts MRI segmentation. We introduce Multi-Scan 2D (MS2D) , a parameter-free module for Mamba-based architectures that facilitates exploring diverse scan paths without additional computational cost. We conduct a large-scale benchmark of 21 scan strategies on three public datasets (BraTS 2020, ISLES 2022, LGG), covering over 70,000 slices. Our analysis shows conclusively that scan order is a statistically significant factor (Friedman test: χ 20 2 = 43.9 , p = 0.0016 \chi^{2}_{20}=43.9,p=0.0016 ), with performance varying by as much as 27 Dice points. Spatially contiguous paths—simple horizontal and vertical rasters—consistently outperform disjointed diagonal scans. We conclude that scan order is a powerful, cost-free hyperparameter, and provide an evidence-based shortlist of optimal paths to maximize the performance of Mamba models in medical imaging. I Introduction Magnetic resonance imaging (MRI) is the primary modality for visualising soft-tissue pathology in the brain. Automated slice-level segmentation of tumours, oedema and ischaemic stroke lesions accelerates clinical workflows, supports longitudinal outcome studies and reduces the variability that plagues manual delineation. Convolutional neural networks (CNNs) such as U-Net have driven the last decade of progress by learning local texture cues efficiently [ 1 ] . Despite continual refinements in depth, width and receptive-field manipulation, CNNs still extract context through cascades of limited-scope filters; the effective receptive field covers only a small fraction of the theoretical area [ 2 ] , which hampers modelling of long-range dependencies critical for accurate neuro-anatomical segmentation. Transformer-based architectures extend the receptive field to the entire image by applying self-attention across patch tokens. Although this mechanism has produced impressive gains in natural and medical vision, its memory footprint scales quadratically with the number of patches. For high-resolution brain MRI or multi-volume studies, quadratic scaling translates into prohibitive cost during training and inference, constraining adoption in clinical environments where GPU capacity and latency budgets are tight [ 3 , 4 ] . State-space sequence models (SSMs) offer a compelling alternative. By framing sequence processing as a discretised linear time-invariant system, SSMs enable convolutional implementations that run in linear time and constant memory with respect to sequence length [ 5 ] . Mamba augments this framework with an input-dependent selective gate and a parallel-scan kernel that preserve global context while lowering overhead [ 6 ] . Vision-adapted variants—including Vision Mamba and VMamba—embed Mamba blocks in hierarchical backbones, serialising the two-dimensional image into a one-dimensional patch sequence before processing [ 7 , 8 ] . Serialisation introduces a design variable in state-space vision models: the scan order . Each scan defines how an H\times W grid of patches is traversed and therefore decides which spatial neighbours appear adjacent in the sequence. Adjacency in sequence space influences how easily the model can rebuild local structure; poorly chosen orders may force the network to expend capacity restoring spatial coherence rather than modelling semantics. Existing Vision Mamba studies adopt fixed heuristics—usually four orthogonal directions or a bidirectional raster inherited from natural-image practice [ 7 , 8 , 9 ] . Zhu et al. systematically ablated twenty-two orders on aerial imagery and concluded that a single left-to-right scan sufficed for land-cover segmentation [ 10 ] . Whether this conclusion generalises to brain MRI is unknown. Neuro-anatomical data possess strong directional regularities: white-matter tracts align along preferred axes, and cortical gyri and vascular territories introduce patterned anisotropy [ 11 , 12 ] . These priors suggest that patch order may influence performance more strongly in brain MRI than in the less structured layout of remote-sensing scenes. A non-parametric Friedman test across LGG, BraTS 2020 and ISLES 2022 rejects the null hypothesis that all scan orders perform equally ( \chi^{2}_{20}=43.9,\;p=0.0016 ), confirming that scan direction matters for brain-MRI segmentation. To investigate this effect we extend VMamba with Multi-Scan-2D (MS2D), a module that offers twelve primitive scan paths instead of the default four while preserving parameter count and runtime. MS2D feeds four parallel Mamba streams inside each residual unit; users select one, two or four unique directions, and the module duplicates them as needed so that exactly four streams are processed, keeping computational load constant. Leveraging MS2D we construct a benchmark of twenty-one scan strategies—twelve unidirectional scans, six bidirectional pairs and three balanced four-direction mixes—and test them on three public brain-MRI datasets: LGG for low-grade glioma, BraTS 2020 for heterogeneous high-grade tumours and ISLES 2022 for acute ischaemic stroke. Together these datasets contain more than seventy thousand slices. Our contributions are threefold. 1. Architecture : We introduce MS2D, a plug-in for VMamba that expands the scan library to twelve directions without adding parameters or latency. 2. Benchmark : We release a systematic twenty-one-experiment matrix evaluated on LGG, BraTS 2020 and ISLES 2022, enabling reproducible study of scan-order effects in brain-MRI segmentation. 3. Guidance : Across all datasets, contiguous horizontal scans and one balanced four-direction order consistently achieve the highest Dice ranks, while the Friedman analysis confirms the statistical significance of scan choice. These simple orders deliver near-optimal accuracy with no compute penalty, giving practitioners an evidence-based shortlist of scans to try first when deploying Vision Mamba in clinical pipelines. In short, our study establishes scan order as a consequential yet cost-free hyper-parameter for state-space vision models in neuro-imaging and provides both the tools and empirical evidence needed to exploit it. II Related Work II-A State–Space Sequence Models State-Space Models (SSMs) are a cornerstone of control theory, providing a framework for modeling systems that evolve over time. A continuous-time linear SSM describes how an input signal x(t) is mapped to an output signal y(t) via a latent state vector h(t)\in\mathbb{R}^{N} : \dot{h}(t)=\mathbf{A}h(t)+\mathbf{B}x(t),\qquad y(t)=\mathbf{C}h(t)+\mathbf{D}x(t). (1) The components of this system are: • \mathbf{A}\in\mathbb{R}^{N\times N} : The state matrix , which governs the internal dynamics of the system. • \mathbf{B}\in\mathbb{R}^{N\times D_{in}} : The input matrix , which maps the input x(t)\in\mathbb{R}^{D_{in}} to the state. • \mathbf{C}\in\mathbb{R}^{D_{out}\times N} : The output matrix , which maps the state to the output y(t)\in\mathbb{R}^{D_{out}} . • \mathbf{D}\in\mathbb{R}^{D_{out}\times D_{in}} : The feedthrough matrix , providing a direct skip connection from the input to the output. For use with discrete sequential data, such as a sequence of image patches, the continuous system must be discretized. Using a fixed sampling step size \Delta , a common method like the zero-order hold transforms the continuous parameters (\mathbf{A},\mathbf{B}) into discrete counterparts (\bar{\mathbf{A}},\bar{\mathbf{B}}) : h_{k}=\bar{\mathbf{A}}h_{k-1}+\bar{\mathbf{B}}x_{k},\qquad y_{k}=\mathbf{C}h_{k}+\mathbf{D}x_{k}, (2) where \bar{\mathbf{A}}=e^{\mathbf{A}\Delta} and \bar{\mathbf{B}}=(\mathbf{A}\Delta)^{-1}(e^{\mathbf{A}\Delta}-\mathbf{I})\mathbf{B}\Delta . The model can then be computed either recurrently as in Eq. 2 or globally as a single long convolution. The high computational cost of these operations historically limited their use in deep learning. The Structured State Space Sequence Model (S4) provided a breakthrough by constraining \mathbf{A} to a specific diagonal-plus-low-rank structure, which enabled

Read the complete text