Arcee: Differentiable Recurrent State Chain for Generative Vision Modeling with Mamba SSMs
Jitesh Chavan Rohit Lal Anand Kamat Mengjia Xu · 2025
Read Arcee: Differentiable Recurrent State Chain for Generative Vision Modeling with Mamba SSMs by Jitesh Chavan Rohit Lal Anand Kamat Mengjia Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-11243.txt.
Introduction
Jitesh Chavan, Rohit Lal, and their colleagues present Arcee, a state-chain design that improves Mamba state-space models for vision tasks. Mamba scans inputs in order with a selective step that keeps a running state, which suits sequences but fits images awkwardly. Standard Mamba restarts each block's state from zero, throwing away what the prior block had built up.
Arcee instead hands each block's final state to the next block as its starting point. The handoff is built as a smooth boundary map, so gradients flow across blocks end to end. The design is parameter-free, costs almost nothing, and works with all prior vision Mamba variants, treating the passed state as a mild directional prior.
Catalog record
| CREATOR | Jitesh Chavan Rohit Lal Anand Kamat Mengjia Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-11243
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Arcee: Differentiable Recurrent State Chain for Generative Vision Modeling with Mamba SSMs Jitesh Chavan Rohit Lal Anand Kamat Mengjia Xu Abstract State-space models (SSMs), Mamba in particular, are increasingly adopted for long-context sequence modeling, providing linear-time aggregation via an input-dependent, causal selective-scan operation. Along this line, recent “Mamba-for-vision” variants largely explore multiple scan orders to relax strict causality for non-sequential signals (e.g., images). Rather than preserving cross-block memory, the conventional formulation of the selective-scan operation in Mamba reinitializes each block’s state-space dynamics from zero, discarding the terminal state-space representation (SSR) from the previous block. Arcee, a cross-block recurrent state chain, reuses each block’s terminal state-space representation as the initial condition for the next block ( h 0 ( l ) = 𝒯 ( l ) ( h T ( l − 1 ) ) ) (h_{0}^{(l)}=\mathcal{T}^{(l)}(h_{T}^{(l-1)})) . Handoff across blocks is constructed as a differentiable boundary map whose Jacobian enables end-to-end gradient flow across terminal boundaries. Key to practicality, Arcee is compatible with all prior “vision-mamba” variants, parameter-free, and incurs constant, negligible cost. As a modeling perspective, we view terminal SSR as a mild directional prior induced by a causal pass over the input, rather than an estimator of the non-sequential signal itself. To quantify the impact, for unconditional generation on CelebA-HQ (256 × \times 256) with Flow Matching, Arcee reduces FID ↓ \downarrow from 82.81 82.81 to 15.33 15.33 ( 5.4 × 5.4\times lower) on a single scan-order Zigzag Mamba baseline. Extensible CUDA kernels and training code are released to support reproducibility and further research at https://github.com/JiteshChavan/rc2 1 Introduction Flow matching and diffusion models have revolutionized generative frameworks for images, videos, protein structures, and many other modalities (Lipman et al . , 2022 21 ; Albergo et al . 2023 1 ; Liu et al . 24 ; Bose et al . , 2024 6 ; Song et al . 32 2020; Karras et al . 19 ). These models generate realistic images, videos or samples from corresponding data distribution by simulating an ordinary or stochastic differential equation (ODE/SDE) with a sample from a simple prior (usually gaussian noise) as the initial value condition for the differential equation, where the vector field (and score function in case of SDEs) that defines the differential equation is approximated by a neural network. Recently, transformer architectures have proliferated as a choice for the neural network, a consequence of their superior scalability 30 ; 3 and effectiveness in multi-modal training 4 . Despite their effectiveness for in-context learning tasks in non sequential modalities, transformers bear a significant computational cost that scales quadratically with input sequence length. While there have been efforts to alleviate the quadratic complexity of the attention mechanism by instrumenting methods such as FlashAttention, FlashAttention 2 7 ; 8 , it still remains the bottleneck for employing transformer-based models 35 . State-Space Models have emerged as competitive architectures for long context sequence modeling, offering linear time information aggregation across input signals via continuous time State-Space transitions 13 ; 16 ; 12 . Recent work improves SSM robustness and efficiency through better initializations 15 , parametrizations 13 , diagonalizations 16 , and recurrence parallelizations 14 . Mamba 11 in particular extends prior works, making SSMs more expressive with input dependent state-space transitions through hardware-aware and work-efficient selective scan, yielding linear scaling in sequence length. While selective scan mechanism introduced in Mamba excels at efficient long sequence modeling, its causal aggregation of information creates friction when adapting Mamba to non sequential modalities, motivating architectures that preserve efficiency while relaxing the inherent strict causality. Prior vision-SSM work typically flattens 2D signals into a token sequence u and applies multiple scan orders within the same block, followed by simple feature fusion, adding parameters for each scan order 25 ; 38 ; 23 ; 27 . A complementary strategy amortizes layerwise heterogeneous scan orders across depth with no per-block parameter increase, as in Zigma 17 , where each causal scan manifold u\mapsto y , captures dependencies between tokens at varying degrees of spatial vicinity across layers. Mamba was originally introduced for autoregressive sequence modeling; consequently, most vision variants inherit a design in which each block’s state–space dynamics are initialized with a zero state instead of retrieving global causal summary encoded within terminal state-space representation h_{T} from previous block to avoid information leakage and preserve causality; This is restrictive for non-sequential signals (e.g., images), because h_{T} summarizes a full pass over the input in a given scan order and thus encodes potentially useful global context that is currently discarded. In this work, we introduce a cross-block R ecurrent State C hain ( Arcee ) that generalizes the conventional causal selective-scan in Mamba by using the terminal state-space representation (SSR) from block l-1 as the initial value condition for SSM dynamics in block l . Any prior Mamba baselines can be adapted to propagate this compact global summary across depth, yielding a plug-and-play mechanism with zero additional parameters and constant, negligible overhead (independent of sequence length). Arcee is orthogonal to scan-order design; whereas standard selective-scan fixes each block’s initial state to 0 , Arcee reuses the previous block’s final SSR to provide a mild, architecture-agnostic inductive bias for generative visions tasks with Mamba SSMs. To summarize, we make following contributions: 1. We identify the legacy zero-init initial value condition for state-space dynamics in conventional selective scan operation in Mamba that discards the terminal state-space representation (SSR) between blocks (see 1 , restrictive for non-sequential signals (images). 2. We hypothesize that the terminal SSR potentially encodes a useful global summary and serves as a mild inductive cue for downstream selective-scan dynamics, despite SSR being a severe compression of the non-sequential signal. 3. We introduce a zero parameter overhead, plug and play solution Arcee by generalizing the selective scan manifold in Mamba 11 from u\mapsto y to (u,h_{0}^{(l)})\mapsto(y,h_{T}^{(l)}) via a differentiable boundary map h_{0}^{(l)}=\mathcal{T}^{(l)}(h_{T}^{(l-1)}) ; its Jacobian \mathcal{J}_{\mathcal{T}}^{(l)}=\partial h_{0}^{(l)}/\partial h_{T}^{(l-1)} is trained end to end so terminal SSRs (h_{T}^{(k)}\forall k\in[0,depth)) rendered by each block align across depth. In our default, \mathcal{T}^{(l)} is identity. 4. Plugging Arcee into naive single-scan order ZigZag-Mamba baseline (Zigma_1) 17 for unconditional generation on CelebA-HQ (256x256) 18 with Flow Matching 21 , we observe a large FID drop from 82.81 to 15.33 (-81.49%, 5.40x lower ) at negligible cost. As scan-order diversity increases (or in case of order-specific weights), the improvements attenuate; we analyze these trends in the experiments. 5. We release a general Arcee-selective-scan CUDA implementation with exposed boundary hooks (h_{0},h_{T}) to enable research on cross-block state handoffs. Figure 1 : Conventional vs. Arcee selective scan. (a) In a vanilla Mamba block, the selective scan is strictly causal: the state is initialized with h^{(k)}(0)=0 , the terminal state h^{(k)}(T) is discarded after producing y , and the next block again starts from zero. Darker cells indicate positions that have accumulated more context (later timesteps have seen a larger prefix of the sequence). (b) Arcee extends the scan to a two-port block: the terminal SSR h^{(k)}(T) is reused as the initial state h^{(k+1)}(0) of the next block via a differentiable boundary map, creating a recurrent state chain across depth with a valid gradient path and no change to the intra-block dynamics. 2 Background and Motivation