eMamba: Efficient Acceleration Framework for Mamba Models in Edge Computing

Jiyong Kim, Jaeho Lee, Jiahao Lin, Alish Kanani, Miao Sun, Umit Y. Ogras ∗, Jaehyun Park ∗ · 2025

Read eMamba: Efficient Acceleration Framework for Mamba Models in Edge Computing by Jiyong Kim, Jaeho Lee, Jiahao Lin, Alish Kanani, Miao Sun, Umit Y. Ogras ∗, Jaehyun Park ∗ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10370.txt.

Introduction

Seven researchers wrote a computer engineering paper in two thousand twenty-five on running a new kind of AI model on small devices. The model family, called Mamba, handles sequences with accuracy that rivals transformers at lower cost. That thrift suits edge devices with tight power and memory, but no hardware support had been built for it.

Their framework, eMamba, redesigns the costly parts of the model for hardware. Bulky normalization layers are swapped for light ones, and hard operations are approximated with the target use in mind. A search step then tunes the settings used in those approximations. Tests on image tasks show the design keeps accuracy while speeding the model up.

Catalog record

CREATORJiyong Kim, Jaeho Lee, Jiahao Lin, Alish Kanani, Miao Sun, Umit Y. Ogras ∗, Jaehyun Park ∗
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-10370

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

eMamba: Efficient Acceleration Framework for Mamba Models in Edge Computing Journal: TECS Volume: 1 1 1 9 Thanks: ∗ Corresponding authors Jiyong Kim, Jaeho Lee, Jiahao Lin, Alish Kanani, Miao Sun, Umit Y. Ogras ∗, Jaehyun Park ∗ Abstract . State Space Model (SSM)-based machine learning architectures have recently gained significant attention for processing sequential data. Mamba, a recent sequence-to-sequence SSM, offers competitive accuracy with superior computational efficiency compared to state-of-the-art transformer models. While this advantage makes Mamba particularly promising for resource-constrained edge devices, no hardware acceleration frameworks are currently optimized for deploying it in such environments. This paper presents eMamba, a comprehensive end-to-end hardware acceleration framework explicitly designed for deploying Mamba models on edge platforms. eMamba maximizes computational efficiency by replacing complex normalization layers with lightweight hardware-aware alternatives and approximating expensive operations, such as SiLU activation and exponentiation, considering the target applications. Then, it performs an approximation-aware neural architecture search (NAS) to tune the learnable parameters used during approximation. Evaluations with Fashion-MNIST, CIFAR-10, and MARS, an open-source human pose estimation dataset, show eMamba achieves comparable accuracy to state-of-the-art techniques using 1.63–19.9 × \times fewer parameters. In addition, it generalizes well to large-scale natural language tasks, demonstrating stable perplexity across varying sequence lengths on the WikiText2 dataset. We also quantize and implement the entire eMamba pipeline on an AMD ZCU102 FPGA and ASIC using GlobalFoundries (GF) 22 nm technology. Experimental results show 4.95–5.62 × \times lower latency and 2.22–9.95 × \times higher throughput, with 4.77 × \times smaller area, 9.84 × \times lower power, and 48.6 × \times lower energy consumption than baseline solutions while maintaining competitive accuracy. 1. Introduction Deep learning (DL) models are progressing at an unprecedented pace, driving breakthroughs across a wide range of domains. However, the growing complexity of recent models, particularly transformer-based architectures such as Vision Transformers (ViTs), comes at a steep cost ( Xia et al. 2024 ; Shekhar et al. 2024 ) . They demand substantial computational and storage resources, which require powerful GPUs that consume hundreds of watts. This results in massive energy consumption and a carbon footprint that has started impacting the power grid ( De Vries 2023 ; Van Wynsberghe 2021 ) . Moreover, these requirements render complex DL models impractical for edge processing, where inference can be performed with orders of magnitude lower energy footprint while preserving privacy. Hence, there is a strong need for lightweight, energy-efficient DL models for resource-constrained edge processing . One of the key limitations of transformers is their quadratic complexity during inference, driven by costly key-value calculations. State-space models (SSMs) have recently been proposed to address this problem and enable more efficient DL. SSMs are a classical mathematical framework for describing dynamic systems using first-order differential equations, with applications in control systems, signal processing, and electrical circuit design ( Bishop 2011 ; Alexander et al. 2007 ) . They have recently gained attention in machine learning (ML) for their remarkable effectiveness in modeling sequential data ( Gu et al. 2022 ; Gu et al. 2023 ) . Modern SSM-based ML architectures have demonstrated promising results, offering faster training and efficient inference, especially for long sequences ( Gu and Dao 2024 ; Wang et al. 2024 ) . Mamba is a recent SSM-based architecture that achieves competitive performance with transformers with linear time complexity , a substantial improvement over the quadratic complexity of attention mechanisms in transformers ( Gu and Dao 2024 ) . Moreover, prior work established its equivalence to Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), proving its versatility and applicability across various ML tasks ( Gu et al. 2021 ) . Figure 1 . Overview of Mamba architecture. The highlighted blocks are described in detail. The Mamba begins with layer normalization to ensure consistent feature scaling and stabilize training, as illustrated in Figure 1 . Then, the data is processed through two parallel linear layers that expand the feature dimension. The upper path with a SiLU activation serves as a gating mechanism, controlling the output of SSM via element-wise multiplications. A 1D convolution is applied in the lower path to mix input features, preparing them for temporal modeling. Then, the SSM layer, Mamba’s key feature, models the system dynamics and sequence dependencies using a discretized SSM, offering an efficient alternative to attention mechanisms. Finally, a linear projection reverts the expanded features back to the original dimension, and a residual connection is added to preserve input information and further stabilize training. Since the key advantage of SSM and Mamba models is providing competitive performance with lower complexity, this paper focuses on their resource-constrained implementation for edge AI applications . We present a novel end-to-end framework to design efficient Mamba (eMamba) hardware accelerators. While substantial research has targeted accelerating mainstream ML models like deep neural networks and transformers ( Chen et al. 2016 ; Samajdar et al. 2018 ; You et al. 2023 ) , Mamba is still relatively underexplored, despite its promising potential. Only two recent studies present reconfigurable accelerator architectures for Mamba ( Li et al. 2024a ; Wei et al. 2025a ) . However, both target large language models (LLM), as discussed in the related work. No comprehensive frameworks exist for accelerating Mamba models, specifically in edge scenarios, where their linear time complexity could be fully leveraged. To address this gap, we present eMamba, an end-to-end optimization framework designed to accelerate Mamba models for edge applications. We demonstrate its effectiveness on both an FPGA platform and using GlobalFoundries (GF) 22nm technology, highlighting its suitability for real-world, energy-efficient deployment. eMamba introduces a suite of hardware-friendly optimizations to tackle the unique challenges of accelerating Mamba models, including managing recurrence in SSMs, handling non-linear operations, and minimizing latency through computation unrolling. It replaces computationally expensive layer normalization and SiLU with efficient alternatives like range normalization and piecewise linear approximations, achieving substantial gains in efficiency with minimal impact on accuracy. All weights and activations are quantized to minimize computation and memory footprint further. In addition, eMamba uses approximation-aware training and architecture search to find cost-effective model configurations that balance accuracy and resource efficiency. It also applies custom unrolling within the pipeline to optimize the latency–resource usage trade-off. We evaluate eMamba on three representative vision datasets, Fashion-MNIST ( Xiao et al. 2017 ) , CIFAR-10 ( Krizhevsky et al. 2009 ) , and MARS ( An and Ogras 2021 ) , and compare it against recent ViT and CNN models. Across these tasks, we achieve comparable accuracy to the baseline ViT models ( Rodriguez and Krishnan 2023 ; Zhu et al. 2023 ; Dosovitskiy et al. 2021 ) while reducing the model size by 19.9 \times , 2.02 \times , and 1.63 \times for Fashion-MNIST, CIFAR-10, and MARS, respectively. In addition to small-scale vision tasks, eMamba demonstrates strong generalization to complex language tasks, as reflected by its stable performance across varying sequence lengths on the WikiText2 ( Merity et al. 2016 ) dataset, outperforming RNN ( Mikolov et al. 2010 ) , LSTM ( Sak et al. 2014 ) , and ViT models of comparable size. We also evaluated the performance and area using AMD Vivado on the ZCU102 FPGA board and Synopsys Design Compiler & IC Compiler II with GF 22nm technology. Hardware

Read the complete text