GEM3D-CIM: Ge neral Purpose M atrix Computation Using 3D -Integrated SRAM–eDRAM Hybrid C ompute- I n- M emory-on-Memory Architecture

Subhradip Chakraborty, Ankur Singh, and Akhilesh R. Jaiswal · 2026

Read GEM3D-CIM: Ge neral Purpose M atrix Computation Using 3D -Integrated SRAM–eDRAM Hybrid C ompute- I n- M emory-on-Memory Architecture by Subhradip Chakraborty, Ankur Singh, and Akhilesh R. Jaiswal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13969.txt.

Introduction

Chakraborty, Singh, and Jaiswal build a 3D memory-on-memory compute-in-memory architecture that runs general matrix operations, not just dot products. SRAM-eDRAM hybrid crossbars in 22-nanometer technology handle transpose, element-wise addition, and multiplication at 4-bit precision. A transpose-based design plus peripheral-aware layout balances latency, energy, and compute density while staying compatible with standard dot-product CIM.

Catalog record

CREATORSubhradip Chakraborty, Ankur Singh, and Akhilesh R. Jaiswal
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47118
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13969

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GEM3D-CIM: Ge neral Purpose M atrix Computation Using 3D -Integrated SRAM–eDRAM Hybrid C ompute- I n- M emory-on-Memory Architecture Subhradip Chakraborty, Ankur Singh, and Akhilesh R. Jaiswal Abstract With the rapid growth of deep neural networks (DNNs), compute-in-memory (CIM) has emerged as a promising energy-efficient paradigm for accelerating multiply-and-accumulate (MAC) operations. Yet, current CIM architectures are largely limited to dot-product computations and struggle to efficiently support general-purpose matrix operations, such as transpose, element-wise addition, and multiplication. This work presents a 3D-integrated, memory-on-memory SRAM-eDRAM hybrid CIM architecture, implemented in GlobalFoundries 22 nm FDSOI technology, capable of performing general matrix operations directly within the memory crossbar with 4-bit precision. By leveraging a specialized transpose-based architecture, in-memory arithmetic operations, peripheral-aware design, and 3D SRAM–eDRAM integration, the proposed architecture balances latency, energy efficiency, and compute density for general purpose matrix operations while remaining compatible with the conventional CIM dot product architectures. Overall, this memory-on-memory CIM framework generalizes CIM beyond dot products, enabling versatile matrix processing and paving the way for broader applications in AI acceleration and general-purpose high performance computing. I Introduction The recent advances in deep learning and neural networks for critical AI applications have led to an increasing demand for data-intensive computations [ 31 ] . Modern systems suffer from frequent data transfer between processing cores and memory severely limiting both throughput and energy efficiency [ 42 ] . While transistor scaling predicted by Moore’s law has slowed, the number of processing cores has still steadily increased [ 24 ] . In contrast, memory architectures, bandwidth, and energy efficiency have not scaled proportionally, giving rise to the well-known memory wall [ 38 ] . To overcome this challenge, a wide range of compute-in-memory (CIM) architectures have been proposed. These architectures primarily target dot-product operations by enabling computation within the memory bit-cells themselves, thereby reducing data movement and improving both energy efficiency and latency in range of workloads including AI [ 30 ] . CIM has been explored using various memory technologies, with designs primarily dominated by SRAM, DRAM, and non-volatile memories (NVMs). Prior NVM-based CIM implementations have demonstrated very high compute density and throughput [ 39 ] ; however, they often face limitations such as low signal-to-noise ratio (SNR), reduced ON/OFF conductance ratio, high write energy, and fabrication challenges for large-scale integration [ 4 ] . SRAM- [ 15 ] and DRAM- [ 21 ] based analog CIM architectures are generally classified into two categories: charge-domain and current-domain CIM. Charge-domain CIM offers better linearity and higher compute density and energy efficiency but suffers from variations and reduced SNR [ 28 ] due to the presence of analog to digital converter (ADC). Current-domain CIM provides improved energy efficiency, compute density, and SNR, but suffers from non-linearity and ADC quantization [ 14 ] . Despite the massive parallelism inherent in CIM architectures, performance is ultimately constrained by dot-product reduction . Specifically, CIM achieves single cycle parallel compute capability only when all rows are activated simultaneously, with each column performing multiply-and-accumulate (MAC) operations whose results must be reduced by ADCs. These characteristics make CIM particularly well-suited for deep learning applications, where MAC operations dominate and high precision is not a critical requirement [ 11 , 19 ] . While CIM architectures have shown promise in accelerating dot-product operations for inference application in deep learning [ 5 , 25 , 2 ] , their applicability to other scientific computing and general-purpose applications remains limited. Domains such as physics-based computation (e.g., Sod shock tube [ 33 ] ), recurrent neural networks including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU) [ 7 ] , masking operations, and tensor-based algorithms [ 6 ] often rely not only on dot products but also on general matrix element-wise operations, such as matrix transpose, element-wise multiplication, and addition. In these applications, both parallelism and precision play a critical role. Conventional CIM accelerators, being optimized primarily for dot products, lose their inherent parallelism advantage when extended to element-wise operations. For example, transpose operations have been demonstrated in CIM frameworks [ 17 , 9 ] ; however, these implementations are typically optimized only for backpropagation, where the dot-product readout enables transpose functionality at the cost of reduced precision. As a result, transpose operations cannot be efficiently stored or represented within the memory crossbar. Similarly, element-wise multiplication, also known as the Hadamard product, has only recently gained attention, with prior works focusing primarily on algorithmic optimization and peripheral circuit design rather than improving the underlying crossbar architecture [ 7 , 10 ] . Matrix addition in CIM has been investigated largely in the context of digital CIM using adder trees [ 1 ] , but equivalent exploration for elementwise operation in analog CIM frameworks remains limited. In recent years, advancements in packaging technology and semiconductor manufacturing have enabled the development of monolithic 3D integrated CIM architectures [ 22 , 8 , 40 , 35 ] . These architectures significantly enhance compute density and open the door for new design possibilities in CIM applications. In this work, we leverage the monolithic 3D-integrated framework to design CIM cores capable of performing operations beyond conventional dot products , including matrix transpose, element-wise matrix multiplication, and matrix addition as shown in Fig. 1 . The main contributions of this paper are summarized as follows: Fig. 1: The proposed overall architecture comprises two Layers: Layer A, which implements the SRAM array, and Layer B, which implements the eDRAM array. The architecture is organized by monolithic 3D integrating [ 26 ] the two Layers with each one of them divided into subarrays capable of performing matrix transposition, element-wise multiplication, and accumulation operations. • This work leverages advanced packaging to 3D-integrate SRAM and eDRAM layers, enabling a novel compute-in-memory-on-memory architecture capable of supporting general matrix operations beyond traditional dot-product–centric CIM designs. • Specialized on-memory architecture blocks are introduced to efficiently perform matrix transpose operations within the memory fabric, eliminating costly data movement. • The design further supports element-wise matrix multiplication and accumulation directly inside the memory crossbar, with all core computations executed in the analog domain. • Despite its analog compute nature, both input matrices and computed outputs are represented and stored in fully digital form within the memory crossbar. This ensures seamless integration with conventional digital systems and accelerators. • The architecture repurposes 8T SRAM cells as DACs and employs eDRAM-based structures as lightweight ADCs, minimizing peripheral overhead without sacrificing precision. The remainder of this paper is organized as follows. Section II introduces the proposed memory bit-cell designs. Section III describes the matrix transpose operation, while Section IV presents the element-wise matrix multiplication and addition operations. Section V discusses the conventional MAC operation. Section VI provides the results obtained using the GlobalFoundries 22 nm PDK, and Section VII concludes the paper. Fig. 2: Transpose based: (a) T-SRAM bit-cell; (b) T-eDRAM bit-cell; Multiplication and Addition based: (c) MA-SRAM bit-cell; (d) MA-eDRAM bit-cell. II Proposed Memory Bit-cells The proposed 3D-integrated CIM architecture for general-purpose matrix operations is realized using two vertically stacked device Layers (Layer A and Layer B) formed by a monolithic 3D fabrication flow, with fine-pitch monolithic inter-Layer vias providing high-density vertical electrical connections between Layers [ 27 , 12 , 32 ] . Layer A contains SRAM arrays

Read the complete text