Energy-Efficient CNN Acceleration with MSDF Digit-Serial Arithmetic on FPGA
Muhammad Usman, Yousef Sadegheih, Dorit Merhof · 2026
Read Energy-Efficient CNN Acceleration with MSDF Digit-Serial Arithmetic on FPGA by Muhammad Usman, Yousef Sadegheih, Dorit Merhof (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25562.txt.
Introduction
Medical image segmentation networks are heavy arithmetic, and edge hardware must run them on a tight energy budget. Muhammad Usman, Yousef Sadegheih, and Dorit Merhof accelerate the U-Net architecture on a programmable chip in 2026. They use digit serial arithmetic, which processes numbers one digit at a time starting from the most significant. That style is compact but each operation stalls before its first digit emerges, and chained operations stack the stalls.
Their merged multiply add unit fuses the two operations into one pipeline with a single, shorter startup delay per pass. Running many input depths in parallel lifts throughput further. Against a processor baseline the design delivered up to ten times the energy efficiency. It also used about nine times less energy than earlier digit serial chip designs. That suits it to latency sensitive imaging at the edge.
Catalog record
| CREATOR | Muhammad Usman, Yousef Sadegheih, Dorit Merhof |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| LICENSE | http://creativecommons.org/publicdomain/zero/1.0/ |
| RIGHTS | CC0 |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.25562
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Energy-Efficient CNN Acceleration with MSDF Digit-Serial Arithmetic on FPGA Muhammad Usman, Yousef Sadegheih, Dorit Merhof Abstract This paper presents an energy-efficient hardware acceleration of the convolutional layers in the U-Net architecture for image segmentation, implemented on FPGA. While digit-serial arithmetic, particularly most-significant-digit-first (MSDF) techniques, offers a compact hardware footprint, it suffers from initial latency before producing the first output digit. This delay accumulates in cascaded operations like multiplication followed by addition, where each unit introduces its own startup overhead. To overcome this, we propose a merged multiply-add (MMA) architecture that fuses these operations into a unified pipeline. Instead of incurring separate delays, the MMA introduces a single streamlined latency per iteration, shorter than the combined latency of conventional cascaded units, resulting in enhanced throughput and efficiency. The MMA units are designed to process spatial input depths in parallel, achieving significantly higher performance than both standalone MSDF-based and conventional designs. We evaluate the proposed design using U-Net as a target application. Despite operating at a lower frequency than a CPU, the FPGA-based accelerator achieves up to an order of magnitude higher energy efficiency, delivering up to 15.14 15.14 GOPS/W compared to 1.93 1.93 GOPS/W for CPU-based inference. The design also shows approximately 9 × 9\times reduction in energy consumption compared to MSDF-based FPGA implementations. These results highlight the efficacy of the merged arithmetic approach for resource-constrained, latency-sensitive edge applications in medical imaging and computer vision. Keywords bit-serial \cdot most significant digit first arithmetic \cdot FPGA acceleration \cdot U-Net 1 Introduction Accurate brain tumor segmentation plays a vital role in medical imaging by assisting clinicians in diagnosis, treatment planning, and postoperative assessment. U-Net, a convolutional neural network (CNN)-based architecture, has become the de facto standard in biomedical image segmentation due to its ability to capture both contextual and localization features, making it well-suited for delineating complex anatomical structures such as gliomas, meningiomas, and metastases in brain MRI scans [ 1 ] . The computation in U-Net is primarily dominated by convolution operations, which demand high computational resources and energy when implemented on CPU and GPU [ 2 ] , and poses challenges for embedded deployment. FPGAs (Field-Programmable Gate Arrays) have emerged as a promising hardware platform for accelerating CNN workloads [ 11 ] . They offer inherent advantages such as parallelism, low power consumption, and the ability to tailor architectures to specific applications. These characteristics make FPGAs well-suited for edge AI applications and embedded systems where energy efficiency and adaptability are critical, such as real-time diagnostic systems [ 15 ] . Nearly 85\% of the computation time in CNN-based models is spent on inner product calculations. To accelerate these operations, various inner product computation units have been developed with a focus on optimizing power, performance, and silicon area. Among these, bit-serial computation techniques have shown significant promise. By reducing interconnect bandwidth and simplifying hardware complexity, bit-serial methods are especially well-suited for low-power and area-constrained systems [ 3 ] . Unlike traditional fixed-width multiplication, bit-serial architectures perform multiplication sequentially, one bit at a time, accumulating results into partial sums. This breakdown simplifies hardware design, reduces logic complexity, and allows dynamic precision control, which is especially useful for applications requiring flexible accuracy and power trade-offs. Moreover, bit-serial processing can lead to substantial energy savings by eliminating redundant computation and minimizing data movement [ 10 ] . To this end, we propose a digit-serial MSDF arithmetic-based merged multiply-add (MMA) unit to accelerate convolution operations in U-Net, offering enhanced throughput with low hardware overhead. The key contributions of this work are as follows: • A novel MSDF-based multiply-add architecture fuses digit-serial multiplication and accumulation into a single pipeline. Unlike conventional designs with separate multiplier and adder delays, the proposed unit incurs only a single combined delay. This reduces per-iteration latency and improves throughput. • The MMA units are structured to process multiple spatial input depths in parallel, significantly increasing the throughput of convolution operations central to U-Net. • An energy-efficient FPGA-based accelerator is implemented for the convolutional layers in U-Net, demonstrating its applicability to resource-constrained, latency-sensitive edge applications. • The performance of the proposed design is compared with CPU, bit-parallel, conventional and MSDF arithmetic-based convolution computation units in terms of latency, throughput, area and energy efficiency. 2 MSDF Arithmetic Most-significant digit first (MSDF) arithmetic has emerged as a promising technique in CNN acceleration [ 6 ] . It enables digit-wise computation that supports early termination and natural pipelining, which are particularly advantageous for reducing latency and improving throughput in deep CNN inference. The computation of most significant digit first offers multiple benefits, including early termination, low latency, and progressive refinement of results. It naturally supports approximate computing, as early digits often provide enough precision for decision-making [ 13 ] . Furthermore, its digit-wise pipelining structure enables overlapping computation and communication, which helps maximize throughput. Unlike traditional multiply-accumulate (MAC) units, MSDF-based inner product units are lightweight and can be deeply pipelined, making them ideal for FPGA fabric without reliance on DSP blocks. One inherent limitation of MSDF arithmetic is the presence of initial delay ( \delta ), a fixed number of cycles that must elapse before the first digit of the result becomes available, increasing the computation latency. Traditionally, the MSDF inner product unit is developed using a multiplier followed by an adder tree structure to accumulate partial products [ 12 ] . In such implementations, each stage of the adder tree introduces its own delay, compounding the overall latency. To mitigate this issue, we propose to use a MSDF inner product computation with merged multiplication and addition (MMA) unit [ 4 ] . This architecture eliminates the need for a separate adder tree and thus removes the associated initial delays, resulting in a more efficient datapath with reduced latency in MSDF-based computation systems. Figure 1: System-level architecture of the proposed U-Net convolution accelerator. Each merged multiplication-addition (MMA) unit computes the inner product across T_{N} input channels using bit-serial activations and parallel n -bit weights. A group of k^{2} MMAs forms a Kernel Processing Block (KPB), handling the k\times k convolution over the spatial window. The outputs of the k^{2} MMA units are summed using an MSDF adder tree to produce a single partial sum corresponding to one output pixel. The complete system instantiates 16 parallel KPBs, enabling the computation of 16 output pixels per clock group. 3 Proposed Design 3.1 Overall Architecture The overall architecture of the proposed design is shown in Fig. 1 . The core computation unit of the accelerator is the MSDF arithmetic-based merged multiplication and addition unit (MMA), which computes the inner product across T_{N}=32 input channels. The detailed working principle of MMA is explained in Section 3.2 . To implement the k\times k ( k=3 ) kernel convolutions of U-Net efficiently, k\times k=9 MMA units are grouped to form a column. Each MMA unit corresponds to one spatial location within the k\times k convolution window but processes the same N input channels in parallel. Outputs from the nine MMA units are combined using a \lceil\log_{2}(k^{2})\rceil -stage MSDF adder tree (MAT) to form a Kernel Processing Block (KPB), preserving the digit-serial nature for continuous streaming and overlapping computation with communication. Collectively, the KPB computes the convolution over a k\times k\times T_{N} volume,