An Analytical Cost Model for Fast Evaluation of Multiple Compute-Engine CNN Accelerators

Fareed Qararyah , Mohammad Ali Maleki , Pedro Trancoso · 2025

Read An Analytical Cost Model for Fast Evaluation of Multiple Compute-Engine CNN Accelerators by Fareed Qararyah , Mohammad Ali Maleki , Pedro Trancoso (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-07242.txt.

Introduction

Fareed Qararyah, Mohammad Ali Maleki, and Pedro Trancoso present a fast cost model for chips that speed up neural networks. Their 2025 work targets designs that split work among several compute engines on a programmable chip. Such designs vary widely, so picking the best layout is hard.

The model scores speed and efficiency without slow chip synthesis, running about one hundred thousand times faster. Its accuracy stays above ninety percent on average. Three case studies show its range: broad ranking of top designs, fine checks that expose bottlenecks, and wide searches of the design space. No single layout wins on all counts. Fast scoring lets searches find designs that beat current best ones.

Catalog record

CREATORFareed Qararyah , Mohammad Ali Maleki , Pedro Trancoso
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-07242

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

An Analytical Cost Model for Fast Evaluation of Multiple Compute-Engine CNN Accelerators Thanks: © 2025 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Fareed Qararyah , Mohammad Ali Maleki , Pedro Trancoso Abstract Convolutional Neural Networks (CNNs) serve various applications with diverse performance and resource requirements. Model-aware CNN accelerators best address these diverse requirements. These accelerators usually combine multiple dedicated Compute Engines (CEs). The flexibility of Field-Programmable Gate Arrays (FPGAs) enables the design of such multiple Compute-Engine (multiple-CE) accelerators. However, existing multiple-CE accelerators differ in how they arrange their CEs and distribute the FPGA resources and CNN operators among the CEs. The design space of multiple-CE accelerators comprises numerous such arrangements, which makes a systematic identification of the best ones an open challenge. This paper proposes a M ultiple- C E accelerator analytical C ost M odel (MCCM) and an evaluation methodology built around MCCM. The model and methodology streamline the expression of any multiple-CE accelerator and provide a fast evaluation of its performance and efficiency. MCCM is in the order of 100000 × 100000\times faster than traditional synthesis-based evaluation and has an average accuracy of > 90 % >90\% . The paper presents three use cases of MCCM. The first describes an end-to-end evaluation of state-of-the-art multiple-CE accelerators considering various metrics, CNN models, and resource budgets. The second describes fine-grained evaluation that helps identify performance bottlenecks of multiple-CE accelerators. The third demonstrates that MCCM fast evaluation enables exploring the vast design space of multiple-CE accelerators. These use cases show that no unique CE arrangement achieves the best results given different metrics, CNN models, and resource budgets. They also show that fast evaluation enables design space exploration, resulting in accelerator designs that outperform state-of-the-art ones. MCCM is available at https://github.com/fqararyah/MCCM . I Introduction The use of Convolutional Neural Networks (CNNs) in various application domains and across a spectrum of environments, ranging from edge to cloud, brings a wide range of performance requirements and optimization goals [ 39 , 11 ] . While some applications are latency-critical, others target high throughput. Moreover, resource efficiency is crucial when having limited resources [ 12 , 43 ] , and when resources are shared by multiple applications [ 40 ] . FPGAs’ reconfigurability enables the design of accelerators tailored to such diverse requirements, which makes FPGA-based CNN accelerators popular [ 13 , 12 , 49 , 3 , 42 , 43 ] . Accelerators that organize FPGA resources into generic reusable Compute Engines (CEs) have limited adaptability to the varying characteristics of CNN layers, leading to dynamic resource underutilization [ 41 , 50 , 29 , 5 , 30 ] . This limitation is often described as one size does not fit all. Multiple Compute-Engine ( multiple-CE ) accelerators overcome this limitation. They organize FPGA resources into a number of dedicated CEs tailored to the structure of the CNN model and the available resources [ 1 , 50 , 25 , 30 , 32 , 33 , 38 , 41 , 5 ] . However, the existing multiple-CE accelerators differ in how they arrange their CEs and distribute the FPGA resources and CNN layers among the CEs. Identifying CE arrangements that achieve the best performance and efficiency requires a systematic approach to multiple-CE accelerator design. A systematic accelerator design approach requires analyzing the bottlenecks and inefficiencies and exploring the design space to identify optimizations that mitigate them. The literature reveals that the inefficiencies of Deep Learning (DL) accelerators, in general, have three root causes, (1) Processing Element (PE) underutilization [ 4 , 23 , 48 ] , (2) large on-chip buffers [ 4 , 27 , 46 ] , (3) and the time and energy costly off-chip access [ 48 , 23 ] . Considering multiple-CE accelerators, the second and third causes are still open challenges. However, multiple-CE accelerators have less PE underutilization than generic ones. Multiple-CE accelerators mitigate PE underutilization bottleneck but in a way that creates throughput and latency trade-off [ 41 , 30 , 5 ] . Hence, a systematic multiple-CE accelerator design approach requires analyzing PE underutilization with latency-throughput trade-offs, on-chip buffer requirements, and off-chip access. The large design space of CE arrangement possibilities and the lack of a fast methodology that models and evaluates their impact on multiple-CE accelerators’ performance and efficiency make a systematic design approach impractical. While High-Level Synthesis (HLS) shortens the design time, synthesizing a single accelerator instance can take hours. To overcome this challenge, this paper proposes a Multiple-CE accelerator analytical Cost Model (MCCM), and an evaluation methodology built around it. The proposed model and methodology enable the expression of any multiple-CE accelerator using simple notation and provide a fast and accurate evaluation of its throughput, latency, on-chip buffer requirements, and off-chip accesses. This paper makes the following contributions: • It proposes a multiple-CE accelerator analytical cost model (MCCM). MCCM applies bottom-up modeling of the impact of CE arrangement possibilities on the performance and efficiency of multiple-CE accelerators. • It proposes MCCM-based evaluation methodology, which enables expressing any multiple-CE accelerator and evaluates its latency, throughput, on-chip buffer requirements, and off-chip accesses orders of magnitude faster than the traditional approach. • It presents three use cases of MCCM. First, an end-to-end evaluation of state-of-the-art multiple-CE accelerators considering different metrics, CNNs, and resource budgets. Second, fine-grained evaluation of multiple-CE accelerators’ performance bottlenecks. Third, a design space exploration of multiple-CE accelerators enabled by MCCM fast evaluation. MCCM is in the order of 100000\times faster than synthesis-based evaluation and has an average accuracy of >90\% . The presented MCCM use cases using three state-of-the-art multiple-CE accelerators, five CNNs, and four FPGA boards show that no single multiple-CE architecture is the best, given different metrics, CNN models, and resource budgets. Moreover, the evaluation shows the impact of engine arrangements on multiple-CE accelerators’ performance bottlenecks. Finally, using MCCM fast evaluation as a basis for multiple-CE accelerators design space exploration enables identifying designs that outperform the state-of-the-art. II Background and Motivation II-A Convolutional Neural Networks (CNNs) Convolutional Neural Networks (CNNs) are feed-forward Deep learning (DL) algorithms [ 20 ] . A CNN comprises a sequence of layers that perform feature extraction and classification [ 21 ] . Convolutional layers are the primary layers in a CNN. A convolutional layer consists of a collection of trainable parameters, known as weights , which are organized into filters . These filters extract features from multi-dimensional inputs or intermediate results. The inputs of a layer are referred to as Input Feature Maps ( IFMs ), and the outputs as Output Feature Maps ( OFMs ). The term Feature Maps ( FMs ), or activations, refers to both IFMs and OFMs. FMs consist of 2D slices known as channels . II-B CNN Compute-Engines Fig. 1 : Compute-Engine (CE) parallelism examples As convolutions represent more than 90% of modern CNN operations [ 23 ] , CNN Compute-Engine (CE) optimizations focus on convolutional layers. A CE comprises a grid of

Read the complete text