MATCHA: Efficient Deployment of Deep Neural Networks on Multi-Accelerator Heterogeneous Edge SoCs
Enrico Russo † \dagger , Mohamed Amine Hamdi ‡ \ddagger , Alessandro Ottaviano ⋆ \star , Francesco Conti § \lx@sectionsign , Angelo Garofalo ⋆ \star § \lx@sectionsign , Daniele Jahier Pagliari ‡ \ddagger , Maurizio Palesi † \dagger , Luca Benini ⋆ \star § \lx@sectionsign , Alessio Burrello ‡ \ddagger · 2026
Read MATCHA: Efficient Deployment of Deep Neural Networks on Multi-Accelerator Heterogeneous Edge SoCs by Enrico Russo † \dagger , Mohamed Amine Hamdi ‡ \ddagger , Alessandro Ottaviano ⋆ \star , Francesco Conti § \lx@sectionsign , Angelo Garofalo ⋆ \star § \lx@sectionsign , Daniele Jahier Pagliari ‡ \ddagger , Maurizio Palesi † \dagger , Luca Benini ⋆ \star § \lx@sectionsign , Alessio Burrello ‡ \ddagger (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-09124.txt.
Introduction
Russo and eight colleagues present MATCHA, a unified system for running deep neural networks on edge chips with multiple mixed speedup engines. Most systems cannot fully exploit such mix. MATCHA builds highly joint plans for mixed engines and uses constraint code to tune memory sharing and timing across levels. Pattern matching, tiling, and mapping across single hardware units enable side-by-side runs. The system targets lean use on multi-engine mixed edge systems-on-chip.
Catalog record
| CREATOR | Enrico Russo † \dagger , Mohamed Amine Hamdi ‡ \ddagger , Alessandro Ottaviano ⋆ \star , Francesco Conti § \lx@sectionsign , Angelo Garofalo ⋆ \star § \lx@sectionsign , Daniele Jahier Pagliari ‡ \ddagger , Maurizio Palesi † \dagger , Luca Benini ⋆ \star § \lx@sectionsign , Alessio Burrello ‡ \ddagger |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32118 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.09124
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MATCHA: Efficient Deployment of Deep Neural Networks on Multi-Accelerator Heterogeneous Edge SoCs 1267 Conference: 63rd ACM/IEEE Design Automation Conference; July 26–29, 2026; Long Beach, CA, USA Thanks: Corresponding authors: [email protected] , [email protected] This work was accepted at the 63rd ACM/IEEE Design Automation Conference (DAC26) Enrico Russo † \dagger , Mohamed Amine Hamdi ‡ \ddagger , Alessandro Ottaviano ⋆ \star , Francesco Conti § \lx@sectionsign , Angelo Garofalo ⋆ \star § \lx@sectionsign , Daniele Jahier Pagliari ‡ \ddagger , Maurizio Palesi † \dagger , Luca Benini ⋆ \star § \lx@sectionsign , Alessio Burrello ‡ \ddagger Abstract . Deploying DNNs on System-on-Chips (SoC) with multiple heterogeneous acceleration engines is challenging, and the majority of deployment frameworks cannot fully exploit heterogeneity. We present MATCHA, a unified DNN deployment framework that generates highly concurrent schedules for parallel, heterogeneous accelerators and uses constraint programming to optimize L3/L2 memory allocation and scheduling. Using pattern matching, tiling, and mapping across individual HW units enables parallel execution and high accelerator utilization. On the MLPerf Tiny benchmark, using a SoC with two heterogeneous accelerators, MATCHA improves accelerator utilization and reduces inference latency by up to 35% with respect to the the state-of-the-art MATCH compiler. 1. Introduction Deep Neural Networks (DNNs) are increasingly being deployed at the edge, where they must execute at high performance under tight latency, energy, and cost constraints ( 11 ) . To meet these competing objectives, modern edge systems-on-chip (SoCs) are becoming increasingly heterogeneous, integrating multiple specialized accelerators alongside general-purpose host CPUs ( 27 ; 9 ; 7 ) . Equipping a SoC with more than one kind of DNN accelerators enhances flexibility, as different architectural templates, e.g., Single-Instruction Multiple Data (SIMD) ( 16 ) , vector units ( 22 ) , systolic or dataflow processors for General Matrix Mutiplication (GEMM) ( 27 ; 10 ) , etc, provide different latency/energy efficiency trade-offs as a function of the accelerated operation(s) and corresponding tensors geometries ( 7 ) . However, deployment frameworks for these SoCs often fail to fully exploit this flexibility, resulting in low hardware utilization, as they either offload entire DNNs to a single accelerator, or at most, map each DNN layer (or fused layer sequence) to the most suited compute unit, in a purely sequential fashion, controlled synchronously by the host ( 12 ; 5 ; 28 ; 25 ; 21 ) . As a result, only one accelerator is active at any given time, while the others remain idle. In this paper, we examine two approaches for optimizing the utilization of available hardware resources. The first one leverages the structure of modern DNN computational graphs, which often include multiple independent branches (e.g., in ResNet-like ( 13 ) networks or Attention blocks ( 29 ) ), that can be potentially offloaded in parallel to different accelerators. However, this approach is limited to multi-branch DNN models, and the optimization space is restricted to one-to-one layer to accelerator mappings, which are often insufficient to eliminate idleness entirely, as branches could be highly unbalanced in workload. Thus, at a finer grain, we explore the parallelization opportunities provided by tiling DNN operators and assigning each tile to a different accelerator. While a trivial implementation of this offloading scheme is classic model parallelism, in which layer workloads are split evenly across homogeneous compute units ( 3 ) , accelerators’ heterogeneity in terms of operator support and performance makes that solution either sub-optimal or unfeasible, requiring a more general problem framing. To fully support both these offloading schemes, we propose a novel DNN optimizing compiler that simultaneously explores layer-level and tile-level parallelization, targeting OS-less, multi-accelerator SoCs for edge inference. Our compiler is based on TVM ( 6 ) and on its extension MATCH for heterogeneous accelerators support ( 12 ) , but we completely redesign both the mapping optimization engine and the runtime to support concurrent, asynchronous execution on multiple compute units. Hence, we name it MATCHA (MATCH Asynchronous). Our main contributions are the following: • We present a unified DNN deployment flow supporting concurrent offloading onto multiple accelerators, possibly with different internal architectures, thanks to a flexible model-based latency cost abstraction. To our knowledge, MATCHA is the first tool to support such kind of mapping for OS-less heterogeneous SoCs. • We propose a constraint programming-based, tile-centric mapping framework to perform heterogeneous pattern matching and distribute work among accelerators, maximizing utilization. • Through an experimental campaign on a complex heterogeneous SoC, equipped with a host CPU and two distinct accelerators, and on multiple full networks and layer blocks ( 1 ; 13 ; 30 ; 29 ) , we show that MATCHA reduces end-to-end latency by up to 35% with respect to the state-of-the-art MATCH compiler. 2. Background and Related Work Table 1. Comparison of state-of-the-art DNN deployment tools. Work Open Source HSoC Support Operator Extension Device Extension Multi ISA Async. Execution Pattern Tiling TFLite ( 8 ) \bullet \bullet TelaMalloc ( 17 ) \bullet \bullet Bolt ( 31 ) \bullet COMB ( 33 ) HTVM ( 28 ) DORY ( 5 ) Deeploy ( 25 ) NEST-C ( 21 ) Map-and-Conquer ( 4 ) MATCH ( 12 ) Ours Yes No Limited \bullet Not Applicable Deploying deep neural networks efficiently on embedded and edge platforms has driven the design of a wide range of DNN accelerators and optimizing compilers. Despite the diversity of hardware and software stacks, most deployment frameworks follow a similar pipeline with four main stages: (1) representing the DNN as a computation graph in an intermediate representation (IR) and applying graph-level transformations, (2) partitioning the workload into subgraphs or tiles with an appropriate granularity, (3) mapping each subgraph onto the available execution modules, and (4) generating target-specific code. The third step, commonly referred to as the mapping phase, has been extensively studied for spatial and systolic DNN accelerators ( 26 ; 14 ; 15 ; 24 ; 20 ; 32 ) . Mapping decisions involve loop tiling (to fit large tensors into small local scratchpad memories), unrolling (to parallelize computation across the available processing elements), and ordering (to maximize data reuse and minimize data movement across the memory hierarchy). Modern edge platforms increasingly adopt heterogeneous systems-on-chip (HSoCs), where a general-purpose host core is coupled with multiple specialized accelerators exposing different ISAs, data types, and memory organizations. Examples range from multi-processor SoCs (MPSoCs) that integrate CPU cores, GPUs, and NPUs on a shared DRAM (e.g., NVIDIA Xavier, Apple and Qualcomm SoCs) ( 7 ) , to extreme edge-oriented SoCs that combine RISC-V control cores with tightly coupled accelerator clusters ( 27 ; 9 ) . In this work we target edge HSoCs that offer several accelerators and processor clusters connected through a software-managed, multi-level scratchpad memory hierarchy. Table 1 compares representative DNN deployment frameworks for these systems along key dimensions: open-source availability, explicit support for heterogeneous SoCs, extensibility of operator and device backends, support for multi-ISA systems, asynchronous execution capability, and tile-level inter-device parallelization. TFLite ( 8 ) provides portability and pluggable kernels but offers limited control over mapping and tiling. DORY ( 5 ) optimizes memory allocation for single accelerators using constraint programming, while TelaMalloc ( 17 ) and Bolt ( 31 ) target specific subproblems such as memory allocation or GPU tuning without full