DOSA: D ifferentiable Model-Based O ne-Loop S earch for DNN A ccelerators Conference: 56th Annual IEEE/ACM International Symposium on Microarchitecture; October 28-November 1, 2023; Toronto, ON, Canada 56th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO ’23), October 28-November
Charles Hong, Qijing Huang, Grace Dinh, Mahesh Subedar, Yakun Sophia Shao · 2025
Read DOSA: D ifferentiable Model-Based O ne-Loop S earch for DNN A ccelerators Conference: 56th Annual IEEE/ACM International Symposium on Microarchitecture; October 28-November 1, 2023; Toronto, ON, Canada 56th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO ’23), October 28-November by Charles Hong, Qijing Huang, Grace Dinh, Mahesh Subedar, Yakun Sophia Shao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-10702.txt.
Introduction
Designing a neural network accelerator means choosing hardware and the mapping of the algorithm onto it, together. Charles Hong and four co-authors automate that joint search with DOSA in this 2025 computer architecture paper. Earlier tools explored the two spaces separately, and both are huge and jagged, so the combined problem explodes combinatorially.
DOSA replaces the search with differentiable performance models and gradient descent, sliding through hardware parameters and mappings at once. It beat random search and Bayesian optimization by wide margins on the energy-delay product, the field's combined score for speed and power. The design is modular: adding a learned model beside the analytical one let them optimize a real accelerator's buffers and mappings for another large gain. For chip designers, co-optimization becomes a smooth descent instead of a lottery.
Catalog record
| CREATOR | Charles Hong, Qijing Huang, Grace Dinh, Mahesh Subedar, Yakun Sophia Shao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-10702
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DOSA: D ifferentiable Model-Based O ne-Loop S earch for DNN A ccelerators Conference: 56th Annual IEEE/ACM International Symposium on Microarchitecture; October 28-November 1, 2023; Toronto, ON, Canada 56th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO ’23), October 28-November 1, 2023, Toronto, ON, Canada DOI: 10.1145/3613424.3623797 ISBN: 979-8-4007-0329-4/23/10 CCS: Hardware Hardware-software codesign CCS: Hardware Application specific processors CCS: Computing methodologies Machine learning CCS: Computing methodologies Modeling and simulation CCS: Computing methodologies Search methodologies Charles Hong, Qijing Huang, Grace Dinh, Mahesh Subedar, Yakun Sophia Shao Abstract . In the hardware design space exploration process, it is critical to optimize both hardware parameters and algorithm-to-hardware mappings. Previous work has largely approached this simultaneous optimization problem by separately exploring the hardware design space and the mapspace—both individually large and highly nonconvex spaces—independently. The resulting combinatorial explosion has created significant difficulties for optimizers. In this paper, we introduce DOSA, which consists of differentiable performance models and a gradient descent-based optimization technique to simultaneously explore both spaces and identify high-performing design points. Experimental results demonstrate that DOSA outperforms random search and Bayesian optimization by 2.80 × \times and 12.59 × \times , respectively, in improving DNN model energy-delay product, given a similar number of samples. We also demonstrate the modularity and flexibility of DOSA by augmenting our analytical model with a learned model, allowing us to optimize buffer sizes and mappings of a real DNN accelerator and attain a 1.82 × \times improvement in energy-delay product. Figure 1 . Hardware-first, two-loop (left) and mapping-first, one-loop (right) DSE approaches. 1. Introduction Deep neural network (DNN) accelerators ( NVIDIA 2020 ; Jouppi et al. 2017 ; Lauterbach 2021 ; Amazon 2018 ) have become a critical driving force for the recent breakthroughs ( Krizhevsky et al. 2012 ; Goodfellow et al. 2014 ; He et al. 2016 ; Vaswani et al. 2017 ; OpenAI 2020 ) in artificial intelligence. To develop efficient DNN accelerators in a fast and cost-effective manner, automated design space exploration (DSE) has emerged as a powerful technique. The hardware DSE flow ( Venkatesan et al. 2019 ; Parashar et al. 2019 ; Kwon et al. 2020 ) involves optimizing over two search spaces: the hardware design space , which describes hardware design parameters such as interconnect topology and buffer and systolic array sizes, and the mapspace , which describes how applications are executed on the target hardware and encompasses decisions such as tiling, dataflow, and spatio-temporal mapping. For both the hardware design space and the mapspace, the goal is to optimize a performance metric, such as energy-delay product (EDP), subject to certain constraints. These include design budgets, such as bounds on the area or power consumption, as well as constraints ensuring that the selected mapping can be executed on the selected hardware configuration (e.g. that the hardware buffers are sufficiently large to contain the tiles). As these constraints encompass both the mapspace and the hardware design space, the two spaces must be simultaneously optimized over; techniques solely tackling hardware search ( Kumar et al. 2021 ; Yazdanbakhsh et al. 2021 ) or mapping optimizations ( Paszke et al. 2019 ; Abadi et al. 2016 ; NVIDIA 2018 ; Chen et al. 2015 ; Chen et al. 2018 ; Sabne 2020 ; Kjolstad et al. 2017 ) are insufficient to achieve the optimal hardware design in DSE. Both the hardware design and mapping spaces are vast, high-dimensional, and comprised of both categorical and discrete variables. Furthermore, evaluating the performance of a hardware configuration and a mapping can be computationally expensive. The size of the combined optimization space and the cost of evaluating points in it pose formidable challenges to DSE algorithms. Much prior work ( Venkatesan et al. 2019 ; Zhang et al. 2022 ; Huang et al. 2022 ; Sakhuja et al. 2023 ; Xiao et al. 2021 ; Lin et al. 2021 ) has approached this problem using hardware-first search . These methods directly search over the space of possible hardware configurations. The performance of each hardware configuration is calculated by first constraining the mapspace to mappings that are compatible with the hardware configuration, then optimizing over the constrained (highly discontinuous) mapspace. In most cases, the mapspace optimization is done iteratively, rendering this process a two-loop approach iterating over both the hardware space and mapspace. As a result, these approaches must contend with a combinatorial explosion of possible configurations. Alternatively, mapping-first approaches, as proposed in ( Kao et al. 2022b ; Yang et al. 2020 ) and illustrated in Figure 1 , optimize primarily over the mapspace. For each mapping, optimizing over the hardware design space is a straightforward process consisting of finding the minimal hardware configuration capable of supporting the mapping. As a result, the loop for hardware search is eliminated, allowing the entire DSE process to be encapsulated in a single loop. Furthermore, the lack of hardware resource constraints also significantly simplifies the mapspace search problem. Despite these advantages, mapping-first approaches must still contend with the size of the mapspace and the nonconvexity of the performance over this space. Prior works have either directly applied black-box optimization methods ( Kao et al. 2022b ; Shi et al. 2020 ) , which rely on a large number of (often expensive to collect) samples, or pruned the search space using architecture-specific heuristics constructed by hand based on observations of a limited subset of the DSE search space ( Yang et al. 2020 ) . Reducing the sample complexity of DSE while still allowing for a systematic exploration of the entire space requires leveraging domain knowledge—for instance, a generalizable performance model like Timeloop ( Parashar et al. 2019 ; Wu et al. 2019 ) (a popular analytical model for DNN accelerators), that is not reliant on an expensive training process. This paper follows this approach, using performance models as an optimization target for mapping-first search. Specifically: • We build closed-form differentiable and interpretable performance models for latency and energy on DNN accelerators. Our models are as precise as the state-of-the-art program-based analytical models, while also being amenable to white-box optimization techniques such as gradient descent. • We then introduce DOSA 1 1 1 Code open-sourced at https://github.com/ucb-bar/dosa . Artifacts available: see Appendix A . , a mapping-first one-loop DSE flow that uses gradient descent to find the most efficient hardware parameters and mappings to target multi-layer DNNs; to the best of our knowledge, this is the first work to use a mapping-first strategy to simultaneously perform DSE for multiple layers of a neural network. DOSA converges at least 40% faster than state-of-the-art DSE approaches. • We take a step beyond DSE for architectural model by introducing a DNN model to predict the variation between analytical model and real hardware accelerator performance, and using it to augment our differentiable model for real hardware DSE. • We benchmark our results on the Gemmini accelerator, showing a 1.82 \times EDP improvement over hand-designed configurations. 2. Background Name Mapspace Search Hardware Search Two-loop Searchers Spotlight ( Sakhuja et al. 2023 ) BB-BO BB-BO VAESA ( Huang et al. 2022 ) ILP ( Huang et al. 2021 ) VAE+BB-BO/GD FAST ( Zhang et al. 2022 ) BB-LCS ( Karro et al. 2017 ) +ILP BB-LCS HASCO ( Xiao et