Concorde: Fast and Accurate CPU Performance Modeling with Compositional Analytical-ML Fusion

Arash Nasr-Esfahany ∗ , Mohammad Alizadeh ∗ , Victor Lee , Hanna Alam , Brett W. Coon David Culler , Vidushi Dadu , Martin Dixon , Henry M. Levy , Santosh Pandey ∗ Parthasarathy Ranganathan , Amir Yazdanbakhsh Google , MIT , University of Washington , Rutgers University , Google DeepMind { arashne , · 2025

Read Concorde: Fast and Accurate CPU Performance Modeling with Compositional Analytical-ML Fusion by Arash Nasr-Esfahany ∗ , Mohammad Alizadeh ∗ , Victor Lee , Hanna Alam , Brett W. Coon David Culler , Vidushi Dadu , Martin Dixon , Henry M. Levy , Santosh Pandey ∗ Parthasarathy Ranganathan , Amir Yazdanbakhsh Google , MIT , University of Washington , Rutgers University , Google DeepMind { arashne , (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23076.txt.

Introduction

Chip designers rely on cycle-level simulators to test new processors, but those tools are far too slow to explore many designs. A team from Google, MIT, and partner universities presents Concorde in a 2025 paper, a model that blends simple analytical formulas with machine learning. Instead of replaying each instruction, it summarizes a program by how each processor part limits it.

The model runs more than a hundred thousand times faster than the reference simulator while missing true cycle counts by only about two percent. That speed let the team run a first-of-its-kind study assigning performance losses to specific processor parts across millions of evaluations.

Catalog record

CREATORArash Nasr-Esfahany ∗ , Mohammad Alizadeh ∗ , Victor Lee , Hanna Alam , Brett W. Coon David Culler , Vidushi Dadu , Martin Dixon , Henry M. Levy , Santosh Pandey ∗ Parthasarathy Ranganathan , Amir Yazdanbakhsh Google , MIT , University of Washington , Rutgers University , Google DeepMind { arashne ,
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-23076

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Concorde: Fast and Accurate CPU Performance Modeling with Compositional Analytical-ML Fusion Arash Nasr-Esfahany ∗ , Mohammad Alizadeh ∗ , Victor Lee , Hanna Alam , Brett W. Coon David Culler , Vidushi Dadu , Martin Dixon , Henry M. Levy , Santosh Pandey ∗ Parthasarathy Ranganathan , Amir Yazdanbakhsh Google , MIT , University of Washington , Rutgers University , Google DeepMind { arashne , alizadeh }@mit.edu , santosh.pandey @rutgers.edu , { vwlee , hannaalam , bwc , dculler , vidushid , mgdixon , hanklevy , parthas , ayazdan }@google.com Abstract . Cycle-level simulators such as gem5 are widely used in microarchitecture design, but they are prohibitively slow for large-scale design space explorations. We present Concorde , a new methodology for learning fast and accurate performance models of microarchitectures. Unlike existing simulators and learning approaches that emulate each instruction, Concorde predicts the behavior of a program based on compact performance distributions that capture the impact of different microarchitectural components. It derives these performance distributions using simple analytical models that estimate bounds on performance induced by each microarchitectural component, providing a simple yet rich representation of a program’s performance characteristics across a large space of microarchitectural parameters. Experiments show that Concorde is more than five orders of magnitude faster than a reference cycle-level simulator, with about 2% average Cycles-Per-Instruction (CPI) prediction error across a range of SPEC, open-source, and proprietary benchmarks. This enables rapid design-space exploration and performance sensitivity analyses that are currently infeasible, e.g., in about an hour, we conducted a first-of-its-kind fine-grained performance attribution to different microarchitectural components across a diverse set of programs, requiring nearly 150 million CPI evaluations. $\ast$ $\ast$ footnotetext: Work done at Google. 1. Introduction Microarchitecture simulators are a key tool in the computer architect’s arsenal ( Skadron et al., 2003 ; Binkert et al., 2011 ; Patel et al., 2011 ; Austin et al., 2002 ; Carlson et al., 2011 ; Sanchez and Kozyrakis, 2013 ; Akram and Sawalha, 2019 ; Elrabaa et al., 2017 ; Alves et al., 2015 ) . From SimpleScalar ( Austin et al., 2002 ) to gem5 ( Binkert et al., 2011 ; Lowe-Power et al., 2020 ) , simulators have enabled architects to explore new designs and optimize existing ones without the prohibitive costs of fabrication. CPU simulation, in particular, has become increasingly important as hyperscale companies like Google (Axion) ( Vahdat, 2024 ) , Amazon (Graviton) ( Amazon, 2023 ) , Microsoft (Cobalt) ( Microsoft, 2024 ) increasingly invest in developing custom CPU architectures tailored to their specific workloads. The landscape of CPU performance modeling is characterized by a critical tension between model accuracy and speed. This trade-off manifests in the various levels of abstraction employed by different performance models ( Binkert et al., 2011 ; Genbrugge et al., 2010 ; Van den Steen et al., 2016 ) . At one end of the spectrum lie analytical models ( Van den Steen et al., 2016 ; Abel et al., 2023 ) , which provide simplified mathematical representations of microarchitectural components and their interactions. Although they are fast, analytical models often lack the detailed modeling necessary to capture the dynamics of modern processors accurately. At the other end of the spectrum reside cycle-level simulators like gem5 ( Binkert et al., 2011 ) , which can provide high-fidelity results by meticulously modeling every cycle of execution. However, this level of detail comes at a steep computational cost, becoming prohibitively slow for large-scale design space exploration ( Karandikar et al., 2018 ; Genbrugge et al., 2010 ; Li et al., 2022 ) , programs with billions of instructions, or detailed sensitivity studies. Recognizing the limitations of conventional methods, there has been growing interest in using machine learning (ML) to expedite CPU simulation ( Mendis et al., 2019 ; Li et al., 2022 ; Pandey et al., 2024 ; Li et al., 2023 ) . Rather than explicitly model every cycle and microarchitectural interaction, these methods learn an approximate model of the architecture’s performance from a large corpus of data. A typical approach is to pose the problem as learning a function mapping a sequence of instructions to the target performance metrics. For example, recent work ( Pandey et al., 2024 ; Kumar et al., 2022 ; Li et al., 2022 ; Li et al., 2023 ) train sequence models (e.g., LSTMs ( Graves and Schmidhuber, 2005 ) and Transformers ( Vaswani et al., 2017 ) ) on ground-truth data from a cycle-level simulator to predict metrics such as the program’s Cycles Per Instruction (CPI). These methods show promise in providing fast performance estimates with reasonable accuracy. However, relying on black-box ML models operating on instruction sequences has several limitations. First, the computational cost of these methods scales proportionally with the length of the instruction sequence, i.e. \mathcal{O}(L) where L is the instruction sequence length. The \mathcal{O}(L) complexity limits the potential speedup of these methods, e.g., to less than 10\times faster than cycle-level simulation with a single GPU ( Li et al., 2022 ; Pandey et al., 2024 ) . This speedup is mainly due to replacing the irregular computations of cycle-level simulation with accelerator-friendly neural network calculations ( Pandey et al., 2024 ) . By contrast, analytical models can be several orders of magnitude faster than cycle-level simulation (and current ML approaches) because they fundamentally operate at a higher level of abstraction, i.e., mathematical expressions relating key statistics (e.g., instruction mix, cache behavior, branch misprediction rate, etc.) to performance. Second, existing ML approaches must learn all the dynamics impacting performance from raw instruction-level training data. In many cases, this learning task is unnecessarily complex since it does not exploit the CPU performance modeling problem structure. For example, TAO’s Transformer model ( Pandey et al., 2024 ) must learn the performance impact of register dependencies from per-instruction register information, even though there exist higher-level abstractions (e.g., instruction dependency graphs ( Mendis et al., 2019 ) ) that concisely represent dependency behavior (§ 3.2 ). By ignoring the problem structure, blackbox methods require a significant amount of training data to learn. For example, TAO trains on a dataset of 180 million instructions across four benchmarks and two microarchitectures ( Pandey et al., 2024 ) , with further training required for each new microarchitecture. To address these challenges, we propose a novel approach to performance modeling—compositional analytical-ML fusion—where we decompose the task into multiple lightweight models that work together to progressively achieve high fidelity with low computational complexity. We demonstrate this approach in Concorde , a CPU performance model that uses simple analytical models capturing the first-order effects of individual microarchitectural components, coupled with an ML model that captures complex higher-order effects (e.g., interactions of multiple microarchitectural components). Concorde achieves constant-time \mathcal{O}(1) inference complexity, independent of the length of the instruction stream, while maintaining high accuracy across diverse workloads and microarchitectures. Unlike existing ML methods that operate on instruction sequences, Concorde predicts performance based on a compact set of performance distributions . It trains a lightweight ML model — a shallow multi-layer perceptron (MLP) — to map these performance distributions to the target performance metric. We focus on modeling CPI in this paper as it directly reflects program performance, though in principle our

Read the complete text