LLMulator: Generalizable Cost Modeling for Dataflow Accelerators with Input-Adaptive Control Flow Conference: 58th IEEE/ACM International Symposium on Microarchitecture; October 18–22, 2025; Seoul, Republic of Korea 58th IEEE/ACM International Symposium on Microarchitecture (MICRO ’25), October 18–2
Kaiyan Chang, Wenlong Zhu, Shengwen Liang, Huawei Li, Ying Wang · 2025
Read LLMulator: Generalizable Cost Modeling for Dataflow Accelerators with Input-Adaptive Control Flow Conference: 58th IEEE/ACM International Symposium on Microarchitecture; October 18–22, 2025; Seoul, Republic of Korea 58th IEEE/ACM International Symposium on Microarchitecture (MICRO ’25), October 18–2 by Kaiyan Chang, Wenlong Zhu, Shengwen Liang, Huawei Li, Ying Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-17826.txt.
Introduction
Kaiyan Chang and four co-authors used large language models to predict how fast a dataflow accelerator chip will run, in a 2025 paper. Designers need quick, accurate cost guesses to explore chip layouts, but older predictors fail when the hardware, the program, or the input shape changes. Input-dependent control flow makes the guessing harder still.
Their system, LLMulator, treats performance numbers as token sequences, which lets it estimate values at any scale and state its confidence. Program meaning drawn from a pre-trained model keeps the guesses aware of both hardware and task. A reinforcement step tunes the model to each input's control flow, cutting cycle error by about ten percent. A staged data plan broadens coverage across layouts, which lifts accuracy on unseen hardware.
Catalog record
| CREATOR | Kaiyan Chang, Wenlong Zhu, Shengwen Liang, Huawei Li, Ying Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-17826
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLMulator: Generalizable Cost Modeling for Dataflow Accelerators with Input-Adaptive Control Flow Conference: 58th IEEE/ACM International Symposium on Microarchitecture; October 18–22, 2025; Seoul, Republic of Korea 58th IEEE/ACM International Symposium on Microarchitecture (MICRO ’25), October 18–22, 2025, Seoul, Republic of Korea DOI: 10.1145/3725843.3756027 ISBN: 979-8-4007-1573-0/2025/10 CCS: Computer systems organization Data flow architectures Kaiyan Chang, Wenlong Zhu, Shengwen Liang, Huawei Li, Ying Wang Abstract . Precise and rapid performance prediction for dataflow-based accelerators is essential for efficient hardware design and design space exploration. However, existing methods often fall short due to limited generalization across hardware architectures, applications, and input-dependent control flows. Considering the rich program semantic knowledge contained in pre-trained large language models (LLMs), which is used for text and code generation, we propose a progressive numeric modeling paradigm based on pre-trained LLMs. This is an approach to achieve hardware, application, and control flow-sensitive generalization in dataflow accelerator performance prediction. Specifically, to make accurate performance estimates for unseen applications beyond the scope of the training data, we propose a numeric prediction model capable of estimating any performance range. This is achieved by treating the numerical data of the dataflow program as separate tokens and using categorical output for performance values, allowing us to observe confidence at each numerical position. Second, LLMulator supports input-adaptive performance prediction by introducing a reinforcement learning-based dynamic calibration framework, enabling accurate modeling of applications whose control flow varies with input—unlike prior methods that assume fixed execution paths. The cycles prediction error converges to within 11.2% after several iterations, reducing the error by 9.7% over static models. Finally, to generalize across diverse hardware architectures and configurations, we introduce a progressive data augmentation framework that systematically generates rich datasets spanning software and hardware variations. This includes multi-level dataflow structures and a structured representation of memory parameters and loop mapping primitives, significantly improving the performance of LLMulator for diverse hardware configurations. Experiments show that LLMulator achieves an average absolute percentage error (MAPE) of 12.2% in ASIC design scenarios, reducing errors by 16.7%-7.8% compared to existing SOTA methods (TLP, GNNHLS). These results establish LLMulator as a highly accurate and interpretable framework for performance prediction, showcasing new opportunities for applying pre-trained LLMs to hardware modeling. † † cc-license: by 1. Introduction Figure 1. Motivation. LLMulator is an end-to-end cost modeling approach without parser and it produces the result with confidence. WWW denotes the World Wide Web. The rapid evolution of heterogeneous computing architectures has positioned dataflow accelerators ( e.g., TPUs, NPUs) ( Shivdikar et al. 2024 ; Jouppi et al. 2017 ; Moon et al. 2021 ) as critical infrastructure for key workloads such as deep learning and scientific simulations. Accurate performance prediction for dataflow-based accelerators is essential in early design stage, enabling designers and design space exploration tools to quickly assess and obtain insights into potential design candidates ( Li et al. 2022b ; Huang et al. 2021 ; Bi et al. 2023 ; Chen et al. 2018b ; Xing et al. 2022 ; Jung et al. 2021 ; Gui et al. 2022 ; Xia et al. 2024 ; Cummins et al. 2021 ; Wang et al. 2024 ; Yang et al. 2020 ; Vasilache et al. 2018 ; Baghdadi et al. 2019 ; Zhao et al. 2021 ) . Current performance prediction methods broadly fall into three categories: (1) rule-based simulators, (2) graph neural network (GNN)-based methods, and (3) language model regression techniques, as summarized in Table 1 . Rule-based simulators ( e.g., Timeloop ( Parashar et al. 2019 ) ) rely on manually defined models of hardware behavior. However, their coverage is limited by predefined rules, restricting their ability to generalize to novel or dynamic operators ( e.g., dynamic sparse convolutions) and complex control flow scenarios ( Yu et al. 2018 ) ( e.g., conditional branches). Graph neural network ( GNN ( Fey and Lenssen 2019 ; Hamilton et al. 2017 ) )- based methods ( e.g., GNNHLS ( Wu et al. 2022 ; Cummins et al. 2021 ) ) compile dataflow programs into graph representations to leverage structural information during inference ( Cummins et al. 2021 ) . However, they primarily address static graph structures and do not effectively incorporate runtime-sensitive features such as dynamic tensor sizes or loop bounds. Consequently, they yield high prediction errors—up to 27.7%—in scenarios involving dynamic operator behaviors. Language model regression methods ( e.g., TLP ( Zhai et al. 2023 ) ) directly map program text to performance metrics. These data-driven models, which learn from real-world data, offer greater accuracy and flexibility ( Zheng et al. 2020b ; Zheng et al. 2020a ; Baghdadi et al. 2021 ; Wu et al. 2022 ; Makrani et al. 2019 ; Lin et al. 2020 ; Zhou et al. 2019 ; Xie et al. 2021 ; Li et al. 2022a ) , which can be represented as a unified workflow in the brown area of Figure 1 , consisting of two components: the parser and the regression model. However, their prediction accuracy dramatically deteriorates when encountering large-scale loops or edge cases that exceed the scope of their training data, resulting in errors up to threefold higher in extreme cases. Collectively, these methods suffer from critical generalization issues across three dimensions: application diversity, input adaptivity, and hardware variability. Table 1. Dataflow Level Cost Modeling Approach Comparison. Rule-Based GNN-based LM Regression Representative Tools Timeloop ( Parashar et al. 2019 ) , Maestro ( Kwon et al. 2019 ) , Tenet ( Lu et al. 2021 ) GNNHLS ( Wu et al. 2022 ; Cummins et al. 2021 ) TLP ( Zhai et al. 2023 ) Technical Principle Manually modeled hardware performance predicted through mathematical formulas Compiling dataflow programs into graph structures for inference Mapping program features to performance metrics Applicable Scenarios Specific scenarios ( e.g., DNNs) Static dataflow Fixed operator scheduling Disadvantages 1. Cannot generalize to new operators. 2. Reduced edge accuracy due to model generalization 1. Cannot generalize to dynamic input and dataflow loop extensive applications. 2. Reduced edge accuracy due to model generalization 1. Cannot generalize to new hardware. 2. Cannot generalize to input. 3. Reduced edge accuracy due to regression mode. Typical Error Range Not explicitly quantified, but accuracy is limited by manual rules. Errors in dynamic operator scenarios up to 50%. Extreme latency errors are three times higher than those in the central region. Application Generalization. Rule-based simulators ( e.g., Timeloop) inherently restrict their applicability to predefined operator structures, often limited to perfectly nested loops without complex control flows ( e.g., perfect nested loops without control flow). For example, the ADI application in Polybench cannot be described by Timeloop ( Grauer-Gray et al. 2012 ) . Meanwhile, regression-based methods such as TLP are bounded by normalized predictions within training ranges, causing significant errors (MAPE exceeding 40%) in extreme loop-scale scenarios. As a result, the cost model often struggles to accurately predict performance for dataflow applications with operators and task graphs that fall outside its training scope. Input Generalization. Existing cost models ( e.g., Timeloop, TLP) typically ignore input-dependent variations, although inputs often critically influence program performance, particularly through control-flow impacts ( Eeckhout et al. 2004 ; Krogmann et al. 2010 ; Josipović et al. 2021 ) . Even slight mispredictions in conditional branch activations can