Can Large Language Models Predict Parallel Code Performance? CCS: Computing methodologies Natural language generation CCS: Computing methodologies Machine translation Conference: arXiv Submission; Apr 22, 2025; Virginia Tech, USA arXiv ’25, Apr 22, 2025, Virginia Tech, VA, USA
Gregory Bolet, Giorgis Georgakoudis, Harshitha Menon, Konstantinos Parasyris, Niranjan Hasabnis, Hayden Estes, Kirk W. Cameron, Gal Oren · 2025
Read Can Large Language Models Predict Parallel Code Performance? CCS: Computing methodologies Natural language generation CCS: Computing methodologies Machine translation Conference: arXiv Submission; Apr 22, 2025; Virginia Tech, USA arXiv ’25, Apr 22, 2025, Virginia Tech, VA, USA by Gregory Bolet, Giorgis Georgakoudis, Harshitha Menon, Konstantinos Parasyris, Niranjan Hasabnis, Hayden Estes, Kirk W. Cameron, Gal Oren (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-03988.txt.
Introduction
Gregory Bolet and seven co-authors test if large language models can foretell, from code alone, whether a graphics-chip program is bound by compute or by data flow. Their paper is from two thousand twenty-five.
To know a program's speed, one most often runs it on the true chip. Top chips are scarce, so that step costs more and more. The team frames the ask as a roofline sort: given the code and the chip's facts, which bound holds? They build a fair set of hundreds of such programs from a known bench. The work tests if a model can stand in for a costly run on real gear.
Catalog record
| CREATOR | Gregory Bolet, Giorgis Georgakoudis, Harshitha Menon, Konstantinos Parasyris, Niranjan Hasabnis, Hayden Estes, Kirk W. Cameron, Gal Oren |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-03988
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can Large Language Models Predict Parallel Code Performance? CCS: Computing methodologies Natural language generation CCS: Computing methodologies Machine translation Conference: arXiv Submission; Apr 22, 2025; Virginia Tech, USA arXiv ’25, Apr 22, 2025, Virginia Tech, VA, USA Gregory Bolet, Giorgis Georgakoudis, Harshitha Menon, Konstantinos Parasyris, Niranjan Hasabnis, Hayden Estes, Kirk W. Cameron, Gal Oren Abstract . Accurate determination of the performance of parallel GPU code typically requires execution-time profiling on target hardware – an increasingly prohibitive step due to limited access to high-end GPUs. This paper explores whether Large Language Models (LLMs) can offer an alternative approach for GPU performance prediction without relying on hardware. We frame the problem as a roofline classification task: given the source code of a GPU kernel and the hardware specifications of a target GPU, can an LLM predict whether the GPU kernel is compute-bound or bandwidth-bound? For this study, we build a balanced dataset of 340 GPU kernels, obtained from HeCBench benchmark and written in CUDA and OpenMP, along with their ground-truth labels obtained via empirical GPU profiling. We evaluate LLMs across four scenarios: (1) with access to profiling data of the kernel source, (2) zero-shot with source code only, (3) few-shot with code and label pairs, and (4) fine-tuned on a small custom dataset. Our results show that state-of-the-art LLMs have a strong understanding of the Roofline model, achieving 100% classification accuracy when provided with explicit profiling data. We also find that reasoning-capable LLMs significantly outperform standard LLMs in zero- and few-shot settings, achieving up to 64% classification accuracy of GPU source codes, without any profiling information. Lastly, we find that model accuracy does not benefit meaningfully from few-shot prompting compared to zero-shot, and that LLM fine-tuning will require much more data than what we currently have available. This work is among the first to use LLMs for source-level roofline performance prediction via classification, and illustrates their potential to guide optimization efforts when runtime profiling is infeasible. Our findings suggest that with better datasets and prompt strategies, LLMs could become practical tools for HPC performance analysis and performance portability. Code and datasets are publicly available at https://github.com/Scientific-Computing-Lab/ParallelCodeEstimation . 1. Introduction High-Performance Computing (HPC) applications increasingly rely on GPUs for acceleration, yet ensuring performance portability across diverse GPU architectures has become a pressing challenge ( Yang et al. 2020 ; Trott et al. 2022 ; Beckingsale et al. 2019 ) . Modern supercomputers feature a mix of GPUs from different vendors, and codes optimized for one platform may not achieve optimal performance on another. Identifying performance bottlenecks, whether a kernel is limited by computation or by memory throughput, is crucial to guide optimizations. Traditionally, this determination requires running detailed profiles on target hardware ( Laksono Adhianto et al. 2009 ; José Morgado et al. 2024 ; Le Chen et al. 2024 ) , but access to cutting-edge GPUs is often limited and expensive ( Cohan 2024 ; Seonho Lee et al. 2024 ) . There is a growing need for hardware-free performance prediction methods that can provide insights without extensive benchmarking on physical devices. This work addresses that need by exploring a novel, static approach to classify GPU kernel performance. A well-established framework for reasoning about code performance is the Roofline model ( Williams et al. 2009 ) . The Roofline model correlates a kernel’s arithmetic intensity ( AI ) (operations per byte of memory traffic) with hardware peak performance (operations per second) to determine a performance ceiling. Kernels with low AI tend to be Bandwidth-Bound ( BB ) (limited by memory bandwidth), while those with high AI are Compute-Bound ( CB ) (limited by the processor’s peak FLOPs) ( Yang et al. 2018 ) . By plotting a kernel’s AI against achievable performance, one can visualize whether performance is capped by bandwidth or compute limits. This understanding of AI and performance relative to a target hardware roofline can then be used to intuitively guide kernel optimization decisions to reach peak system performance ( Yu Jung Lo et al. 2014 ; Yang et al. 2020 ) . However, obtaining a kernel’s AI and achieved performance usually entails profiling – measuring runtime operations and memory transfers on the actual GPU. Similarly, previous GPU performance modeling works also depend on performance counter metrics ( Lorenz Braun et al. 2020 ; Arunavo Dey et al. 2024 ) . This requirement hampers rapid performance analysis, especially when developers lack access to the target hardware. Static analysis works ( Gargi Alavani et al. 2018 ; Guerreiro et al. 2019 ; Fan et al. 2019 ; Alavani et al. 2020 ) have shown potential for GPU power and execution time prediction, however, these still require hardware access for micro-benchmarking. Large Language Models (LLMs) offer a fresh opportunity to address this problem of requiring hardware access and runtime information to predict performance. Recent advances in code-focused LLMs have demonstrated remarkable capabilities in understanding and generating code. These models can perform tasks such as summarization, translation, bug fixing, and optimization ( Chen and et al. 2021 ; Xu and et al. 2022 ; Bin Lei et al. 2023 ) . In HPC contexts, early studies have begun applying LLMs to assist in automatic parallelization ( Wang and et al. 2023 ) , GPU kernel execution time prediction ( Minh-Khoi Nguyen-Nhat et al. 2024 ) , performance slowdown prediction ( Daniel Nichols et al. 2023 ) , and performance optimization ( Bowen Cui et al. 2025 ; Daniel Nichols et al. 2024 ) . These developments suggest that LLMs not only understand code syntax but may also capture deeper semantic features and performance-affecting structures. The advantage of the Roofline model is that it can intuitively guide developers – and thus, LLMs, towards an optimal program. Our work is the beginning of an effort that seeks to leverage this nature of the Roofline model to guide LLMs towards delivering performant code. However, integration of modern LLMs into Roofline-based performance analysis remains unexplored. This work seeks to establish a base understanding of how well LLMs recall and use the Roofline model to predict program performance. We build upon the trends of: Roofline modeling, static prediction, and LLM-driven code understanding – by framing GPU performance prediction as a binary classification task. We set our sights on a simple sub-problem within Roofline modeling: classifying a program as BB or CB based on its AI. We first assess how well LLMs can classify kernels when given known AI values; expecting the LLMs to do simple calculations and reasoning to arrive at an answer. We then profile CUDA-based and OpenMP-based GPU kernels from the HeCBench benchmark suite ( Jin 2024 ; Jin and Vetter 2023 ) and obtain their ground-truth AI classification labels. Given a GPU kernel, we have LLMs predict whether it is BB or CB using only the source code and hardware/execution specs. By comparing LLM predictions to our ground-truth labels, we assess whether LLMs can intuit about AI, and if they can serve as practical tools for source-code-level performance classification. From this point forward, we refer to this task of classifying the AI of a program (to be BB or CB) as the roofline classification task . Research Questions. To study how LLMs can be applied