Agentic Auto-Scheduling: An Experimental Study of LLM-Guided Loop Optimization

Massinissa Merouani 1 , Islem Kara Bernou 1 , and Riyadh Baghdadi 1 · 2025

Read Agentic Auto-Scheduling: An Experimental Study of LLM-Guided Loop Optimization by Massinissa Merouani 1 , Islem Kara Bernou 1 , and Riyadh Baghdadi 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00592.txt.

Introduction

Massinissa Merouani, Islem Kara Bernou, and Riyadh Baghdadi let language models drive compiler optimization in a closed loop. Tuning loop nests for modern hardware is notoriously hard, and their framework, ComPilot, hands the search to an off-the-shelf model with no task-specific fine-tuning. The model proposes transformations for a loop nest, and the compiler reports back which are legal and whether the code sped up or slowed down.

That concrete feedback guides the model's next proposal, round after round. Across the PolyBench suite, the zero-shot agent achieved geometric mean speedups of well over twofold in a single run, and higher still when taking the best of five runs. It also proved competitive with Pluto, a leading polyhedral optimizer, beating it in many cases. Grounded feedback, the study shows, turns general models into effective optimizers.

Catalog record

CREATORMassinissa Merouani 1 , Islem Kara Bernou 1 , and Riyadh Baghdadi 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-00592

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Agentic Auto-Scheduling: An Experimental Study of LLM-Guided Loop Optimization Massinissa Merouani 1 , Islem Kara Bernou 1 , and Riyadh Baghdadi 1 Abstract Automatic code optimization remains a difficult challenge, particularly for complex loop nests on modern hardware. This paper investigates a novel approach to code optimization where Large Language Models (LLMs) guide the process through a closed-loop interaction with a compiler. We present ComPilot , an experimental framework that leverages off-the-shelf LLMs, without any task-specific fine-tuning, as interactive optimization agents. ComPilot establishes a feedback loop where an LLM proposes transformations for a given loop nest to a compiler. The compiler attempts the transformations, reporting back legality status and measured speedup or slowdown. The LLM utilizes this concrete feedback to iteratively refine its optimization strategy. Our extensive evaluation across the PolyBench benchmark suite demonstrates the effectiveness of this zero-shot approach. ComPilot achieves geometric mean speedups of 2.66x (single run) and 3.54x (best-of-5 runs) over the original code. Furthermore, ComPilot demonstrates competitive performance against the state-of-the-art Pluto polyhedral optimizer, outperforming it in many cases. This experimental study demonstrates that general-purpose LLMs can effectively guide the code optimization process when grounded by compiler feedback, opening promising research directions for agentic AI in code optimization. I Introduction Improving program performance remains a cornerstone of computer systems research and practice, directly impacting energy consumption, cloud computing costs, and the turnaround time for critical scientific and commercial applications. However, achieving optimal performance on modern hardware is notoriously difficult. The complex interplay of multi-level caches, sophisticated instruction pipelines, and diverse parallel execution resources creates a challenging landscape. Consequently, manual performance tuning requires prohibitive effort and expertise given the vastness of the search space of possible code transformations. Yet, automated compiler heuristics frequently struggle to deliver consistent results across today’s diverse applications and hardware. Loop nests are critical performance bottlenecks, particularly in scientific computing, image processing, and machine learning domains. Decades of research have yielded powerful optimization techniques, from compiler heuristics (e.g., in GCC [ 1 ] and LLVM [ 2 ] ) and sophisticated polyhedral methods [ 3 ] (e.g., Pluto [ 4 ] ) that analyze dependencies and orchestrate complex transformations (like tiling, fusion, parallelization), to empirical autotuning frameworks [ 5 , 6 , 7 ] . Despite these advances, consistently achieving optimal performance across diverse applications and hardware remains challenging, motivating the exploration of complementary optimization strategies. Recent advancements in Large Language Models (LLMs) open up intriguing new possibilities. Beyond their well-known text generation capabilities, LLMs exhibit remarkable abilities in understanding complex instructions, reasoning about problems, and even analyzing source code structure. This potential has spurred recent investigations into using LLMs for code optimization tasks [ 8 , 9 , 10 , 11 , 12 , 13 , 14 ] . However, approaches generating optimized code directly [ 12 , 13 , 11 , 14 ] often struggle to guarantee the semantic correctness of complex transformations without costly formal verification or brittle output checking. Alternatively, other works focus on selecting compiler passes or flags [ 8 , 9 , 10 ] . These methods are useful for orchestrating existing compiler heuristics, often at the Intermediate Representation (IR) level. However, they typically lack fine-grained control for composing sequences of high-level source transformations which are often essential for maximizing data locality and parallelism gains. Additionally, prior works often prioritized objectives other than execution speed, such as code size, and necessitated domain-specific LLM fine-tuning. These challenges motivate exploring a more agentic paradigm, where the LLM acts as a proactive decision-maker interacting with its environment. This leads to our central research question: Can off-the-shelf LLMs, grounded by empirical compiler feedback, effectively guide the complex process of loop optimization? Fig. 1: Overview of the ComPilot framework, depicting the interaction between the LLM, the different modules, and the compiler. To explore this question, we introduce ComPilot ( Com piler Pilot ), an experimental framework built around a closed-loop interaction between an LLM and a compiler infrastructure. In ComPilot , the LLM acts as an optimization agent, iteratively proposing sequences of loop transformations (schedules) for a given loop nest. These schedules are then passed to a compiler infrastructure (in our case, the Tiramisu compiler [ 15 ] ) which attempts to apply the transformations, checks their legality based on dependence analysis [ 16 , 17 ] , and generates code. ComPilot incorporates a feedback loop: the outcome of applying a schedule (success, failure type, and measured speedup/slowdown if successful) is reported back to the LLM. This closed-loop interaction allows the LLM to potentially learn from its successes and failures and refine its optimization strategy based on direct empirical evidence from the target machine. This approach leverages LLM capabilities to explore the transformation space while relying on the compiler’s rigor for legality checking and code generation, all without requiring LLM fine-tuning or direct generation of transformed code. This paper presents an experimental study evaluating the efficacy and characteristics of this agentic optimization approach. Our key contributions are: • The design and implementation of ComPilot , a framework enabling LLMs to interactively guide loop optimization using compiler-generated feedback. • A demonstration of its effectiveness on PolyBench [ 18 ] , showing that off-the-shelf LLMs can achieve significant speedups over strong baselines by discovering high-performance sequences of transformations. • A detailed experimental study covering performance scaling, cost, variations across different LLMs, and ablation studies justifying the agentic design. Our evaluation demonstrates the potential of LLM-guided optimization. Using a best-of-5 runs strategy, our approach achieves a geometric mean speedup of 3.54 \times over the original code and 2.94 \times over the state-of-the-art Pluto polyhedral compiler across the PolyBench suite [ 18 ] . On certain benchmarks, ComPilot discovers schedules yielding speedups exceeding 100 \times compared to the original code. II System Design and Methodology We implement the idea of LLM-assisted loop optimization following the methodology described in Figure 1 . We will first provide a high-level overview and then delve into the specifics of each component and the reasoning behind our design choices later in this section. At the core of our system, ComPilot facilitates an interaction where the LLM acts as an agent within the compiler environment. For any given input program, this interaction is structured as an optimization dialogue—a dynamic conversation thread that serves two key functions in this agentic model: it is the interface for the agent’s perception and action, and its history serves as the agent’s episodic memory. This memory enables the agent to adapt its optimization strategy based on the concrete outcomes of past actions. This dialogue proceeds in iterations until specific stopping criteria are met. In each back-and-forth exchange, the LLM proposes loop transformations, and the compiler provides feedback on their validity, legality, and performance impact. The process is composed of two main phases: the context initialization phase and the iterative optimization phase . During the context initialization phase , managed by the context initializer module , the system briefs the LLM on its task and explains the overall flow of the optimization process through a context prompt (or a system instruction). The context initializer module then extracts the relevant loop nest from the input program, presenting it to the LLM as a C/C++ style loop nest in a

Read the complete text