Learning When to Optimize: Verified Optimization Skills from Expert GPU-Kernel Lineages

Shuoming Zhang, Qiuchu Yu, Yangyu Zhang, Ruiyuan Xu, Xiyu Shi, Guangli Li, Xiaobing Feng, Huimin Cui, Jiacheng Zhao · 2026

Read Learning When to Optimize: Verified Optimization Skills from Expert GPU-Kernel Lineages by Shuoming Zhang, Qiuchu Yu, Yangyu Zhang, Ruiyuan Xu, Xiyu Shi, Guangli Li, Xiaobing Feng, Huimin Cui, Jiacheng Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28213.txt.

Introduction

Zhang and colleagues teach GPU-kernel agents the missing when of optimization: not just what experts did, but when each trick applies. KLineage walks expert kernels backward through checked simplifications, turning each reversed step into a skill recording intent, conditions, effects, and pitfalls. On five expert workloads it beat memory-based rivals in kernel quality and efficiency.

Catalog record

CREATORShuoming Zhang, Qiuchu Yu, Yangyu Zhang, Ruiyuan Xu, Xiyu Shi, Guangli Li, Xiaobing Feng, Huimin Cui, Jiacheng Zhao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51703
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.28213

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning When to Optimize: Verified Optimization Skills from Expert GPU-Kernel Lineages Shuoming Zhang, Qiuchu Yu, Yangyu Zhang, Ruiyuan Xu, Xiyu Shi, Guangli Li, Xiaobing Feng, Huimin Cui, Jiacheng Zhao Abstract LLM-based agents are increasingly used to generate GPU kernels, but they often know what optimizations to try without knowing when those optimizations are sound. We introduce KLineage , which learns this missing “when” knowledge from expert kernels: instead of relying on forward rollouts, KLineage walks expert implementations backward through validation-gated simplifications and reverses each accepted step into a reusable optimization skill. Each skill records not only the optimization intent, but also where it applies in code, what conditions made it valid, what effect it had, and what failures its assumptions avoid. A downstream LLM materializes these skills on new code surfaces under the same compile/correctness/profile gate. On five expert workloads across two NVIDIA architectures, these lineage-derived skills serve as an effective optimization curriculum, exceeding recent memory-based LLM-kernel baselines in both final kernel quality and optimization efficiency under the same fixed budget. We additionally use a separate 22-instance held-out check as a sanity test against source-case memorization. 1 Introduction LLM-based agents are becoming a practical interface for GPU-kernel generation: they can draft an implementation, run it, inspect compiler or execution feedback, and iterate. Benchmarks such as KernelBench ( Ouyang et al. 2025 ) and TritonBench ( Li et al. 2025b ) have made this setting measurable, while recent agents improve kernels through feedback, search, post-training, or skill memory ( Andrews and Witteveen 2025 ; Wang et al. 2025 ; Lange et al. 2025 ; Du et al. 2026a ; Sun et al. 2026 ; Du et al. 2026b ) . The setting is also increasingly multi-surface: agents may write CUDA, Triton, CuTe/CUTLASS-style templates, TileLang programs, or a restricted kernel DSL, each exposing different scheduling knobs and failure modes ( Li et al. 2025b ; Wang et al. 2025 ; Saba et al. 2026 ; Li et al. 2025a ; Hari et al. 2026 ; Chu et al. 2025 ; Qu et al. 2026 ) . Yet strong optimization from a naive kernel remains unreliable. The problem is often not that the model lacks the names of useful optimizations. It can suggest tiling, shared-memory staging, vectorized loads, software pipelining, or hardware intrinsics. What is missing is the condition under which each optimization is valid and useful. Vectorized loads require alignment and valid tail handling; pipelining assumes staged data movement; matrix intrinsics require compatible tile shapes, layouts, and hardware support. A forward-search agent may therefore apply a familiar optimization to the wrong state, producing a compile error, a correctness failure, or a slower kernel. Pretraining captures much of the what of optimization, but not reliably the when . We treat this missing “when” knowledge as the object to be learned. The memory item we need is not a free-form hint such as “use vectorized loads”, but a skill in a narrow sense: a verified, code-anchored specification of an optimization intent, including where the intent acts on a code or IR surface, what carrier can guide implementation, the context in which it has been observed to work, the effect it tends to have, and the risks when its assumptions are violated. Existing kernel-agent memories expose parts of this signal through failure rules, profile-state knowledge bases, or expert-skill snippets ( Du et al. 2026b ; Cao et al. 2026 ; Dong et al. 2026 ; Sun et al. 2026 ) , while post-training approaches may absorb it into model weights ( Du et al. 2026a ; Baronio et al. 2025 ; Li et al. 2025d ) . These memories are useful, but when they are built from forward rollouts, their coverage is limited by the trajectories the agent was able to discover. Our goal is to externalize the missing “when” knowledge from expert artifacts instead. We introduce KLineage , a framework for inducing reusable skills from expert GPU-kernel lineages. The key observation is that expert kernels already contain the missing conditional knowledge, but only implicitly. A high-performance implementation reflects a sequence of compatible decisions about tiling, staging, vectorization, parameter selection, and hardware features. KLineage recovers these decisions by walking an expert kernel backward through named, validation-gated deoptimization steps. Each accepted simplification is stored as its inverse forward transition, yielding a curriculum from a naive implementation toward the expert kernel; the forward transitions observed across experts become evidence for reusable skills. To our knowledge, KLineage is the first LLM-kernel memory constructed by deoptimizing expert implementations rather than accumulating forward-rollout states. At reuse time, KLineage retrieves skills by scope. Given a new target, it retrieves skills along the target case, surface language, platform, and already-applied optimizations. The target state need not be naive: KLineage can start from any current kernel attempt and use the retrieved lineage to complete missing intents or repair mis-materialized ones. It then lets a downstream LLM materialize selected skills on the target code surface using their anchors and carriers. This keeps the LLM in the role where it is useful—instantiating a validated code-anchored intent and repairing local mismatches under feedback—while the memory constrains it with validated preconditions and observed risks. We evaluate KLineage on five expert workloads across two NVIDIA architectures (H100/SM90 and RTX PRO 6000/SM120), plus 22 held-out FlagGems Triton-bound operators that did not contribute lineages. Under a fixed per-workload $ 10 optimization-cost budget on the same backbone, the main result is 1.12{\times} success-only geometric mean over vendor references on 10/10 main-tier pairs, against 0.25{\times} on 6/10 for AdaExplore ( Du et al. 2026b ) and 0.54{\times} on 8/10 for AccelOpt ( Zhang et al. 2025 ) after filtering non-kernel wrappers; the AdaExplore count includes one post-hoc adapter audit that exposes a real but slow Conv2d kernel. The same library transfers to TileLang at 1.06{\times} . Two ablations (roundtrip recovery: 41/50 ; generated-only: 5{-}14{\times} slowdown on Conv2d, 105{\times} on GDN) attribute the gain to the lineage’s conditional structure rather than labels alone, and a cost-trajectory analysis shows KLineage saturates well under the cap where neither baseline does. The held-out FlagGems check (App. D ) confirms the induced library is not memorized to the five experts. Contributions. This paper introduces KLineage and makes three technical contributions: • Conditioned procedural skills. We formulate reusable GPU-kernel optimization knowledge as skills : verified, code-anchored specifications of optimization intents with anchors, carriers, preconditions, effects, risks, evidence, and scope over case, surface language, platform, and prior actions, making both the “when” knowledge and the code surface explicit. • Guided deoptimization. We propose a validation-gated procedure for inducing such skills from expert kernels: walking the kernel backward through dependency-aware deoptimization steps and storing each accepted simplification as its inverse forward transition, yielding an executable curriculum from naive to expert. • Verification-gated materialization and reuse. We introduce a reuse mechanism in which a candidate skill enters the library only after a held-out roundtrip admits it, and on a new target the agent materializes retrieved skills under the same compile/correctness/profile gate used for all submissions. 2 Related Work Figure 1: Motivation of KLineage . (a) Forward search often knows which optimizations to try but not when their preconditions hold. (b) KLineage walks expert kernels backward to recover validated forward transitions from

Read the complete text