HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learning CCS: Hardware High-level and register-transfer level synthesis
Qingyun Zou, Feng Yu, Hongshi Tan, Yao Chen, Bingsheng He, Weng-fai Wong · 2026
Read HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learning CCS: Hardware High-level and register-transfer level synthesis by Qingyun Zou, Feng Yu, Hongshi Tan, Yao Chen, Bingsheng He, Weng-fai Wong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13536.txt.
Introduction
Qingyun Zou and five colleagues propose HLS-Seek, a quality-aware framework for synthesis code writing from natural language. High-level synthesis turns C and C++ into hardware, with latency and costs set by pragmas. Current training ignores quality of results. Reward-based learning needs only paired rankings. It avoids full synthesis training through a proxy reward with 99.53 percent Pareto score. Monte Carlo dropout switching invokes real builds for unsure cases. HLS-Seek reaches 84.7 percent syntax and 81.4 percent functional correctness, beating GPT-5.1.
Catalog record
| CREATOR | Qingyun Zou, Feng Yu, Hongshi Tan, Yao Chen, Bingsheng He, Weng-fai Wong |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43410 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13536
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learning CCS: Hardware High-level and register-transfer level synthesis Qingyun Zou, Feng Yu, Hongshi Tan, Yao Chen, Bingsheng He, Weng-fai Wong Abstract . High-Level Synthesis (HLS) compiles algorithmic C/C++ descriptions into hardware, with Quality of Results (QoR)—latency and resource utilization—critically governed by pragma configurations and code structure. Existing LLM-based HLS approaches train for functional correctness but ignore QoR entirely. We observe that reinforcement learning (RL) for HLS does not require absolute synthesis results—only relative comparisons between candidates. Based on this insight, we propose HLS-Seek , a QoR-aware NL-to-HLS framework that replaces expensive synthesis-in-the-loop RL with a comparative proxy reward model achieving 99.53% Pareto-dominance accuracy. To prevent reward hacking, we introduce uncertainty-aware Monte Carlo (MC) dropout switching that selectively invokes real Vitis HLS synthesis for low-confidence candidates and online updates the proxy, creating a self-improving reward system. HLS-Seek achieves 81.5% syntax correctness pass@1 and 81.4% Func@5 on HLS-eval with only 7B parameters, surpassing GPT-5.1 and other frontier models while achieving 8.5 × \times faster training than real-reward RL. On QoR evaluation, HLS-Seek achieves the lowest latency on 16/30 kernels and Pareto-dominates HLS-specific baselines on 9 kernels. Figure 1 . Three training methods for LLM-based HLS code generation. Comparison of three RL training paradigms for HLS code generation. 1. Introduction High-Level Synthesis (HLS) compiles algorithmic C/C++ descriptions into Register Transfer Level (RTL) hardware implementations, and has become a widely adopted methodology for rapid prototyping across diverse domains, including machine learning accelerators ( 8 ; 28 ; 5 ) , graph processing engines ( 4 ; 7 ; 26 ) , and data-intensive applications ( 18 ; 15 ; 1 ) , on both Field-Programmable Gate Array (FPGA) and Application-Specific Integrated Circuit (ASIC) platforms ( 20 ; 22 ; 14 ) . Recently, there has been growing interest in leveraging large language models (LLMs) to automate HLS code generation from high-level specifications ( 9 ; 16 ; 24 ; 23 ; 21 ) . For example, Gai et al. ( 9 ) fine-tune CodeLlama on HLS benchmarks for functional correctness, and SAGE-HLS ( 16 ) incorporates AST-guided representations to improve code structure awareness, demonstrating that LLMs can generate synthesizable hardware code with increasing reliability. These efforts highlight the potential of LLMs to significantly improve developer productivity and lower the barrier to hardware design. However, these approaches primarily aim to generate runnable kernels, while the Quality of Results (QoR), which is critical for real-world accelerator deployment ( 14 ) , is largely overlooked. QoR reflects low-level hardware performance and efficiency, and is typically measured by execution latency and resource utilization, including LUTs, DSPs, BRAMs, and FFs. It is determined by the microarchitectural structure of the kernel, the pragma configurations and code optimizations applied in the HLS compilation flow, as well as hardware resource instantiation behavior and downstream physical implementation optimizations ( 14 ) . As a result, QoR cannot be accurately inferred from software code analysis alone and instead requires evaluation through the synthesis and implementation. As shown in Figure 1 , existing training paradigms face limitations for QoR-aware HLS generation. Supervised Fine-Tuning (SFT)-based approaches ( 9 ; 16 ) produce functionally correct code but are QoR-agnostic—all compilable designs receive equal training signal regardless of hardware performance. In the RTL domain, ChipSeek-R1 ( 6 ) demonstrates that real synthesis feedback can serve as an effective Reinforcement Learning (RL) reward. However, extending this synthesis-in-the-loop paradigm to HLS is prohibitively expensive: each HLS synthesis run takes minutes to hours (e.g., ThunderGP ( 4 ) ), and RL training requires over 12,500 evaluations, making it infeasible at scale. To overcome this bottleneck, we observe a key insight: RL training does not require absolute QoR values from synthesis—only relative comparisons between candidates. Knowing which of two designs is better suffices for policy improvement via group-relative advantage estimation, without needing exact latency or resource numbers. This motivates us to train a lightweight comparative reward model that predicts, given two candidate HLS designs, which achieves a better resource-latency trade-off—achieving 99.53% dominance accuracy and 99.42% latency accuracy without invoking synthesis. To safeguard against reward hacking as the RL policy explores out-of-distribution pragma configurations, the reward model estimates its own confidence via MC dropout and selectively falls back to real Vitis HLS synthesis for high-uncertainty candidates. Crucially, these synthesis results are fed back to online update the reward model, continuously expanding its coverage. This self-improving mechanism reduces the synthesis trigger rate over the course of training, keeping overall cost close to a proxy-only baseline while maintaining reward fidelity. Based on this reward design, we propose HLS-Seek , a QoR-aware NL-to-HLS code generation framework trained in three stages. To address training data scarcity, we build upon the open-source dataset and apply a novel Pareto-Proximal Quality-Diversity Sampling strategy to curate QoR-competitive HLS implementations from its DSE results. We further leverage commercial reasoning models to generate chain-of-thought reasoning traces , step-by-step rationales for code optimization. Training then proceeds as: ( i ) diversity-oriented SFT on a broad HLS corpus with QoR descriptions, exposing the model to diverse pragma configurations; ( ii ) cold-start SFT on Pareto-proximal data with reasoning traces, instilling hardware reasoning capability before RL; and ( iii ) Group Relative Policy Optimization (GRPO) ( 25 ) with a four-component reward—reasoning format, compilation, functional correctness, and predicted QoR from the comparative reward model. Our main contributions are: • We identify that RL for HLS code generation requires only relative QoR signals, not absolute synthesis results. Based on this insight, we propose a comparative proxy reward model with uncertainty-aware MC dropout switching that provides fast, accurate, and self-improving reward feedback during GRPO training. • We present HLS-Seek 1 1 1 https://anonymous.4open.science/r/HLS_code_generations-057E , a complete QoR-aware NL-to-HLS framework comprising: a three-stage training pipeline (diversity SFT \to cold-start SFT with reasoning traces \to proxy reward GRPO), a Siamese comparative reward model that provides fast pairwise QoR ranking, and an uncertainty-aware MC dropout switching mechanism that online updates the proxy with real synthesis. • HLS-Seek achieves 81.5% syntax correctness pass@1 and 81.4% Func@5 on HLS-eval with only 7B parameters, surpassing GPT-5.1 and other frontier models. On QoR evaluation, HLS-Seek achieves the lowest latency on 16/30 kernels, Pareto-dominates SAGE-HLS on 5 kernels and Gai et al. on 4 kernels with only 1 reverse, and demonstrates algorithmic code restructuring beyond pragma insertion. 2. Background and Motivation Table 1 . Comparison of LLM-Assisted HLS Code Tasks. NL = Natural Language, AST = Abstract Syntax Tree. Method Task Input LLM Strategy Goal HLSPilot ( 27 ) HLS Code Optimization C/HLS-C Prompt Engineering QoR RALAD ( 12 ) HLS Code Optimization HLS-C RAG QoR C2HLSC ( 17 ) HLS Code Optimization C Prompt Engineering Functional, QoR SynthAI ( 24 ) HLS Code Generation Spec Multi-Agent Functional Gai et al. ( 9 ) HLS Code Generation NL Fine-tuning Functional SAGE-HLS ( 16 ) HLS Code Generation NL+AST Fine-tuning Functional HLS-Seek HLS Code Generation NL Fine-tuning Uncertainty-aware proxy reward Functional, QoR As summarized in Table 1 , LLM-based HLS research falls into two categories: HLS code optimization and HLS code generation . HLS Code Optimization. These methods take existing C source code as input and apply LLMs to pragma insertion or design transformation for