CodeV-R1: Reasoning-Enhanced Verilog Generation
Yaoyu Zhu, Di Huang, Hanqi Lyu, Xiaoyun Zhang, Chongxiao Li, Wenxuan Shi, Yutong Wu, Jianan Mu, Jinghua Wang, Yang Zhao, Pengwei Jin, Shuyao Cheng, Shengwen Liang, Xishan Zhang, Rui Zhang, Zidong Du, Qi Guo, Xing Hu✉, Yunji Chen✉ · 2025
Read CodeV-R1: Reasoning-Enhanced Verilog Generation by Yaoyu Zhu, Di Huang, Hanqi Lyu, Xiaoyun Zhang, Chongxiao Li, Wenxuan Shi, Yutong Wu, Jianan Mu, Jinghua Wang, Yang Zhao, Pengwei Jin, Shuyao Cheng, Shengwen Liang, Xishan Zhang, Rui Zhang, Zidong Du, Qi Guo, Xing Hu✉, Yunji Chen✉ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24183.txt.
Introduction
Yaoyu Zhu and a large team of co-authors present CodeV-R1 in two thousand twenty-five. CodeV-R1 trains language models to write Verilog, a hardware design language. Learning with checked rewards works well for code and math. Hardware design lacks auto checks, good paired data, and cheap training.
The team built a rule based test bench that checks code against gold designs. A round trip method makes paired data and filters bad pairs. Training first distills reasoning, then runs a low cost reward phase. The small end model beat past best scores on two known tests. It even topped a far larger rival on one of them.
Catalog record
| CREATOR | Yaoyu Zhu, Di Huang, Hanqi Lyu, Xiaoyun Zhang, Chongxiao Li, Wenxuan Shi, Yutong Wu, Jianan Mu, Jinghua Wang, Yang Zhao, Pengwei Jin, Shuyao Cheng, Shengwen Liang, Xishan Zhang, Rui Zhang, Zidong Du, Qi Guo, Xing Hu✉, Yunji Chen✉ |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-24183
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CodeV-R1: Reasoning-Enhanced Verilog Generation Yaoyu Zhu, Di Huang, Hanqi Lyu, Xiaoyun Zhang, Chongxiao Li, Wenxuan Shi, Yutong Wu, Jianan Mu, Jinghua Wang, Yang Zhao, Pengwei Jin, Shuyao Cheng, Shengwen Liang, Xishan Zhang, Rui Zhang, Zidong Du, Qi Guo, Xing Hu✉, Yunji Chen✉ Abstract Large language models (LLMs) trained via reinforcement learning with verifiable reward (RLVR) have achieved breakthroughs on tasks with explicit, automatable verification, such as software programming and mathematical problems. Extending RLVR to electronic design automation (EDA), especially automatically generating hardware description languages (HDLs) like Verilog from natural-language (NL) specifications, however, poses three key challenges: the lack of automated and accurate verification environments, the scarcity of high-quality NL–code pairs, and the prohibitive computation cost of RLVR. To this end, we introduce CodeV-R1, an RLVR framework for training Verilog generation LLMs. First, we develop a rule-based testbench generator that performs robust equivalence checking against golden references. Second, we propose a round-trip data synthesis method that pairs open-source Verilog snippets with LLM-generated NL descriptions, verifies code–NL–code consistency via the generated testbench, and filters out inequivalent examples to yield a high-quality dataset. Third, we employ a two-stage “distill-then-RL” training pipeline: distillation for the cold start of reasoning abilities, followed by adaptive DAPO, our novel RLVR algorithm that can reduce training cost by adaptively adjusting sampling rate. The resulting model, CodeV-R1-7B, achieves 68.6 % and 72.9 % pass@1 on VerilogEval v2 and RTLLM v1.1, respectively, surpassing prior state-of-the-art by 12 ∼ \sim 20 %, while matching or even exceeding the performance of 671B DeepSeek-R1. We will release our model, training pipeline, and dataset to facilitate research in EDA and LLM communities. \NewTableCommand \seprule \NewTableCommand \uniquerule 1 Introduction Large language models (LLMs) have recently demonstrated remarkable progress on reasoning tasks when trained via reinforcement learning with verifiable reward (RLVR). Notable examples include OpenAI-o1 [ 25 ] and DeepSeek-R1 [ 3 ] , which exhibit emergent reasoning capabilities on problems endowed with explicit verification procedures—such as software programming and mathematical problem solving. This success suggests a promising opportunity to apply RLVR within electronic design automation (EDA), specifically to the automatic generation of hardware description languages (HDLs) like Verilog from natural-language (NL) specifications [ 36 ] . However, the three foundational components required for effective RLVR — (i) a reliable verification environment, (ii) high-quality NL-code data, and (iii) an efficient training algorithm — each present significant challenges in training reasoning LLMs for Verilog generation: (1) Automated verification of hardware designs remains difficult. RLVR requires a verification environment capable of providing accurate rewards. However, even in the data-rich software coding domain, such environments are rare. For example, most problems in the programming-contest dataset APPS [ 6 ] have only one or two sets of unit tests, and they exhibit a false-positive rate of up to 60% when evaluated with an average of 20 unit tests [ 12 ] . Consequently, the software community has adopted the practice of using LLMs to generate additional unit tests in order to improve verification quality [ 39 , 11 ] . Nevertheless, this approach is both costly and of limited effectiveness for hardware designs, because LLMs lack the hardware-specific knowledge needed to handle the complex state spaces and corner cases of sequential circuits. For example, if the reset and clock signals are not correctly configured, the intended functionality cannot be properly verified. (2) High-quality NL–code pairs for hardware designs are scarce. The proprietary nature of hardware designs severely limits the availability of annotated Verilog examples. Although several LLM-based methods have been proposed to synthesize NL–code pairs [ 43 , 46 , 16 , 5 ] , the resulting datasets often suffer from low-quality data (see Appendix for examples), rendering them inadequate for RLVR’s stringent requirements. (3) The computational cost of RLVR is prohibitive. Training a 32B LLM on 1K data for 5 epochs using 16 NVIDIA H100 GPUs with supervised fine-tuning (SFT) takes only 0.5 hours [ 23 ] . In contrast, training a 14B LLM on 24K verifiable coding problems with reinforcement learning can take over 2.5 weeks on 32 NVIDIA H100 GPUs [ 20 ] , making it prohibitively expensive to train a Verilog reasoning LLM using RLVR. To overcome these challenges, we introduce CodeV-R1, a comprehensive RLVR framework for Verilog generation. Our contributions are threefold: (1) Automated testbench generation. We develop a rule - based testbench generation framework to verify the equivalence between a given Verilog implementation and its golden reference as accurately as possible. For each golden reference, the framework first performs circuit - structure analysis to extract information such as input/output (I/O) ports and reset/clock signals. It then enumerates all reset and clock-synchronization scenarios to improve verification accuracy. Experiments demonstrate that our testbench achieves 35.5 % fewer false negatives than the LLM-generated counterpart and detects 62.5 % more injected errors in fuzzing tests for sequential circuits. Detailed experimental results are presented in Section 3.3.4 . (2) Round-trip data synthesis for high-quality NL–code pairs. Leveraging our testbench generation framework, we propose the round-trip data synthesis approach that can automatically synthesize high-quality NL–code pairs from code snippets. Specifically, candidate code snippets are first paired with LLM-generated NL descriptions, and then verified by regenerating the code from the NL and comparing against the original for equivalence with our testbench. Only code that passes the testbench is retained and combined with the NL to form high-quality data. We theoretically prove that, given strong LLMs and an ideal verification environment, this procedure yields NL–code pairs of sufficiently high quality for RLVR with a high probability. (3) Two-stage training with adaptive DAPO for cost-effective RLVR. We adopt a two-stage “distill-then-RL” training pipeline to cold start LLMs’ reasoning ability through SFT and reduce the overall training cost. Specifically, we use DeepSeek-R1 as the NL-to-code LLM in our round-trip data synthesis to produce (NL, Thought, Code) triplets without equivalence checking, based on which we perform SFT on our base LLM to obtain a distilled LLM with basic reasoning ability. Then, we apply RLVR on the distilled LLM using the equivalence-checked high-quality data to further enhance its Verilog generation capability. Additionally, recognizing that RLVR’s bottleneck lies in sampling and verification [ 20 ] , we extend dynamic sampling policy optimization (DAPO) [ 42 ] with an adaptive mechanism that dynamically adjusts the number of samples per training step based on past sample discard rates. This approach significantly reduces unnecessary sampling and verification overhead, thereby achieving a 1.25x acceleration. Based on these techniques, we develop CodeV-R1-7B, the first specialized reasoning LLM for Verilog generation with only around 2,656 A100-GPU-hours. On the VerilogEval v2 [ 27 ] and RTLLM v1.1 / v2 [ 19 ] benchmarks, CodeV-R1-7B achieves 68.8% pass@1 and 72.9% / 68.0% pass@1, respectively. Remarkably, it surpasses the 671B DeepSeek-R1 by 8.1% on RTLLM v1.1 and 3.3% on RTLLM v2, demonstrating its strong RTL generation capabilities. We will release the model, training pipeline, and dataset to foster further research in this domain. 2 Methods Figure 1 : The overview of CodeV-R1. The core components of our framework include an automated testbench (Section 2.1 ), a supervised fine-tuning process (Section 2.2 ), and a reinforcement learning process (Section 2.3 ). Our framework comprises one automated testbench generation framework and 5 stages (Figure 1 ). Stages 1