HDLCoRe: A Training-Free Framework for Mitigating Hallucinations in LLM-Generated HDL
Heng Ping, Shixuan Li, Peiyu Zhang, Anzhe Cheng, Shukai Duan, Nikos Kanakaris, Xiongye Xiao, Wei Yang, Shahin Nazarian, Andrei Irimia, Paul Bogdan · 2025
Read HDLCoRe: A Training-Free Framework for Mitigating Hallucinations in LLM-Generated HDL by Heng Ping, Shixuan Li, Peiyu Zhang, Anzhe Cheng, Shukai Duan, Nikos Kanakaris, Xiongye Xiao, Wei Yang, Shahin Nazarian, Andrei Irimia, Paul Bogdan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-16528.txt.
Introduction
Language models asked to write chip code often invent parts that do not exist, because hardware languages starve them of examples. Heng Ping and co-authors built a fix that needs no retraining. Their framework pairs a hardware-aware reasoning prompt with self-checking steps. Tasks are sorted by type and difficulty before solving. A two-stage retrieval system then fetches matching examples, cleaned of format quirks.
On a standard hardware benchmark the framework beat rivals in both syntax and function. Self-simulation inside the prompt catches errors before output. Retrieval quality, often the weak link, gets its own filtering and re-ranking stages. The work shows prompt craft plus good examples can substitute for scarce training data.
Catalog record
| CREATOR | Heng Ping, Shixuan Li, Peiyu Zhang, Anzhe Cheng, Shukai Duan, Nikos Kanakaris, Xiongye Xiao, Wei Yang, Shahin Nazarian, Andrei Irimia, Paul Bogdan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-16528
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HDLCoRe: A Training-Free Framework for Mitigating Hallucinations in LLM-Generated HDL Heng Ping, Shixuan Li, Peiyu Zhang, Anzhe Cheng, Shukai Duan, Nikos Kanakaris, Xiongye Xiao, Wei Yang, Shahin Nazarian, Andrei Irimia, Paul Bogdan Abstract Recent advances in large language models (LLMs) have demonstrated remarkable capabilities in code generation tasks. However, when applied to hardware description languages (HDL), these models exhibit significant limitations due to data scarcity, resulting in hallucinations and incorrect code generation. To address these challenges, we propose HDLCoRe, a training-free framework that enhances LLMs’ HDL generation capabilities through prompt engineering techniques and retrieval-augmented generation (RAG). Our approach consists of two main components: (1) an HDL-aware Chain-of-Thought (CoT) prompting technique with self-verification that classifies tasks by complexity and type, incorporates domain-specific knowledge, and guides LLMs through step-by-step self-simulation for error correction; and (2) a two-stage heterogeneous RAG system that addresses formatting inconsistencies through key component extraction and efficiently retrieves relevant HDL examples through sequential filtering and re-ranking. HDLCoRe eliminates the need for model fine-tuning while substantially improving LLMs’ HDL generation capabilities. Experimental results demonstrate that our framework achieves superior performance on the RTLLM2.0 benchmark, significantly reducing hallucinations and improving both syntactic and functional correctness. † † footnotetext: * Equal Contribution. † Corresponding to: [email protected] . 1 Introduction With the rapid advancement of semiconductor technology, the design of very large-scale integration (VLSI) has become increasingly vital across industries Huang et al. 2021 . Hardware description language (HDL) code, as the foundation of VLSI design, plays a critical role in defining the circuit architecture and functionality Palnitkar 2003 . In recent years, large language models (LLMs) have experienced explosive growth and demonstrated extraordinary capabilities in many aspects Kanakaris et al. 2025 ; Li et al. 2025 , especially in automated code generation Brown et al. 2020 ; Chen et al. 2021 . LLMs like GPT-4 Achiam et al. 2023 and Claude Anthropic 2024 not only match expert programmers in general-purpose languages such as Python and C++, but also show unexpected competence in HDL. Automating HDL code generation efficiently and accurately represents a significant research opportunity that could substantially accelerate hardware development cycles Sun et al. 2025 ; Zhao et al. 2024b . Despite their impressive performance, commercial closed-source LLMs raise important privacy concerns, especially in hardware design where proprietary HDL code often represents valuable intellectual property requiring protection Gohil et al. 2024 ; Liu et al. 2024c ; Liu et al. 2024a . Open-source LLMs like Llama Touvron et al. 2023 and Qwen Bai et al. 2023 offer a promising alternative that better safeguards user data. However, these open LLMs face significant challenges when generating HDL code. They show a stronger tendency towards hallucination compared to their performance with general-purpose languages Yang et al. 2025 . This means they often generate HDL code that appears plausible but contains syntax and functional errors. The primary reason is the limited HDL-specific data in their training corpora, leading to insufficient knowledge about hardware design principles Sun et al. 2024 . Researchers have explored various approaches to enhance LLMs’ HDL code generation capabilities Pinckney et al. 2025 . Fine-tuning methods construct specialized HDL datasets to adapt open-source LLMs, sometimes achieving or surpassing commercial LLMs in HDL code generation Thakur et al. 2023a ; Thakur et al. 2024 ; Liu et al. 2023 ; Pei et al. 2024 ; Liu et al. 2024b ; Chang et al. 2024 ; Gao et al. 2024 ; Zhang et al. 2024 . However, these approaches typically demand substantial computational resources and complex training procedures, limiting their practicality for individual users or smaller teams. Consequently, training-free methods have gained attention Fu et al. 2023 ; Huang et al. 2024 ; Thakur et al. 2023b ; Nakkab et al. 2024 ; Vijayaraghavan et al. 2024 ; Zhao et al. 2025 . Some leverage Chain-of-Thought (CoT) Wei et al. 2022 prompting to decompose the generation process into sequential reasoning steps Vijayaraghavan et al. 2024 ; Lu et al. 2024 . Others introduce hardware paradigm knowledge, distinguishing between combinational and sequential logic Sun et al. 2025 ; Sun et al. 2024 . Some incorporate simulator feedback through tools like VCS or Design Compiler Huang et al. 2024 ; Thakur et al. 2023b ; Nakkab et al. 2024 . However, these methods either introduce only basic HDL knowledge or require multiple non-end-to-end iterations with external tools, complicating the development process. To overcome these limitations, we propose HDLCoRe (HDL-aware CoT and Retrieval), an efficient training-free and tool-free framework that enhances open-source LLMs’ HDL code generation capabilities through two primary components. First, an HDL-aware CoT prompting technique with self-verification classifies HDL tasks by complexity and type, then augments prompts with tailored HDL knowledge to guide initial code generation. This approach subsequently prompts the LLM to create corresponding testbenches and perform step-by-step self-simulation, enabling identification of functional issues and optimizations for code refinement—all within a structured CoT paradigm with intermediate reasoning steps. Second, our efficient two-stage RAG system establishes a comprehensive heterogeneous database from curated open-source HDL datasets and addresses formatting inconsistencies by extracting multiple key components from task descriptions. The system employs a novel retrieval process that first conducts broad filtering to identify potential candidates, then performs refined re-ranking to select the most relevant HDL code examples. This systematic approach provides contextually appropriate guidance to the LLM, enhancing its ability to generate accurate HDL implementations while maintaining computational efficiency throughout the generation process. Our contributions. The key contributions of this work are as follows: • HDL-CoT : We introduce a novel HDL-aware CoT prompting technique with self-verification that systematically incorporates domain expertise. Our prompting technique integrates HDL knowledge specific to task classifications into prompts and implements an automatic testbench generation mechanism that enables internal verification and refinement of generated code. • Efficient Heterogeneous RAG : We establish a comprehensive heterogeneous database from carefully selected open-source HDL datasets. Our proposed RAG system extracts multiple key components from task descriptions to address formatting inconsistencies across diverse sources, then employs a two-stage retrieval process that efficiently identifies optimal HDL examples through sequential filtering and re-ranking, providing contextually relevant guidance for target tasks. • Superior Performance : Through extensive evaluation on the RTLLM2.0 benchmark, we demonstrate our framework significantly outperforms existing training-free and tool-free methods in both syntactic and functional correctness. Our framework shows substantial improvements over state-of-the-art approaches, achieving up to 28% higher functional pass@1 rates. 2 RELATED WORK 2.1 Fine-tuning Approaches for HDL Code Generation Despite privacy advantages, open-source LLMs struggle with hallucination in HDL code generation due to domain-specific data scarcity. Several fine-tuning approaches address this limitation through different dataset construction strategies Thakur et al. 2023a ; Thakur et al. 2024 ; Liu et al. 2023 ; Pei et al. 2024 ; Liu et al. 2024b ; Chang et al. 2024 ; Gao et al. 2024 ; Zhang et al. 2024 . VerGen Thakur et al. 2024 , Chipnemo Liu et al. 2023 created large-scale HDL datasets by collecting and filtering publicly available code from GitHub repositories. Subsequently, instruction-based fine-tuning methods emerged to enhance task-specific performance. Among these, RTLCoder Liu et al. 2024b generated task descriptions and HDL implementations by prompting GPT-3.5 with hardware keywords, then filtering results through syntax checking. CodeV Zhao et al. 2024b reversed this