VeriDebug: A Unified LLM for Verilog Debugging via Contrastive Embedding and Guided Correction
Ning Wang, Bingkun Yao, Jie Zhou, Yuchen Hu, Xi Wang, Nan Guan, Zhe Jiang, [0.5em] City University of Hong Kong · 2025
Read VeriDebug: A Unified LLM for Verilog Debugging via Contrastive Embedding and Guided Correction by Ning Wang, Bingkun Yao, Jie Zhou, Yuchen Hu, Xi Wang, Nan Guan, Zhe Jiang, [0.5em] City University of Hong Kong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-19099.txt.
Introduction
Ning Wang led a team building a debugging tool for chip design code in a 2025 paper. Large language models debug normal software well, but their use on Verilog, the language for describing hardware, was little explored. Their tool combines two skills in one model: finding a bug and fixing it.
The tool first learns patterns that set buggy code apart from clean code, then uses retrieved internal details to guide the fix. Sharing one parameter space for both jobs streamlines the process. In tests, its best variant fixed bugs far more often than open rivals, and beat a larger closed model too. The authors offer it as a sharper option for hardware teams.
Catalog record
| CREATOR | Ning Wang, Bingkun Yao, Jie Zhou, Yuchen Hu, Xi Wang, Nan Guan, Zhe Jiang, [0.5em] City University of Hong Kong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-19099
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
VeriDebug: A Unified LLM for Verilog Debugging via Contrastive Embedding and Guided Correction Ning Wang, Bingkun Yao, Jie Zhou, Yuchen Hu, Xi Wang, Nan Guan, Zhe Jiang, [0.5em] City University of Hong Kong Abstract Large Language Models (LLMs) have demonstrated remarkable potential in debugging for various programming languages. However, the application of LLMs to Verilog debugging remains insufficiently explored. Here, we present VeriDebug , an approach that integrates contrastive representation and guided correction capabilities for automated Verilog debugging. Unlike existing methods, VeriDebug employs an embedding-based technique to accurately retrieve internal information, followed by bug-fixing. VeriDebug unifies Verilog bug detection and correction through a shared parameter space. By simultaneously learning bug patterns and fixes, it streamlines debugging via contrastive embedding and guided correction. Empirical results show the efficacy of VeriDebug in enhancing Verilog debugging. Our VeriDebug Loc, Type {}_{\text{Loc, Type}} model achieves 64.7% accuracy in bug fixing ( A c c @ 1 Acc@1 ), a significant improvement from the existing open-source SOTA’s 11.3%. This performance not only outperforms open-source alternatives but also exceeds larger closed-source models like GPT-3.5-turbo (36.6%), offering a more accurate alternative to conventional debugging methods. The Repo. of dataset and code: https://anonymous.4open.science/r/VeriDebug-F770/ . I Introduction Large Language Models (LLMs) have revolutionized natural language processing, enabling the generation of human-like text across diverse topics due to their unprecedented scale and complexity. Their application to programming languages has demonstrated remarkable capabilities in understanding, generating, and manipulating code across different languages [ 6 , 9 ] . Specifically, the integration of LLMs into hardware description languages like Verilog marks a significant advancement in Electronic Design Automation (EDA). However, in real-world scenarios, debugging existing Verilog code is often more prevalent and time-consuming than generating new code from scratch [ 4 ] . Despite this practical necessity, current research has predominantly focused on leveraging Large Language Models (LLMs) to fix bugs through generative methods. These approaches involve the LLM generating corrected versions of the code, effectively ”rewriting” faulty sections. While generative techniques have shown promise in automating code fixes, they are susceptible to hallucination issues—where the model produces plausible but incorrect code—which can undermine the reliability of the debugging process [ 9 ] . With that, researchers have integrated guidance mechanisms into the generative process. Techniques like Retrieval-Augmented Generation (RAG) incorporate external information to enhance the LLM’s outputs during debugging tasks [ 5 ] . However, RAG methods introduce their own set of challenges. Relying on external knowledge bases can be problematic if these resources are outdated, inaccessible, or infeasible to use in certain environments. Also, the models used for retrieval often depend on surface-level lexical similarity rather than deep semantic understanding, which may result in retrieving irrelevant or wrong information, leading to misguided fixes and compromise the effectiveness of the debugging process. The key challenge, hence, lies in developing robust automated Verilog debugging methods that effectively harness the generative capabilities of LLMs while solving these limitations. It’s essential to enhance the reliability and correctness of bug fixes by guiding the generative process with accurate and relevant information, without undue dependence on external resources or the risk of brining new errors through hallucinations. Contributions. Here, we propose VeriDebug , a framework that directly tackles these issues by employing guided generation techniques. Our method enhances the LLM’s ability to debug Verilog code through a two-phase approach: representation and generation. In the representation phase, VeriDebug utilizes embedding instructions to process input and generate vector representations for both the code and relevant information. The generation phase then leverages information retrieved based on the similarity between instruction embeddings and these representations, along with extracted bug information and design specifications. This approach reduces reliance on potentially unreliable external knowledge bases and minimizes the risk of hallucinations by providing the generative model with precisely retrieved, context-relevant guidance. By incorporating open-source models, VeriDebug ensures transparency and adaptability, fostering community-driven improvements. By addressing the root causes of deficiencies in existing approaches and streamlining the debugging process through effective information retrieval and guided code correction, VeriDebug significantly improves the effectiveness and reliability of automated debugging in hardware design contexts. Our contributions represent a substantial advancement in applying LLMs to Verilog debugging, offering promising implications for the broader field of EDA. The main contributions of this paper are: • VeriDebug : A debug framework with open-source models, ensuring transparency and fostering collaboration. • An embedding-based method to reduce reliance on external knowledge and mitigate hallucination risks. • An open-source dataset of 8,000 Verilog snippets with bugs and corrections to support further research. • Extensive experiments demonstrating VeriDebug ’s effectiveness in Verilog debugging across various bug types. Fig. 1: Overview of VeriDebug for Verilog bug detection and correction. This figure shows VeriDebug’s architecture and training methodology for identifying and correcting Verilog code bugs. The system learns to analyze buggy inputs and produce corrected outputs using embedding techniques and generative modeling. a) Instruction: Containing SPEC (specification) and BUGGY_PROG (buggy program) to provide essential context for analysis. b) Embedding Tasks: Employing contrastive learning for bug location and classification to transform inputs into vector representations. This distinguishes between positive (actual bugs) and negative examples. c) Generative Task: Producing corrected Verilog code using information retrieved based on similarity between instruction and candidate information to Outputbuggy code and its correction. d) Bug Information: Providing extra context about potential bug locations and types, used as input for the generative bug fix process. In training, we use a mix of ground truth and randomly sampled candidates for bug lines and types, simulating real-world scenarios. Note: Our paper utilizes open-source models to address key challenges in hardware design automation. Protecting intellectual property in hardware design is crucial, making closed-source LLMs problematic due to security risks when processing proprietary designs, particularly in complex SoC environments [ 1 , 2 ] . Recent research has highlighted unique security and privacy challenges with LLMs in various scenarios [ 3 ] . Consequently, there is a pressing need for secure, open-source solutions deployable locally, ensuring confidentiality and integrity of hardware designs while leveraging LLMs’ potential for generating efficient hardware. II Background II-A Embedding or Generative Models The field of Natural Language Processing (NLP) has witnessed remarkable both embedding and generative-based applications. Embedding models, such as text-embedding-ada-002, focus on creating dense vector representations of text that capture semantic relationships [ 11 ] . Recent work has explored multilingual embeddings, enabling zero-shot cross-lingual applications [ 12 ] . Generative language models, exemplified by GPT families, excel at producing human-like text across diverse tasks [ 13 ] . These models have shown remarkable few-shot and zero-shot learning capabilities, pushing the boundaries of natural language understanding and generation. Recent research also explored the integration of embedding and generative capabilities within unified models. Concept embedding generation [ 14 ] represents a promising direction, potentially enabling learning new concepts on-the-fly. GRIT [ 17 ] represents a significant advancement in unifying generative and embedding tasks. II-B Verilog Code Debugging with LLMs In recent years, LLMs have demonstrated remarkable capabilities in understanding, generating, and debugging code across multiple programming languages [ 15 ] . This trend has extended to hardware description languages (HDLs) like Verilog. Open-source models are particularly important in this context as they offer transparency, accessibility, and