Spec2Cov: An Agentic Framework for Code Coverage Closure of Digital Hardware Designs
Sean Lowe, Elias Hilaneh, Alma Babbit, Nakul Gopalan, Vidya Chhabria, Aman Arora · 2026
Read Spec2Cov: An Agentic Framework for Code Coverage Closure of Digital Hardware Designs by Sean Lowe, Elias Hilaneh, Alma Babbit, Nakul Gopalan, Vidya Chhabria, Aman Arora (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15606.txt.
Introduction
Lowe and five colleagues introduce Spec2Cov, an agentic framework that closes code coverage for digital hardware designs by generating test stimulus straight from specifications. It coordinates a language model with a hardware simulator, handling compile and simulation errors, reading coverage reports, and feeding results back for refinement. Across 26 designs it reaches 100 percent coverage on simple ones and up to 49 percent on complex ones.
Catalog record
| CREATOR | Sean Lowe, Elias Hilaneh, Alma Babbit, Nakul Gopalan, Vidya Chhabria, Aman Arora |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30680 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.15606
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Spec2Cov: An Agentic Framework for Code Coverage Closure of Digital Hardware Designs Sean Lowe, Elias Hilaneh, Alma Babbit, Nakul Gopalan, Vidya Chhabria, Aman Arora Abstract Hardware verification is one of the most challenging stages of the hardware design process, requiring significant time and resources to ensure a design is fully validated and production-ready. Verification teams aim to maximize design coverage while ensuring correct behavior and alignment with the specification. Coverage closure, which relies on iterative constrained-random and directed testing, is still largely manual and therefore slow and labor-intensive. Recent advances show that the code generation capabilities of Large Language Models (LLMs) can be integrated with external tools to build agentic workflows that autonomously perform hardware design and verification tasks. In this work, we introduce Spec2Cov, an agentic framework that automatically and iteratively generates test stimulus directly from design specifications to accelerate coverage closure. Spec2Cov coordinates interactions between an LLM and a hardware simulator, managing compilation and simulation errors, parsing coverage reports, and feeding results back to the model for refinement. We present features that improve Spec2Cov’s effectiveness without additional fine-tuning and evaluate their impact. Across 26 designs of varying size and complexity, including problems from the CVDP benchmark suite, Spec2Cov demonstrates promising performance, achieving 100% coverage on simpler designs and up to 49% on more complex designs. I Introduction Pre-silicon hardware verification is an exceedingly laborious and time-intensive process, often representing a critical bottleneck in achieving tapeouts. The goal of verification is to ensure that the implemented design matches the specifications. While faster execution is crucial, the necessity for high-quality verification, characterized by high coverage and low test fail rates, cannot be overstated. Coverage closure is an iterative process that starts by interpreting design specifications, drafting a detailed testplan, and developing a testbench with constrained-random testcases. Coverage data from regression runs is analyzed and directed tests are written to iteratively close coverage holes. In current practice, this cycle is still largely manual and consumes a substantial portion of verification time. Fig. 1 : An overview of Spec2Cov, our agentic framework for rapid coverage closure. Recent research has demonstrated that large language models (LLMs) can be employed to generate design and verification code automatically [ 2 , 11 , 16 , 26 ] , including assertions and testcases. However, prior work has largely overlooked using design specifications as the primary source for verification code generation, which reduces practical applicability. Showing design code early can bias the LLM and leave design bugs unchecked. The design code is best shown to the LLM only during the iterative coverage closure phase. Moreover, the emergence of agentic frameworks - systems that combine LLMs with external tools and feedback loops - has opened new avenues for automation. Leveraging context-awareness, reasoning, and agentic capabilities of modern LLMs, we can significantly reduce human effort for coverage closure. In this work, we present Spec2Cov, an agentic framework that mirrors this workflow with minimal human intervention. It generates testcases directly from design specifications, integrates them into a testbench template, starts with constrained-random testing, and iteratively refines stimuli using coverage reports and simulator errors to close code-coverage holes. While functional coverage offers a more comprehensive assessment of design behavior and can improve overall quality, code coverage is widely regarded as essential across the industry and is typically considered the minimum requirement for verification signoff. Given its critical role, Spec2Cov framework focuses on code coverage, specifically statement (or line) coverage. Nevertheless, the proposed methodology is general and can be easily extended to other types of code coverage and functional coverage. A high-level overview of Spec2Cov is shown in Fig 1 . The LLM is given a specification that contains supporting context about a design’s functionality, submodules and implementation details. The LLM generates testcases which are integrated into a testbench template, starting with constrained random testcases. Based on simulator feedback, the LLM adapts and generates new testcases, progressively improving coverage in an autonomous loop. Spec2Cov accelerates coverage closure by automating stimulus generation and feedback using LLMs and simulator integration, reducing manual effort and speeding up verification. This frees engineers to focus on higher-value tasks like debugging and architectural refinement, boosting productivity without increasing headcount. It requires no additional model training, leveraging off-the-shelf LLMs for easy adoption, and lays the foundation for future agentic verification workflows. TABLE I : Comparison of LLM-assisted hardware verification works Work DV Focus LLMs Used Agentic? Goal Benchmarks / Size ChiRAAG [ 12 ] Assertion gen. ChatGPT (GPT-4) Yes Improve SVA quality 7 OpenTitan mods; 3–16 SVAs AssertLLM [ 5 ] Assertion gen. ChatGPT+RAG No Spec \rightarrow SVAs 1 I2C design; 23 signals AutoBench [ 16 ] Hybrid TB gen. ChatGPT (GPT-4) Yes UVM TB automation VerilogEval-Human; 156 HDLBits LLM4DV [ 9 ] Stimulus gen. GPT-3.5, LLaMA, Claude Yes Func. cov. closure 3 designs; \sim 3.9k bins Verilog Reader [ 11 ] Test gen. GPT-4 / Turbo Yes Code coverage closure 24 custom designs; up to 500+ LOC Hassan et al. [ 7 ] Iterative DV GPT-4 (example) Yes General DV loop No formal dataset (conceptual) Ye et al. [ 25 ] UVM env. gen. GPT-4 Yes UVM automation 9 designs; \leq 1.6k LOC Spec2Cov (Ours) Stimulus gen. GPT-4o Yes Code coverage closure CVDP + GitHub; 26 designs We make the following contributions in this work: • Spec2Cov Framework: We introduce Spec2Cov , an agentic framework that generates testcases directly from specifications and automates code coverage closure. • LLM Enhancement Techniques: We develop several techniques to improve the LLM’s effectiveness in generating high-quality testcases and accelerating coverage closure. These include: Testplan creation from specifications, Batched generation for parallel exploration, Context pruning to manage prompt length. • Comprehensive Evaluation: We evaluate Spec2Cov on 26 designs, including verification tasks from the CVDP [ 15 ] and some designs from GitHub. These designs span a wide range of complexity levels (easy, medium, hard) and code sizes (14 to 4500 lines). • Performance Analysis: We assess Spec2Cov using pass@k and code coverage metrics, and report its computational cost in terms of LLM generation time and token usage. Spec2Cov is the first open-source [ 1 ] framework focused on code coverage closure directly from specifications using CVDP designs for demonstration. II Related Work Several studies have illustrated the use of LLMs for the generation of hardware design code. Chip-Chat [ 2 ] studies the challenges and opportunities in using LLMs for hardware design. The authors use ChatGPT to generate a small 8-bit microprocessor design. In VeriGen [ 22 ] , the generation of hardware designs using ChatGPT and some fine-tuned LLMs is studied using a proprietary dataset created from GitHub and textbooks. GPT4AIGChip [ 6 ] applies ChatGPT to AI accelerator design and explores the limitations of LLMs when creating accelerator designs. Most recently, VerilogDB [ 3 ] and CVDP [ 15 ] present larger and more challenging RTL datasets for benchmarking and fine-tuning uses. Another interesting direction is debugging using LLMs. MEIC [ 24 ] focuses on debugging RTL design code using fine-tuning LLMs in an agentic workflow. UVLLM [ 8 ] identifies syntactical and functional errors in RTL designs using an agentic pipeline. There are some works using LLMs for hardware verification as well (Table I ). ChiRAAG [ 12 ] uses