SANGAM: S ystemVerilog A ssertio n G eneration vi a M onte Carlo Tree Self-Refine
Adarsh Gupta*, Bhabesh Mali*, Chandan Karfa · 2025
Read SANGAM: S ystemVerilog A ssertio n G eneration vi a M onte Carlo Tree Self-Refine by Adarsh Gupta*, Bhabesh Mali*, Chandan Karfa (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-13983.txt.
Introduction
Adarsh Gupta, Bhabesh Mali, and Chandan Karfa present SANGAM, a 2025 framework that writes hardware checks with help from language models. Chip teams state safety and design rules as assertions in SystemVerilog, a hardware language. Writing those rules by hand from long specifications is slow, and recent language models offer a way to draft them.
SANGAM works in three stages. First, helper agents read the specification, its signal lists, and its waveforms. Next, a tree-search method that refines its own drafts reasons out assertions signal by signal. Last, the traces are merged into the final assertion set. On industry-level specifications the framework produced a robust set and beat recent methods in the authors' grading.
Catalog record
| CREATOR | Adarsh Gupta*, Bhabesh Mali*, Chandan Karfa |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-13983
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SANGAM: S ystemVerilog A ssertio n G eneration vi a M onte Carlo Tree Self-Refine Adarsh Gupta*, Bhabesh Mali*, Chandan Karfa Abstract Recent advancements in the field of reasoning using Large Language Models (LLMs) have created new possibilities for more complex and automatic Hardware Assertion Generation techniques. This paper introduces SANGAM, a SystemVerilog Assertion Generation framework using LLM-guided Monte Carlo Tree Search for the automatic generation of SVAs from industry-level specifications. The proposed framework utilizes a three-stage approach: Stage 1 consists of multi-modal Specification Processing using Signal Mapper, SPEC Analyzer, and Waveform Analyzer LLM Agents. Stage 2 consists of using the Monte Carlo Tree Self-Refine (MCTSr) algorithm for automatic reasoning about SVAs for each signal, and finally, Stage 3 combines the MCTSr-generated reasoning traces to generate SVA assertions for each signal. The results demonstrated that our framework, SANGAM, can generate a robust set of SVAs, performing better in the evaluation process in comparison to the recent methods. I Introduction In recent years, the use of LLMs has surged in Electronic Design Automation (EDA), because of their capabilities in natural language understanding, reasoning, and automated code generation. Hardware Assertion Generation using various methods incorporating LLMs, has attracted the major attention of researchers [ 1 , 2 ] . SystemVerilog Assertions (SVAs) play a critical role in Assertion-Based Verification (ABV) for ensuring the correctness and security of hardware designs. Manually writing SVAs is time-consuming for verification engineers and often becomes more complex for larger designs, and can also lead to various errors due to language ambiguity. In this paper, we propose SANGAM, a framework that uses LLM-guided Monte Carlo Tree Search (MCTS) to automate SVA generation from multi-modal specifications. Using LLMs for MCTS requires various approximations in reward calculations and node expansion. SANGAM iteratively explores the solution space by continuously refining assertions based on critic and formal feedback. This iterative exploration allows SANGAM to explore various possible reasoning paths, helping capture a wide range of SVAs. * * footnotetext: These authors contributed equally to this work We developed a three-stage strategy to produce the assertions. The first stage involves processing the multi-modal specification data, consisting of text, waveforms, and images. It involves three custom LLMs: Spec Analyzer, Signal Mapper, and Waveform Analyzer. Each LLM produces significant and essential information about every signal and waveform using the design specification. Then stage two and three use this information to generate the SVAs. Stage two incorporates a modified version of the MCTSr Algorithm [ 3 ] to produce various reasoning paths for the SVAs in a world model given by the LLM. Stage three further de-duplicates the assertions in all the reasoning paths and combines them into a cohesive set of assertions. The code and information are available at online 1 1 1 https://github.com/CoolSunflower/SANGAM . Specifically, the key contributions of our work are: • We proposed a novel framework, SANGAM, to generate SVAs from multi-modal specifications consisting of design documents, architecture, and waveforms. • SANGAM consists of three main steps: Specification Processing, Assertion Reasoning, and Assertion Generation, focusing on signal-wise reasoning and SVA generation. It also uses a novel Waveform Analyzer LLM during Specification Processing to extract signal interdependence from the specification waveforms. • SANGAM framework uses a novel MCTS-based algorithm for effective exploration of the assertion space for each signal. This effective exploration allows it to discover an extensive set of assertions for each signal. • SANGAM generates twice the assertions as state-of-the-art methods, such as AssertLLM [ 4 ] and ChIRAAG [ 2 ] . The coverage analysis demonstrates that these assertions are of high quality. The rest of the paper is organized as follows: Section II covers the background and related works. Section III covers the proposed methodology. Section IV covers the experimental results and discussion. Section V covers the conclusion and future works. II Background and Related Works II-A LLM for EDA The growth of the LLM industry is estimated to increase exponentially from USD 6.4 billion in 2024 to USD 36.1 billion by 2030 [ 5 ] . LLMs are becoming an integral part of the EDA flow. As in [ 6 ] , the authors have introduced the first LLM-enabled high-level synthesis (HLS) framework, HLSPilot. The framework fully automates the HLS code generation, utilizing sequential C/C++ code as input to the LLM. While [ 7 ] have used LLMs to reduce manual efforts while designing ML hardware accelerators. In [ 8 ] , the authors introduce a novel and efficient LLM model designed especially for standard cell layout design optimization for generating high-quality cluster constraints to enhance cell layout performance, power, and area (PPA). While [ 9 , 10 ] shows the capability of LLMs in generating HDL Code. LayoutCopilot [ 11 ] is the first interactive framework for LLM-powered analog layout design. II-B LLM for Assertion Generation Manually writing SVAs is a time-consuming process. Recently researchers have focused on generating assertions using LLMs because of their exceptional ability to generate quality results. In [ 2 ] , the authors have proposed a framework named ChIRAAG that automates the process of generating assertions using OpenAI’s GPT4 model. They provided natural language specification as input to the LLM, further evaluating the generated assertions using testbench, and re-prompted the model to address any error that occurred during evaluation. In [ 12 ] , the authors proposed AssertLLM, a framework that utilizes three different LLMs for generating SVAs. Firstly, natural language specifications are analyzed using Spec Analyzer LLM to extract information for each signal. The Signal Mapper LLM then maps each signal to its Verilog variable name using the provided Verilog code. Finally, this information is given to an SVA Generator LLM to generate the SVAs. However, since the assertions are generated in a single iteration, there is limited scope for refinement in case of errors. Furthermore, most assertions are not aligned with the design’s core functionality. The authors of [ 13 ] have generated hardware security assertions using LLM and also contributed to providing a comprehensive benchmark suite consisting of real-world hardware designs and corresponding golden assertions. They took comments from the assertion’s code file and processed them to feed as input to the LLM. Their experiments showed that with sufficient details in the prompt, LLMs are capable of achieving good results in generating assertions. II-C Monte Carlo Tree Search (MCTS) MCTS is a decision-making algorithm, widely used in games and complex decision processes. It operates by building a search tree and simulating outcomes to estimate the value of actions [ 14 ] . A single iteration of MCTS proceeds in the following four phases: • Selection: Starting from the root node, the algorithm goes through all the nodes and selects the most promising child node based on specific strategies, such as the Upper Confidence Threshold (UCT). • Expansion: At this child node, a feasible child node is added that denotes a potential future move. • Simulation for Evaluation: From this newly added node, the algorithm conducts a random simulation, by arbitrarily selecting moves until it reaches a terminal node. The reward provided at the terminal node is a measure of the goodness of this newly selected action. • Backpropagation: Post-simulation, the results of the simulation are propagated back through all