ASIC-Agent: An Autonomous Multi-Agent System for ASIC Design with Benchmark Evaluation
Ahmed Allam, Youssef Mansour, Mohamed Shalan · 2025
Read ASIC-Agent: An Autonomous Multi-Agent System for ASIC Design with Benchmark Evaluation by Ahmed Allam, Youssef Mansour, Mohamed Shalan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15940.txt.
Introduction
Ahmed Allam, Youssef Mansour, and Mohamed Shalan build an AI team that designs computer chips on its own, in this 2025 paper. Language models can write chip code from plain descriptions, but they cannot run tools, fix errors, or remember past work. Their system, ASIC-Agent, wraps a model in a crew of specialist helpers to close those gaps.
The helpers split the work: writing the logic code, checking it, hardening the layout, and fitting the chip into a carrier frame. All of them work in a sandbox with real design tools and a searchable store of documents and past fixes. The team also built the first benchmark for such agents in chip design. With a strong base model, the system finished tasks of varied difficulty end to end.
Catalog record
| CREATOR | Ahmed Allam, Youssef Mansour, Mohamed Shalan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-15940
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ASIC-Agent: An Autonomous Multi-Agent System for ASIC Design with Benchmark Evaluation Ahmed Allam, Youssef Mansour, Mohamed Shalan Abstract Large Language Models (LLMs) have demonstrated remarkable capabilities in Register Transfer Level (RTL) design, enabling high-quality code generation from natural language descriptions. However, LLMs alone face significant limitations in real-world hardware design workflows, including the inability to execute code, lack of debugging capabilities, and absence of long-term memory. To address these challenges, we present ASIC-Agent, an autonomous system designed specifically for digital ASIC design tasks. ASIC-Agent enhances base LLMs with a multi-agent architecture incorporating specialized sub-agents for RTL generation, verification, OpenLane hardening, and Caravel chip integration, all operating within a comprehensive sandbox environment with access to essential hardware design tools. The system leverages a vector database containing documentation, API references, error knowledge, and curated insights from the open-source silicon community. To evaluate ASIC-Agent’s performance, we introduce ASIC-Agent-Bench, the first benchmark specifically designed to assess agentic systems in hardware design tasks. We evaluate ASIC-Agent with various base LLMs, providing quantitative comparisons and qualitative insights into agent behavior across different design scenarios. Our results demonstrate that ASIC-Agent, when powered by Claude 4 Sonnet, successfully automates a broad range of ASIC design tasks spanning varying levels of complexity, showing the potential of significantly accelerating the ASIC design workflow. Our work is open-source and publicly available on Github 1 1 1 https://github.com/AUCOHL/ASIC-Agent-Bench . I Introduction Digital Application-Specific Integrated Circuits (ASICs) are pivotal to the functionality of modern electronic systems, delivering tailored performance and efficiency across diverse applications, from consumer electronics to high-performance computing. The design of digital ASICs, however, remains a complex and resource-intensive endeavor. Stages such as Register Transfer Level (RTL) modeling, verification, synthesis, and physical design demand significant manual effort, with RTL modeling and verification being particularly labor-intensive and prone to errors. As hardware complexity escalates, the need for automation in these processes becomes increasingly critical to reduce development time, minimize errors, and accelerate time-to-market. The emergence of Large Language Models (LLMs) has opened new avenues for automation in code generation, driven by advancements in artificial intelligence. LLMs have demonstrated remarkable success in producing high-quality code for software languages like Python, Java, and C++ from natural language descriptions [ 1 , 2 , 3 ] , hinting at their potential to transform hardware design workflows. In the context of digital ASIC design, LLMs have shown promise in generating RTL code, such as Verilog, thereby alleviating some of the burdens of manual design tasks [ 4 , 5 ] . However, applying LLMs directly to hardware design reveals significant limitations: they cannot execute code, lack real-time debugging capabilities, and do not possess long-term memory—features essential for navigating the intricacies of ASIC development. Notably, progress in software engineering agents, such as SWE-agent [ 6 ] and OpenHands [ 7 ] , has demonstrated the potential of autonomous systems in automating complex coding tasks. Inspired by these developments, we present ASIC-Agent, an autonomous multi-agent system tailored for digital ASIC design, addressing the unique challenges of hardware development. ASIC-Agent builds upon the generative strengths of LLMs by integrating a multi-agent architecture featuring specialized sub-agents dedicated to key tasks: RTL generation, verification, OpenLane hardening, and Caravel chip integration. These sub-agents operate within a robust sandbox environment equipped with critical open-source hardware design tools, including iverilog, yosys, OpenLane, and more, enabling them to interact dynamically with the design process. Additionally, ASIC-Agent leverages a vector database containing documentation, API references, a curated error knowledge base, and insights from the open-source silicon community, enhancing its ability to tackle complex design challenges. While there are several benchmarks for RTL and hardware, such as VerilogEval [ 8 ] and RTLLM [ 9 ] , they are designed for simpler, non-agentic tasks, typically focusing on generating single-file or standalone Verilog modules. These benchmarks fail to capture the complexity of real-world ASIC design workflows, which often involve multi-file contexts, dynamic tool interactions, and iterative debugging—capabilities inherent to agentic systems. To address this gap, we introduce ASIC-Agent-Bench, the first benchmark specifically tailored to evaluate agentic systems in hardware design tasks. ASIC-Agent-Bench comprises a diverse set of real-world, open-ended design scenarios, offering a standardized and rigorous framework to assess the performance of agentic systems comprehensively. In this paper, we make two primary contributions. First, we introduce ASIC-Agent, an innovative framework that integrates large language models (LLMs) with a multi-agent architecture to overcome the constraints inherent to using standalone LLMs in hardware design. Second, we present ASIC-Agent-Bench, a critical benchmarking tool that enables rigorous quantitative evaluation and fosters further advancements in agent-based systems for ASIC design. We conduct extensive evaluations of ASIC-Agent across multiple base LLMs, offering both quantitative performance comparisons and qualitative analyses of agent behaviors in varied design scenarios. II Related Work Autonomous agents have grown increasingly important in software engineering, utilizing large language models (LLMs) to automate code generation, debugging, and issue resolution. Systems like SWE-agent [ 6 ] and OpenHands [ 7 ] interpret natural language prompts to produce functional code in simulated development environments, effectively handling code repositories and tests across multiple files. While software agents have advanced significantly, hardware agents remain in earlier stages of development, particularly for Register Transfer Level (RTL) design in Verilog. Current approaches include standalone RTL-focused LLMs like ChipNeMo [ 10 ] and RTLCoder [ 5 ] , which generate Verilog but lack execution capabilities and integration with broader design flows. Hardware agent systems such as VerilogCoder [ 11 ] employ task planning and AST-based debugging but are limited to module-level generation without ASIC synthesis capabilities. Similarly, MAGE [ 12 ] uses multiple agents for RTL and testbench generation with recursive debugging, yet it excludes critical physical design stages. Existing approaches—both standalone LLMs and specialized hardware agents—cannot manage the complete ASIC development process, highlighting the need for a more comprehensive system to streamline complex ASIC workflows. III ASIC-Agent Fig. 1 : ASIC-Agent System Architecture ASIC-Agent is a specialized autonomous system for digital ASIC design, built upon the foundation of OpenHands (formerly known as OpenDevin) [ 7 ] and the CodeAct system [ 13 ] . While leveraging OpenHands’ proven infrastructure for autonomous software development, ASIC-Agent extends this framework specifically for hardware design workflows, transforming it into a platform capable of handling the unique challenges of ASIC design. III-A Multi-Agent Architecture ASIC-Agent employs a multi-agent architecture designed to decompose the complex ASIC design workflow into specialized tasks while maintaining cohesive progress toward design objectives. This architecture consists of a central main agent and specialized sub-agents, each with distinct capabilities, specialized tools, and knowledge bases tailored to their specific roles across the ASIC design process. Each agent is equipped with a system prompt template optimized for its specific task domain, ensuring focused and effective performance. III-A 1 Main Agent The Main LLM Agent serves as the central agent and primary interface for the entire system. This agent directly handles RTL generation, synthesizing Verilog code from natural language specifications with appropriate module interfaces, signal declarations, and behavioral logic. The agent performs linting and static analysis on generated code to identify potential syntax errors, design rule violations, and suboptimal coding patterns before verification. It maintains a global state of the project including design constraints, specifications, and progress across all stages of the workflow.