ChipLingo: A Systematic Training Framework for Large Language Models in EDA
Lei Li, Xingwen Yu, Jianguo Ni, Junxuan Zhu, Jian Zhao, Zhi Liu · 2026
Read ChipLingo: A Systematic Training Framework for Large Language Models in EDA by Lei Li, Xingwen Yu, Jianguo Ni, Junxuan Zhu, Jian Zhao, Zhi Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27415.txt.
Introduction
Li and six colleagues present ChipLingo, a training pipeline that adapts large language models to electronic design automation, a knowledge-heavy, document-driven field. Its three stages build a domain corpus, pretrain adaptively, and align instructions with retrieval scenario training. ChipLingo 8B scores 59.7 percent on their EDA-Bench benchmark, beating the same-scale model and some larger general models, while ChipLingo 32B reaches about 70 percent, near leading closed-source models. Question-answer augmentation helps domain performance, and retrieval scenario training limits the usual post-training drop in retrieval use.
Catalog record
| CREATOR | Lei Li, Xingwen Yu, Jianguo Ni, Junxuan Zhu, Jian Zhao, Zhi Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43260 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.27415
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ChipLingo: A Systematic Training Framework for Large Language Models in EDA Lei Li, Xingwen Yu, Jianguo Ni, Junxuan Zhu, Jian Zhao, Zhi Liu Abstract With the rapid advancement of semiconductor technology, the complexity of Electronic Design Automation (EDA) tools has increased substantially, making EDA a highly knowledge-intensive and document-driven engineering domain. Although large language models (LLMs) have demonstrated strong capabilities in general tasks, their direct application to the EDA domain still faces significant challenges, including insufficient domain expertise, cross-tool knowledge confusion, and degradation of retrieval-augmented generation (RAG) capabilities after domain training. To address these challenges, this paper presents ChipLingo, a systematic training pipeline for domain-adapted large language models tailored to EDA scenarios. The proposed pipeline consists of three stages. First, we construct domain-specific corpora through multi-source data curation and question-answering (QA) augmentation. Second, during the domain-adaptive pretraining phase, we compare different parameter training strategies and examine their effects on domain adaptation performance and selected general capability benchmarks. Third, through instruction alignment and RAG scenario training targeting diverse retrieval conditions, we enhance the model’s ability to leverage external knowledge. To evaluate model performance on EDA question-answering tasks, we curate an internal benchmark called EDA-Bench, covering multiple representative EDA tool scenarios, which is planned for public release. Experimental results demonstrate that ChipLingo-8B achieves 59.7% accuracy on EDA-Bench, outperforming the base model of the same scale and surpassing some larger general-purpose models. ChipLingo-32B reaches 70.02% accuracy, approaching the performance of leading closed-source commercial models on this benchmark. Furthermore, we observe that under the current experimental settings, QA augmentation contributes to improved domain performance, Partial FT achieves a better trade-off between domain adaptation and general capability retention compared to LoRA, and explicit RAG scenario training can mitigate the decline in retrieval information utilization that occurs after domain training. These results indicate that a systematic domain training pipeline provides practical value for enhancing LLM performance on knowledge-intensive EDA tasks, while also laying the foundation for building EDA agents or harness capabilities that rely on external knowledge. Keywords: Large Language Models; Electronic Design Automation (EDA); Continued Pretraining; Retrieval-Augmented Generation (RAG); Knowledge-Intensive Tasks 1 Introduction 1.1 Background With the rapid advancement of semiconductor technology, the scale and complexity of modern chip systems continue to grow. State-of-the-art chips now contain billions or even tens of billions of transistors, with design workflows spanning multiple stages including logic synthesis, physical design, timing analysis, verification, and power optimization. To address these challenges, Electronic Design Automation (EDA) tools have become indispensable infrastructure [ 1 ] . However, the EDA ecosystem exhibits substantial complexity. Different design stages typically rely on distinct specialized tools, each providing extensive command interfaces, parameter configurations, and design constraint mechanisms. In practice, engineers must frequently consult large volumes of technical documentation, user manuals, and community forums to understand tool behavior, debug design issues, and optimize design flows. Moreover, significant differences exist among EDA tools in terms of command syntax, design workflows, and configuration approaches, further raising the learning curve and barriers to entry [ 2 ] . Knowledge-Intensive Characteristics. In engineering practice, substantial knowledge exists in the form of documentation, accumulated experience, and question-answering exchanges, distributed across diverse sources. This knowledge-intensive characteristic makes EDA a typical “document-driven” engineering domain. For more sophisticated EDA intelligent assistants, agents, or harness systems, models must not only answer questions but also maintain stable knowledge alignment across external documents, tool feedback, and task workflows. Therefore, reliable retrieval utilization capability can be viewed as a foundational ability for deploying higher-level EDA intelligent systems. 1.2 Problem Definition and Challenges 1.2.1 Limitations of General LLMs in the EDA Domain Large Language Models (LLMs) have achieved significant progress in natural language understanding, code generation, and knowledge-based question answering. General-purpose LLMs have demonstrated strong knowledge representation and task generalization capabilities across multiple domains [ 4 ] . Nevertheless, directly applying general LLMs to answer EDA domain questions still faces notable limitations: • Insufficient Domain Expertise : General models typically lack adequate EDA-specific knowledge, leading to inaccurate or overly generic responses to domain-specific questions. • Cross-Tool Knowledge Confusion : Due to significant differences in command interfaces and design constraints across EDA tools, models may confuse terminology and usage patterns between different tools. • Lack of Real-Time Knowledge : EDA domain questions often require reference to specific documentation or tool version information, necessitating the ability to retrieve up-to-date knowledge. 1.2.2 RAG Capability Degradation After Domain Training Given that EDA domain knowledge updates frequently and relies heavily on documentation, Retrieval-Augmented Generation (RAG) is considered an important technical approach for addressing domain knowledge challenges [ 4 , 5 , 6 ] . However, after domain-specific continued training, model RAG capabilities may exhibit notable degradation. After acquiring domain knowledge, models tend to rely more on parametric knowledge for answering questions rather than utilizing retrieved external knowledge. This phenomenon is particularly pronounced in knowledge-intensive domains and poses new challenges for RAG system stability. Furthermore, this issue affects not only single-turn question-answering scenarios but also directly impacts EDA agent or harness systems that depend on external documents, tool feedback, and contextual state for decision-making. 1.3 Main Contributions This paper presents ChipLingo , developed around knowledge-intensive EDA question-answering scenarios. The main contributions include: 1. We construct a systematic LLM training pipeline for the EDA domain, covering data preparation, domain-adaptive pretraining, instruction alignment, and RAG scenario training. 2. Under the current experimental settings, we systematically compare the effects of QA augmentation, LoRA, Full FT, and Partial FT training strategies on EDA domain adaptation and selected general capability benchmarks. 3. We observe and quantify changes in model retrieval information utilization after domain training, demonstrating that RAG scenario training targeting diverse retrieval conditions can mitigate this issue on the current benchmark. 4. We curate EDA-Bench, a benchmark covering multiple representative EDA tool scenarios for evaluating model performance on EDA engineering question-answering tasks. This benchmark is currently undergoing further curation and preparation for release. 2 Related Work 2.1 Domain-Specific Large Language Models With the successful application of LLMs in natural language processing tasks, an increasing number of studies have explored how to build domain-specific large language models [ 7 , 12 ] . These works demonstrate that continued pretraining or task adaptation on domain-specific corpora can significantly improve model performance in specialized scenarios. In the EDA and chip design domain, prior works have explored domain-specific models for chip design scenarios. ChipNeMo [ 3 ] systematically investigated domain-adaptive tokenizers, continued pretraining, instruction alignment, and retrieval augmentation for chip design. Additionally, ChipExpert [ 2 ] and customized RAG/benchmark work for EDA documentation QA [ 8 ] further demonstrate that general LLMs require domain adaptation to adequately handle EDA tasks. Meanwhile, VerilogEval [ 9 ] and similar works have constructed more focused evaluation frameworks from the perspective of hardware code generation, indicating that benchmarks and application scenarios for chip design tasks are becoming increasingly refined. Compared to these works, the EDA domain exhibits more complex knowledge structures. EDA knowledge exists not only in technical documentation but is also distributed across engineering experience, tool commands, and design workflows in various knowledge forms. 2.2 Domain Adaptation and Continued Pretraining Domain Adaptation is an important approach for building domain-specific models [ 10 , 11 , 7 ] . A common method involves performing continued pretraining on domain