CADmium: Fine-Tuning Code Language Models for Text-Driven Sequential CAD Design

Prashant Govindarajan, Davide Baldelli, Jay Pathak, Quentin Fournier, Sarath Chandar · 2025

Read CADmium: Fine-Tuning Code Language Models for Text-Driven Sequential CAD Design by Prashant Govindarajan, Davide Baldelli, Jay Pathak, Quentin Fournier, Sarath Chandar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-09792.txt.

Introduction

Prashant Govindarajan and co-authors present a 2025 paper on generating engineering designs from plain text. Computer-aided design builds the parts behind cars and planes, but modeling stays slow and manual. The authors teach code language models to write design steps in sequence, the way a human modeler would.

They first built a data set of over 170,000 CAD models paired with natural descriptions written by a strong AI pipeline. Fine-tuning code models on it yielded systems that turn a sentence into a JSON sequence of modeling steps. Since simple scores miss design quality, they add geometry checks based on roundness, curvature, and topology. The results show text-driven CAD is workable. Designers could draft parts by describing them.

Catalog record

CREATORPrashant Govindarajan, Davide Baldelli, Jay Pathak, Quentin Fournier, Sarath Chandar
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-09792

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CADmium: Fine-Tuning Code Language Models for Text-Driven Sequential CAD Design Prashant Govindarajan, Davide Baldelli, Jay Pathak, Quentin Fournier, Sarath Chandar Abstract Computer-aided design (CAD) is the digital construction of 2D and 3D objects, and is central to a wide range of engineering and manufacturing applications like automobile and aviation. Despite its importance, CAD modeling remains largely a time-intensive, manual task. Recent works have attempted to automate this process with small transformer-based models and handcrafted CAD sequence representations. However, there has been little effort to leverage the potential of large language models (LLMs) for sequential CAD design. In this work, we introduce a new large-scale dataset of more than 170k CAD models annotated with high-quality, human-like descriptions generated with our pipeline based on GPT-4.1. Using this dataset, we fine-tune powerful code-LLMs to generate CAD sequences represented in a JSON-based format from natural language descriptions, demonstrating the viability and effectiveness of this approach for text-conditioned CAD generation. Because simple metrics often fail to reflect the quality of generated objects, we introduce geometric and topological metrics based on sphericity, mean curvature, and Euler characteristic to provide richer structural insights. Our experiments and ablation studies on both synthetic and human-annotated data demonstrate that CADmium is able to automate CAD design, drastically speeding up the design of new objects. The dataset, code, and fine-tuned models are available online 1 1 1 The code is available at https://github.com/chandar-lab/CADmium , and the models and dataset are available at https://huggingface.co/collections/chandar-lab/cadmium-6866b402be81f39321af98d4 . . * * footnotetext: Equal contribution. 1 Introduction Figure 1 : The CADmium pipeline for text-to-CAD generation. CADmium reformulates CAD generation as a purely text-to-text task. First, GPT-4.1 generates natural-sounding yet geometrically precise descriptions of 176,017 objects using their construction sequences in minimal JSON, and up to 10 multi-view images rendered with Blender. Then, the Qwen2.5-Coder LLM is fine-tuned with LoRA to translate these descriptions back into CAD sequences. Throughout history, humans have developed ever-more complex objects for survival, architectural, engineering, and artistic purposes. These range from early inventions such as hunting tools, wheels, and clocks, to modern systems such as space shuttles. With the emergence of computers, programs were written to process machining instructions, and soon after, to design 3D objects virtually. Computer-aided design (CAD) has since been widely adopted, significantly improving the speed and efficiency of the engineering and manufacturing process. However, the CAD pipeline still requires significant human effort and expertise. Recognizing that CAD is a sequential process involving 2D sketches, such as lines and circles, followed by 3D operations like extrusions, recent studies have leveraged transformer architectures and publicly available CAD datasets to automatically generate design steps in an autoregressive manner [ 1 , 2 ] . Recently, there has been a growing interest in generating 3D objects from natural language descriptions, as this approach would make CAD accessible to a broader, non-expert audience and speed up the creation process even further. Such an interest likely stems from the progress in large pre-trained foundation models that has resulted in tremendous reasoning and generative capabilities in natural language [ 3 ] and images [ 4 ] . However, unlike these modalities, text-conditioned CAD object generation has not seen the same success as the field still faces two key challenges: (1) the lack of high-quality, human-like textual annotations for CAD samples, and (2) the absence of suitable embeddings for CAD design histories that facilitate the effective use of pre-trained language models. Current approaches address the first challenge by automating text annotation using sophisticated state-of-the-art annotation pipelines with vision-language and mixture of experts models [ 2 , 5 ] and the second with vectorized representations of CAD sequences combined with separate embedding layers and prompt tokens to train language models [ 2 , 5 ] . However, we identify critical shortcomings with both strategies. First, current machine-generated annotations often fail to strike a balance between fluency and geometric specificity, a balance essential for natural human interaction while containing sufficient geometric detail to unambiguously specify the desired 3D object. Second, prior approaches treat CAD generation as a domain-specific task requiring custom embeddings, thereby failing to fully leverage the potential of pre-trained language models and requiring significant computational resources to learn new embeddings from limited data. To address these limitations, we introduce CADmium 2 2 2 The name CADmium reflects the integration of “CAD” (Computer-Aided Design) and draws inspiration from the metal cadmium, known for its malleability and utility in precision applications. , a novel approach that reformulates CAD generation as a purely text-to-text task. Our contributions are as follows. First, we develop a novel annotation pipeline using GPT-4.1 [ 6 ] that processes up to 10 multi-view images of 3D objects along with their construction sequences to generate annotations that combine natural language fluency with geometric precision. We apply our pipeline on 176,017 CAD sequences from DeepCAD [ 1 ] and release a new dataset with improved expert-quality annotations. Second, we fine-tune Qwen 2.5-Coder-14B [ 7 ] , a state-of-the-art instruction-tuned code LLM, on our dataset to generate JSON-formatted CAD sequences from natural language prompts, leveraging the capabilities of the pre-trained model without the need for specialized embedding layers. Recent works have shown that fine-tuning LLMs can be effective for generating code used in visual content creation [ 8 , 9 ] , emphasizing their versatility in visual generative applications. We evaluate our approach against state-of-the-art CAD generation models, including Text2CAD [ 2 ] , using both point-cloud and mesh-based metrics established in prior work. We also validate our models on the Fusion360 Reconstruction dataset [ 10 ] using annotations generated by our pipeline and the CADPrompt benchmark [ 11 ] that contains 200 objects annotated by human experts. Third, we introduce novel metrics based on sphericity, mean curvature, Euler characteristic, and watertightness to more extensively evaluate the structural and topological characteristics of the generated objects. We highlight the human-like fluency and conciseness of CADmium’s expert-level text annotations relative to Text2CAD, supported by extensive empirical analysis. Furthermore, our fine-tuned LLM achieves competitive performance against the pre-trained Text2CAD model, with improvements across some evaluation metrics as model size increases. 2 Related Works The Fusion 360 Gallery [ 10 ] was the first dataset of human-designed CAD objects to include their corresponding construction sequences. However, the Fusion 360 Gallery only contains 8,625 samples. As this was not enough for training, DeepCAD [ 1 ] introduced a significantly larger dataset of 176,017 CAD models constructed using only sketch and extrude operations parsed from the ABC dataset [ 12 ] . Leveraging their large-scale dataset, DeepCAD trained a transformer-based autoencoder within a latent-GAN framework for unconditional CAD generation. SkexGen [ 13 ] and SolidGen [ 14 ] later improved upon DeepCAD by introducing alternative input representations such as disentangled codebooks and boundary representations. However, these approaches were limited by their unconditional nature, offering no mechanism for users to control the generated objects. As a result, subsequent works shifted toward conditional generation from point clouds, images, or text inputs [ 5 , 15 , 16 , 17 , 2 , 18 ] . Progress in text-conditioned CAD generation was, however, hindered by the lack of datasets with textual annotations. As a solution, Text2CAD [ 2 ] proposed a novel annotation pipeline based

Read the complete text