MUSE : Benchmarking Manufacturable, Functional, and Assemblable Text-to-CAD Generation

Xiaoyu Dong, Zhi Li, Xiao-Ming Wu · 2026

Read MUSE : Benchmarking Manufacturable, Functional, and Assemblable Text-to-CAD Generation by Xiaoyu Dong, Zhi Li, Xiao-Ming Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28579.txt.

Introduction

Xiaoyu Dong and three colleagues introduce MUSE, a benchmark for text-to-CAD generation of multi-part assemblies. Existing benchmarks stress single parts and shape similarity, missing how parts fit together and join. MUSE pairs practical design cases with structured design specs and scores outputs on four fronts: runnable code, valid shapes, true part fits and joins, and consistent function. Tests show high execution success does not guarantee correct fits or joins, and a new check combines part graphs with direct boundary-representation analysis.

Catalog record

CREATORXiaoyu Dong, Zhi Li, Xiao-Ming Wu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76332
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.28579

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MUSE : Benchmarking Manufacturable, Functional, and Assemblable Text-to-CAD Generation Xiaoyu Dong, Zhi Li, Xiao-Ming Wu Abstract Large language models (LLMs) have recently advanced text-driven 3D generation, yet Text-to-CAD remains far from supporting industrial product design. Existing benchmarks focus primarily on generating single-part CAD models and evaluate them using geometric similarity metrics that fail to capture functionality, manufacturability, and assemblability. To address this gap, we introduce MUSE , a Text-to-CAD benchmark focused on complex, editable boundary representation (B-Rep) assemblies. MUSE pairs practical design instances with structured Design Specifications and evaluates generated models through a three-stage protocol: code check, geometric check, and design-intent alignment. The final stage uses design-specific rubrics to assess functionality, manufacturability, and assemblability, moving beyond shape matching toward practical design quality. To enable scalable evaluation, we use a rubric-based visual language model (VLM) judge and validate its reliability through human annotation. Experiments on closed-source and open-source LLMs reveal a clear failure cascade from executable code to valid geometry and finally to engineering-ready design, with even the strongest models achieving limited success on fine-grained engineering criteria. Together, MUSE provides a realistic benchmark and evaluation framework for advancing Text-to-CAD from geometric generation toward true engineering design. Our project website, including the leaderboard, dataset, and code, is available at https://dong7313.github.io/muse-benchmark/ . Figure 1: A subset of designs from MUSE. 1 Introduction Text-to-CAD, the task of generating computer-aided design (CAD) models from natural-language descriptions, is emerging as a promising direction for AI-assisted 3D modeling [ 9 ] . Unlike Text-to-3D generation [ 24 , 13 ] , which typically uses mesh representations and emphasizes visual appearance, Text-to-CAD targets design instances that must satisfy practical requirements such as functionality, manufacturability, and assemblability . This shifts the focus from visual plausibility to engineering usability: the generated design must be geometrically valid and support downstream fabrication and assembly. In recent years, Text-to-CAD datasets [ 9 ] have grown substantially. The existing datasets are built from modeling histories, such as command sequences and scripts, as summarized in Table 1 . However, most existing datasets focus on single-part CAD models rather than complete design instances. As a result, they rarely capture multi-component structures or assembly constraints, and provide limited coverage of real-world use and fabrication requirements. Unlike standalone CAD parts, a dataset of design instances considering functionality, manufacturability, and assemblability is costly to build because each design instance requires a designer with extensive experience to expend a tremendous amount of effort. This makes every design instance extremely rare and highly valuable. Beyond dataset construction, evaluation is another key challenge for Text-to-CAD. Most evaluations rely on geometric similarity metrics, such as Chamfer Distance (CD) [ 27 ] , which measure visual resemblance to a reference shape but cannot reliably judge whether a generated model is a good design instance. For example, a chair that differs visually from the reference may still satisfy the same seating function, whereas a reference-like chair may still fail as a design if it is unstable, difficult to manufacture, or hard to assemble [ 20 ] . This highlights the need for Text-to-CAD evaluation protocols that assess design quality rather than geometric similarity alone. To this end, we propose MUSE , a benchmark for functional, manufacturable, and assemblable Text-to-CAD generation. MUSE is constructed by the collaboration of human designers and LLM, focusing on practical CAD design instances rather than single-part CAD models. Each design instance is defined by a structured Design Specification , which decomposes a high-level design goal into physical assembly graph, valid parameter ranges and a manufacturing plan. Furthermore, to support engineering-grounded evaluation, each design instance is paired with program-generated CAD drawings for visual review. All design instances are carefully reviewed by human designers. MUSE further introduces a three-stage evaluation protocol. First, we evaluate the executability of the CadQuery script generated by the latest LLMs and export the resulting CAD model. Second, we check the geometric validity of the exported CAD model, including watertightness, self-intersection, non-manifold features, and overlapping components. Third, we assess design-intent alignment using design-specific rubrics generated from the Design Specification, reference CAD drawings, reference code, and engineering knowledge tables. These rubrics characterize design validity in terms of functionality, manufacturability, and assemblability under the intended requirements. Experiments on MUSE show that current LLMs for Text-to-CAD remain far from producing usable designs . Across both closed-source and open-source models, performance degrades sharply from code executability to geometric validity and then to design-intent alignment, revealing a clear failure cascade in practical CAD generation. Although closed-source models consistently outperform open-source ones, even the strongest model achieves only limited success on fine-grained criteria of functionality, manufacturability, and assemblability. We further show that our rubric-based visual language model (VLM) judge aligns well with human annotations, supporting scalable and reliable automatic evaluation. 2 Related Work Text-to-CAD. Text-to-CAD aims to translate natural language into editable CAD models. Existing methods mainly follow two paradigms: command-sequence generation , which predicts low-level CAD operations such as extrusion or lofting [ 29 , 10 , 28 , 27 , 15 ] , and code-based generation , which produces executable scripts such as CadQuery programs [ 27 , 15 ] . Most existing datasets are derived from CAD construction sequences paired with synthetic or LLM-generated text [ 29 , 10 , 15 , 27 , 28 ] , including DeepCAD [ 29 ] , Text2CAD [ 10 ] , and CADFusion [ 27 ] . However, current benchmarks largely focus on single-part or simple geometries and evaluate outputs using geometric metrics such as chamfer distance, parameter accuracy, or visual similarity [ 10 , 27 ] . These metrics capture shape resemblance, but not whether a model is executable, geometrically valid as a B-Rep, or aligned with design intent. Recent work has begun to explore assembly-aware generation, e.g., ArtiCAD [ 23 ] , but realistic assembly-level benchmarks and engineering-oriented evaluation remain limited. In contrast, MUSE focuses on complex, editable B-Rep assemblies paired with structured Design Specifications, and evaluates outputs by executability, geometric validity, and design-intent alignment, including functionality, manufacturability, and assemblability. VLM-as-Judge. Vision-language models are increasingly used as automatic evaluators, but direct holistic scoring is often unstable and prone to multimodal hallucination [ 4 , 7 , 11 ] . Recent work therefore advocates fine-grained, factorized criteria [ 12 , 18 , 6 , 14 , 16 , 31 ] . This is especially important for CAD, where visually plausible outputs may still contain invalid geometry, incorrect interfaces, unstable structures, or infeasible manufacturing details [ 5 , 30 , 3 ] . Prior Text-to-CAD systems often use rendered views for feedback or evaluation [ 26 , 8 ] , and recent methods incorporate VLMs for CAD verification or refinement, such as CADCodeVerify [ 1 ] , CADSmith [ 3 ] , and EvoCAD [ 21 ] . However, rendered images can obscure CAD-specific failures due to perspective distortion, occlusion, and limited view coverage. Our evaluation instead targets engineering correctness through design-specific rubrics grounded in Design Specifications, reference engineering drawings, reference code, and engineering knowledge tables. Based on this formulation, we develop a rubric-based VLM judge for design-intent alignment and validate it against human annotation. 3 MUSE Benchmark Figure 2: A unified assembly graph can correspond to different designs under different parameter settings, motivating the

Read the complete text