CubePart: An Open-Vocabulary Part-Controllable 3D Generator Journal: TOG Conference: Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers; July 19–23, 2026; Los Angeles, CA, USA Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers (SIGGRAPH Conference Papers ’26), July 19–23, 2026, Los Angeles, CA, USA DOI: 10.1145/3799902.3811117 ISBN: 979-8-4007-2554-8/2026/07 CCS: Computing methodologies Artificial intelligence

Yiheng Zhu, Kangle Deng, Jean-Philippe Fauconnier, Inaki Navarro, Daiqing Li, Ava Pun, Yinan Zhang, Peiye Zhuang, Xiaoxia Sun, Maneesh Agrawala, Kiran Bhat, Tinghui Zhou · 2026

Read CubePart: An Open-Vocabulary Part-Controllable 3D Generator Journal: TOG Conference: Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers; July 19–23, 2026; Los Angeles, CA, USA Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers (SIGGRAPH Conference Papers ’26), July 19–23, 2026, Los Angeles, CA, USA DOI: 10.1145/3799902.3811117 ISBN: 979-8-4007-2554-8/2026/07 CCS: Computing methodologies Artificial intelligence by Yiheng Zhu, Kangle Deng, Jean-Philippe Fauconnier, Inaki Navarro, Daiqing Li, Ava Pun, Yinan Zhang, Peiye Zhuang, Xiaoxia Sun, Maneesh Agrawala, Kiran Bhat, Tinghui Zhou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28763.txt.

Introduction

Yiheng Zhu and eleven colleagues present CubePart, a framework for open-vocabulary, part-controllable 3D mesh making. From a global text prompt and a user-set parts list, it generates one mesh per part that joins into a whole object keeping the set structure. A big data flow builds a large part-labeled 3D dataset, and a two-stage design splits global shape making from part-level reading. The final assets plug directly into game engines and run under motion and action scripts with no manual work.

Catalog record

CREATORYiheng Zhu, Kangle Deng, Jean-Philippe Fauconnier, Inaki Navarro, Daiqing Li, Ava Pun, Yinan Zhang, Peiye Zhuang, Xiaoxia Sun, Maneesh Agrawala, Kiran Bhat, Tinghui Zhou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64913
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.28763

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CubePart: An Open-Vocabulary Part-Controllable 3D Generator Journal: TOG Conference: Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers; July 19–23, 2026; Los Angeles, CA, USA Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers (SIGGRAPH Conference Papers ’26), July 19–23, 2026, Los Angeles, CA, USA DOI: 10.1145/3799902.3811117 ISBN: 979-8-4007-2554-8/2026/07 CCS: Computing methodologies Artificial intelligence Yiheng Zhu, Kangle Deng, Jean-Philippe Fauconnier, Inaki Navarro, Daiqing Li, Ava Pun, Yinan Zhang, Peiye Zhuang, Xiaoxia Sun, Maneesh Agrawala, Kiran Bhat, Tinghui Zhou Abstract . Interactive 3D assets used in games and simulation are typically decomposed into specific semantic parts to support animation, physics, and scripted behaviors, yet most generative 3D models produce either monolithic meshes or arbitrary part decompositions that cannot be aligned with application-specific requirements. We present CubePart, a generative framework for open-vocabulary, part-controllable 3D mesh generation that exposes part structure as an explicit inference-time control signal. Given a global text prompt and a user-defined parts schema expressed as an open-ended list of part names, our method generates a set of meshes—one per schema element—that assemble into a coherent object while respecting the specified semantic structure. To enable this capability, we introduce a scalable data pipeline to construct a large open-vocabulary, part-labeled 3D dataset, along with a two-stage generative architecture that separates global shape synthesis from part-level decoding. We demonstrate that the resulting assets can be directly integrated into game engines and driven by animation and behavior scripts without manual post-processing. Figure 1. We propose CubePart, an open-vocabulary part-controllable 3D generator. (a) Given a text prompt and a schema defining part decomposition, CubePart synthesizes a multi-part 3D object where each component is a distinct, structurally complete mesh. (b) This controllable part-based generative framework directly facilitates the usage of the resulting assets for scripted or physically simulated behaviors (bottom row). Additionally, CubePart can accept an existing mesh as input, decomposing it into semantic multi-part meshes according to the input part schema (last column). Please refer to the video for animated visualizations. † † cc-license: by 1. Introduction 3D assets in modern games and interactive applications are rarely static. Vehicles require rotating wheels, characters must articulate, containers need to open and close, and many objects respond to physics or scripted events. In game engines, these behaviors are governed by simulation systems, animation rigs, and interaction scripts that operate on a pre-defined set of parts. For an asset to be functional, its mesh must be decomposed into specific semantic components that match the "schema" expected by the game’s code. Creating meshes that conform to a target part composition remains a largely manual process. Artists must decompose geometry into parts, assign consistent labels, and ensure that the resulting meshes assemble cleanly—an effort that scales poorly with asset diversity. While recent advances in 3D generative modeling have enabled the synthesis of complex geometries from text or image prompts, these methods either produce monolithic meshes without any explicit part structure ( 45 ; 44 ; 48 ) or an arbitrary set of parts ( 39 ; 28 ) ; the user has no control for aligning the resulting parts with the schema required by downstream game logic. For a developer with a game that specifically expects a car to be composed of four wheel parts and one chassis part, a model that generates a random set of part segments is as unhelpful as a model that generates a car as a single monolithic object. One might attempt to obtain part-level control through 2D grounding, for example, by using an image segmentation model ( 19 ; 2 ) to generate segmentation masks for mask-conditioned 3D part generation models like OmniPart ( 51 ) . However, a 2D mask cannot represent or control parts that are hidden from the input view. For instance, the rear tail of an animal cannot be specified or controlled from a single front-facing view. More fundamentally, 2D control signals are view-dependent and ambiguous when lifted to 3D, making them ill-suited for defining complete semantic decompositions of 3D objects. These limitations highlight a critical need for a 3D-native, schema-driven control interface that allows users to explicitly specify the semantic structure of an object during generation. Such control must also be flexible: different applications may require different decompositions of the same object. For example, one game may need car doors as separate parts to enable opening animations, while another may require the hood to be independently controllable to expose the engine. Fixed or closed-vocabulary part schemas cannot accommodate this diversity of downstream requirements. We argue that text, as a modality, provides a natural and universal interface for such control. Crucially, a text prompt can specify both a global description of the desired object and an explicit parts schema , an open-ended list of part names that serves as a structural blueprint for decomposing the object into semantically meaningful components. In this paper, we present CubePart, the first generative framework for open-vocabulary, part-controllable 3D mesh generation. Our system takes as input a global text prompt describing the object (e.g., “a jellyfish-themed race car”) together with a desired parts schema (e.g., {"car body", "front left wheel", …}). It outputs a set of distinct meshes, one per schema element, that jointly assemble into a coherent object. Because the generation is guided by the user-provided schema, the resulting assets can directly match the requirements of game engines and animation systems without manual intervention (as we demonstrate in Section 6 ). To support this capability, we introduce CubePart , a framework underpinned by a high-fidelity data engine and a novel multi-stage generative architecture. Our data engine leverages vision-language models (VLMs) and a novel 3D-aware "Set-of-Mark" ( 47 ) annotation strategy to curate a semantically grounded dataset of 462K assets and about 2M parts. Compared to existing 3D part datasets, ours is both larger scale (over 11 times larger than PartVerse-XL ( 10 ) ) and produces higher quality part labels required for precise open-vocabulary control. Building on this foundation, our architecture employs a two-stage diffusion process: the first stage generates a full mesh conditioned on both the prompt describing the object and the part schema, and the second stage decomposes the full mesh into corresponding parts specified by the schema while ensuring global geometric coherence through a novel cross-part attention mechanism with zero-initialized attention blocks. In summary, our main contributions include: • A scalable data engine for constructing open-vocabulary, part-labeled 3D datasets from unstructured meshes, leveraging VLMs for 3D-aware clustering and semantic captioning. • A schema-driven two-stage generative architecture that supports open-vocabulary, part-controllable 3D mesh generation while preserving global coherence across parts. • An end-to-end demonstration showing how the generated multi-part meshes can be integrated into game engines and driven by behavior scripts without manual post-processing. 2. Related Work 2.1. 3D Generative Models Recent progress in 3D generative modeling was initially driven by 2D-to-3D lifting approaches, most notably DreamFusion ( 37 ) , which introduced Score Distillation Sampling (SDS) to optimize implicit 3D representations using pretrained 2D diffusion priors. A large body of follow-up work ( 13 ; 27 ; 41 ; 30 ) adopts this paradigm, leveraging strong 2D priors to compensate for limited 3D data. Despite impressive visual quality, these methods rely on view-dependent image

Read the complete text