Sel3DCraft: Interactive Visual Prompts for User-Friendly Text-to-3D Generation

\authororcid Nan Xiang † 0009-0008-8478-2391, \authororcid Tianyi Liang † 0000-0001-8372-8379, \authororcid Haiwen Huang0009-0005-9443-3855, \authororcid Shiqi Jiang0000-0002-8175-874X, Hao Huang, \authororcid Yifei Huang0000-0002-3077-0175, \authororcid Liangyu Chen0009-0005-0243-3613, \authororcid · 2025

Read Sel3DCraft: Interactive Visual Prompts for User-Friendly Text-to-3D Generation by \authororcid Nan Xiang † 0009-0008-8478-2391, \authororcid Tianyi Liang † 0000-0001-8372-8379, \authororcid Haiwen Huang0009-0005-9443-3855, \authororcid Shiqi Jiang0000-0002-8175-874X, Hao Huang, \authororcid Yifei Huang0000-0002-3077-0175, \authororcid Liangyu Chen0009-0005-0243-3613, \authororcid (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00428.txt.

Introduction

Xiang and co-authors built Sel3DCraft, a system that helps users guide text to 3D generation with pictures instead of words alone. Typing prompts and hoping for a good model is slow, and judging a 3D result needs views from many sides. Their 2025 paper aims to turn that blind trial and error into a guided visual process.

The system mixes search and generation to offer varied starting choices. It scores models using views from several angles and measures meant to match expert judgment. A visual workspace then helps users spot flaws and refine the prompt step by step. Tests and user studies showed stronger support for designer creativity than rival systems.

Catalog record

CREATOR\authororcid Nan Xiang † 0009-0008-8478-2391, \authororcid Tianyi Liang † 0000-0001-8372-8379, \authororcid Haiwen Huang0009-0005-9443-3855, \authororcid Shiqi Jiang0000-0002-8175-874X, Hao Huang, \authororcid Yifei Huang0000-0002-3077-0175, \authororcid Liangyu Chen0009-0005-0243-3613, \authororcid
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-00428

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sel3DCraft: Interactive Visual Prompts for User-Friendly Text-to-3D Generation \authororcid Nan Xiang † 0009-0008-8478-2391, \authororcid Tianyi Liang † 0000-0001-8372-8379, \authororcid Haiwen Huang0009-0005-9443-3855, \authororcid Shiqi Jiang0000-0002-8175-874X, Hao Huang, \authororcid Yifei Huang0000-0002-3077-0175, \authororcid Liangyu Chen0009-0005-0243-3613, \authororcid Changbo Wang0000-0001-8940-6418, and \authororcid Chenhui Li0000-0001-9835-2650 Abstract Text-to-3D (T23D) generation has transformed digital content creation, yet remains bottlenecked by blind trial-and-error prompting processes that yield unpredictable results. While visual prompt engineering has advanced in text-to-image domains, its application to 3D generation presents unique challenges requiring multi-view consistency evaluation and spatial understanding. We present Sel3DCraft , a visual prompt engineering system for T23D that transforms unstructured exploration into a guided visual process. Our approach introduces three key innovations: a dual-branch structure combining retrieval and generation for diverse candidate exploration; a multi-view hybrid scoring approach that leverages MLLMs with innovative high-level metrics to assess 3D models with human-expert consistency; and a prompt-driven visual analytics suite that enables intuitive defect identification and refinement. Extensive testing and user studies demonstrate that Sel3DCraft surpasses other T23D systems in supporting creativity for designers. \onlineid 1770 \preprinttext To appear in IEEE Transactions on Visualization and Computer Graphics. \vgtcpapertype Analytics & Decisions \authorfooter N. Xiang † , T. Liang † , H. Huang, S. Jiang, H. Huang, Y. Huang, L. Chen, C. Wang, and C. Li are with East China Normal University. † These authors contributed equally. Chenhui Li and Liangyu Chen are corresponding authors. Email: {51275902056, tyliang, hwhuang, 52265901032, 10215102453}@stu.ecnu.edu.cn, [email protected], [email protected], {cbwang,chli}@cs.ecnu.edu.cn. \teaser The user interface of Sel3DCraft includes the Model Input View (A); the Image Browser View (B), which covers the generated and retrieved images (B1), the multi-view hybrid-level scoring function (B2), and the scoring rationale (B3); the 3D Viewer View (C); the Text Exploration View (D); and the Keyword Contribution View (E). \nocopyrightspace Introduction keywords Prompt engineering, text-to-3D generation, shape exploration, visualization design, visual perception. Text-to-3D (T23D) generation, which transforms textual prompts directly into 3D models, has revolutionized digital content creation across industries such as virtual reality, gaming, film, and animation. Despite reducing model creation time, a critical gap remains between user expectations and generated output quality. Current T23D tools operate in an end-to-end “black-box” manner (see Fig. 1 ). They offer limited control over the generation process and frequently produce models that fail to meet expectations [ 52 , 28 , 18 ] . The absence of effective prompt recommendation mechanisms forces users into costly trial-and-error cycles. This disrupts creative workflows and limits the technology’s practical utility for professional designers. Recent advances in prompt engineering have shown promise in text-to-image domains [ 13 , 5 ] , enabling efficient candidate exploration and prompt refinement. However, translating these approaches to T23D generation introduces unique challenges. Unlike 2D images, 3D models require physical structural integrity. Moreover, effective evaluation must consider geometry, texture, and coherence across multiple views—a multi-dimensional quality assessment problem beyond current prompt engineering frameworks. To bridge this gap, can we develop a visual approach that transforms costly trial-and-error into structured exploration for text-to-3D generation? Our formative study ( Section 2 ) with professional designers reveals key inefficiencies: limited model options, view inconsistency issues, and lack of effective evaluation and iteration mechanisms. Figure 1 : The workflow of text-to-3D generation tools involves four steps: (1) users input text or images and receive four rotating previews of coarse models; (2) a single model is selected for time-intensive refinement; (3) the final model is exported; and (4) if unsatisfied, users can iterate on prompts during generation or refinement. Our system enhances the first and fourth steps, improving efficiency in prompt engineering and model selection. We present Sel3DCraft , an interactive visual prompt engineering system for T23D generation. The system features a dual-branch structure for initial candidate synthesis ( Section 3.1 ). It leverages multi-view images as a 3D visual medium and integrates Multi-modal Large Language Models (MLLMs) for multi-dimensional assessment ( Section 3.2 ). Notably, the prompt recommendation mechanism ( Section 3.3 ) follows the what-you-see-is-what-you-get paradigm. Furthermore, we propose a cross-modal visual analytics representation ( Section 4.2 ) that integrates three coordinated components: (1) multi-view satellite charts for clustering multi-view images via quality metric projection, (2) multi-view hybrid-level scoring heatmaps for model evaluation and defect localization, and (3) a prompt-driven treemap wordle for diverse semantic recommendation. Validation of the semantic scoring approach ( Section 5 ) and user study ( Section 6 ) demonstrate Sel3DCraft ’s potential and superior creativity support. Compared to baselines, Sel3DCraft reduces model creation time by 70.5% (118.83s vs 402.17s) and prompt iterations by 66.2%, with significantly higher model quality ratings (4.58 vs 2.46, Fig. 7 ). Our contributions are threefold: • We introduce T23D visual prompt engineering as a novel task. For the first time, we leverage MLLMs to evaluate multi-view images of 3D models in Visual Analytics Science and Technology . This integration enables iterative refinement and significantly improves both the process and the outcome of 3D generation. • We propose a cross-modal multi-view visual representation approach for T23D generation. A user study validated the usefulness of these visualizations in boosting user engagement. • We present Sel3DCraft , an interactive visual system unifying retrieval and generation. Through user evaluations, we demonstrate that the system achieves faster model creation while maintaining comparable quality. 1 Related Work 1.1 Visual Prompt Engineering Prompt engineering has evolved significantly with advances in large language models [ 38 , 6 ] and text-to-image generation systems [ 36 , 44 ] . Early work focused on automated text-to-text prompt optimization through methods like AutoPrompt [ 47 ] and template generation [ 14 ] . As tasks increased in complexity, interactive systems like PromptIDE [ 49 ] and PromptChainer [ 59 ] enabled collaborative human-AI refinement of prompts. Visual prompt engineering introduces new challenges for text-to-image models. Studies by Liu [ 29 ] and Oppenlaender [ 37 ] established foundational taxonomies for prompt modifiers. Recent advancements such as Li et al. [ 24 ] developed a visual analytics system to guide image captioning and uncover biases, aligning with AI-assisted content creation goals. Wang [ 55 ] demonstrated how visual prompt engineering democratizes content creation, while PromptMagician [ 13 ] refines prompts through multi-level visualizations. However, current prompt engineering is largely confined to 2D. To overcome this limitation, we propose a novel approach leveraging multi-view images as an intermediate representation. The key innovation is our semantic-based treemap wordle, which introduces T23D image-text matching [ 22 ] to link prompts to multi-view images. This allows users to intuitively perceive how different parts of prompts contribute to specific 3D regions and iteratively refine inputs by selecting relevant keywords. 1.2 Creativity Support Systems Creativity support systems (CSS) have been extensively studied in HCI by leveraging generative models and interactive tools to enhance human creativity in design workflows. In text-to-text and text-to-image generation, systems such as TaleBrush [ 11 ] combined sketches with generative language models to support storytelling. PromptPaint [ 10 ] , on the other hand, enabled users to guide image generation through paint-like interactions, highlighting CSS’s potential to inspire both visual and textual creativity. In interface design, Scout [ 50 ] supported rapid exploration of layout alternatives through high-level constraints, underscoring CSS’s role in enhancing design efficiency and fostering innovation. For visual search, GenQuery [ 48

Read the complete text