3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code

Yipeng Gao, Lei Shu, Genzhi Ye, Xi Xiong, Ameesh Makadia, Meiqi Guo, Laurent Itti, Jindong Chen · 2026

Read 3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code by Yipeng Gao, Lei Shu, Genzhi Ye, Xi Xiong, Ameesh Makadia, Meiqi Guo, Laurent Itti, Jindong Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01057.txt.

Introduction

Gao and co-authors benchmark AI agents that build 3D models by writing code in this 2026 paper. Procedural modeling through code gives assets that are exact, editable, and ready for engines. Neural 3D generators cannot match that control. But the craft demands deep skill in 3D software APIs and geometric reasoning in code.

Their benchmark, 3DCodeBench, asks 12 top vision language models to act as modelers. Each gets text and image references and must emit procedural code for 3D software. A sister platform, 3DCodeArena, ranks outputs by pairwise human taste. Pure metrics miss how shapes look, so human duels fill the gap. Two lessons lead the results, they report. Failures cluster on API mismatches between model memory and real software. Even good renders suffer loose or floating parts. More thinking budget and multi turn fixes lift scores overall. The call is for better procedural coding data to train on.

Catalog record

CREATORYipeng Gao, Lei Shu, Genzhi Ye, Xi Xiong, Ameesh Makadia, Meiqi Guo, Laurent Itti, Jindong Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE106 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.01057

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code Yipeng Gao, Lei Shu, Genzhi Ye, Xi Xiong, Ameesh Makadia, Meiqi Guo, Laurent Itti, Jindong Chen Abstract Procedural 3D modeling through code is emerging as a versatile paradigm, offering deterministic, engine-ready, and precisely editable assets that neural 3D generators inherently lack. Authoring such procedural content, however, demands deep expertise in 3D software APIs, parametric design, and code-level geometric reasoning. In this paper, we propose 3DCodeBench, a systematic benchmark for evaluating vision-language model (VLM) agents for procedural 3D generation in 3D modeling software. Specifically, 3DCodeBench evaluates how effectively 12 advanced VLMs can serve as procedural 3D modelers by translating text and image references into procedural code for 3D modeling software. Recognizing that automated metrics may not fully capture the perceptual quality of 3D shapes, we build 3DCodeArena, a ranking platform based on pairwise human preferences over generated 3D outputs. From extensive evaluations and results, we observe that: (1) Failures mostly arise from API mismatches, while successful renders still suffer from disconnected or floating 3D geometric components. (2) Test-time scaling, such as higher thinking budgets and multi-turn refinement, improves performance overall. Our findings highlight a critical need for high-quality procedural coding data to advance commercial VLMs. Furthermore, effective procedural 3D modeling requires a robust execution environment that provides high-fidelity feedback for iterative refinement. We release 3DCodeBench, including the curated large-scale dataset of multimodal (text/image) prompts, procedural code, 3D object triplets, evaluation protocol, and the public 3DCodeArena platform as a foundational toolkit for exploring VLM-based procedural 3D modelers. Project Page: 3dcodebench.com \uselogo \reportnumber \paperurl Figure 1 : Overview of 3DCodeBench. (Top Left) We explore the potential of Vision-Language Models to generate 3D objects via Procedural Code. (Bottom Left) The benchmark offers diverse, seed-addressable procedural test cases across a range of semantic categories. (Top Right) Qualitative comparisons of frontier VLMs (GPT-5.5, Gemini 3.1 Pro, Claude Opus 4.7) reveal varying capabilities in geometric reasoning. (Bottom Right) Alongside automated metrics, 3DCodeArena establishes Elo rankings via pairwise human-preference evaluations. 1 Introduction “Is God a Programmer, Not a Mathematician?” — Gregory J. Chaitin Procedural 3D modeling via coding is a critical pillar of modern digital creation ( 45 ; 28 ; 1 ; 8 ; 20 ) , driving immense commercial value across gaming, industrial design, and high-fidelity simulation environments for robotics training ( 17 ; 25 ; 16 ; 40 ; 41 ) . Nevertheless, authoring these 3D procedural assets remains a labor-intensive and formidable challenge. It demands that human designers have deep expertise in domain-specific coding syntax to manually define intricate meshes, complex geometric shapes, and realistic textures, and to perform precise parametric tuning to ensure physical plausibility. To address these bottlenecks, the rapid advancement of Vision-Language Models (VLMs) and coding agents has paved the way for automating this complex development process. Recognizing this automation potential, the community has been actively exploring ways to enable VLM agents in 3D creative artistry. For instance, Anthropic announced the Claude for Creative Work ( 5 ) initiative to drive 3D modeling software like Blender via Python APIs. Prior to this, an active ecosystem of Model Context Protocol (MCP) servers and function-calling agents ( 9 ; 2 ; 19 ; 39 ; 43 ; 37 ) has already turned frontier VLMs into natural-language drivers for end-to-end procedural modeling. On the research side, a line of works ( 47 ; 27 ; 35 ; 52 ; 51 ; 34 ) also builds LLM-driven agents to author procedural code or compose retrieved assets. However, the community lacks a standardized, reliable benchmark for examining the model capabilities. Evaluating the proficiency of VLMs as procedural modelers remains an open challenge due to four core limitations in existing methodologies. First, there is a severe lack of aligned procedural data. While repositories like ShapeNet ( 10 ) and Objaverse ( 14 ) provide vast collections of static meshes, they lack the underlying procedural code needed to measure generative agency. Second, current environments fail to capture real-world geometric complexity. For instance, BlenderGym ( 26 ) focuses heavily on editing pre-built scenes rather than on from-scratch generation, and VoxelCodeBench ( 54 ) limits construction to simplistic voxel grids. Third, existing frameworks ignore the iterative nature of 3D design by relying on single-shot evaluations and omitting the agentic refinement loops that are crucial to real workflows. Finally, the field lacks standardized evaluation metrics. Current models are tested on ad hoc prompts without a shared reference set, and the community lacks comprehensive perception metrics alongside a systematic 3D code arena for human preference voting. Consequently, the research community lacks a comprehensive, standardized benchmark to rigorously measure the performance of frontier VLMs at generating 3D assets from code. In this paper, we introduce 3DCodeBench (Figure 1 ), a comprehensive benchmark and evaluation framework that directly addresses these four limitations. First, to address data scarcity, we extract procedural factories from Infinigen 40 ; 41 and process them using an agentic curation pipeline with human verification. This yields a high-quality dataset of 26K text/image prompts \leftrightarrow standalone code \leftrightarrow 3D object triplets across 212 categories (Figure 1 , bottom left). Second, our dataset captures real-world geometric complexity rather than constructing objects using simple primitive shapes. Classes such as flying bird, crab, and dragonfly—each averaging over 400 lines of code—push models far beyond simplistic shape primitives (Figure 1 , top right). Third, to reflect the iterative nature of 3D design, our framework moves beyond single-shot prompting by treating VLMs as active agents . We systematically evaluate their abilities in controlled multi-turn scenarios that include execution-error feedback, retries, visual self-critique, and API documentation augmentation. Finally, we establish a standardized and robust evaluation protocol. We extensively analyze 12 advanced VLMs (Gemini ( 24 ) , Claude ( 4 ) , and GPT ( 38 ) series) using comprehensive automated metrics spanning executability, perceptual view similarity (SigLIP-2 ( 49 ) , DINOv3 ( 46 ) ), and 3D geometric alignment (Chamfer distance ( 21 ) , Uni3D ( 55 ) ). To complement these metrics, we launch 3DCodeArena (Figure 1 , bottom right), a public arena that collects pairwise human preferences to produce reliable Elo rankings. In summary, 3DCodeBench provides a comprehensive benchmark for evaluating the procedural 3D generation capabilities of VLM agents. Through extensive evaluations, we mainly find: (1) Physical Plausibility remains the primary bottleneck beyond Executability: Models frequently produce disconnected parts and incorrect structural alignments, revealing a critical lack of physical-world understanding. (2) Test-Time Scaling: Multi-turn refinement improves performance using deterministic execution feedback from Blender. This demonstrates that designing an appropriate agentic harness to process environmental feedback is crucial to unlocking the model’s full potential. Our contributions are summarized as: • 3DCodeBench: A standardized, VLM-driven procedural 3D modeling benchmark. Spanning a diverse range of 212 object classes and 26K 3D object–code pairs, it is curated via the proposed agentic pipeline with human feedback and paired with comprehensive evaluation protocols. • 3DCodeArena: A systematic human-preference platform designed to evaluate the perceptual quality and aesthetic appeal of generated 3D shapes through pairwise Elo rankings. • Extensive VLM Analysis & Insights: A holistic evaluation of 12 advanced VLMs that identifies critical failure modes and establishes multi-turn agentic refinement as the primary driver for successful procedural 3D generation. 2 Related Work

Read the complete text