PhysCodeBench: Benchmarking Physics-Aware Symbolic Simulation of 3D Scenes via Self-Corrective Multi-Agent Refinement

Tianyidan Xie, Peiyu Wang, Yuyi Qian, Yuxuan Wang, Rui Ma, Ying Tai, Song Wu, Qian Wang, Lanjun Wang, Zili Yi · 2026

Read PhysCodeBench: Benchmarking Physics-Aware Symbolic Simulation of 3D Scenes via Self-Corrective Multi-Agent Refinement by Tianyidan Xie, Peiyu Wang, Yuyi Qian, Yuxuan Wang, Rui Ma, Ying Tai, Song Wu, Qian Wang, Lanjun Wang, Zili Yi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23580.txt.

Introduction

Xie and colleagues introduce PhysCodeBench, the first benchmark that tests whether language models can turn physics descriptions into code that simulates the right physics, not merely code that runs. Its 1,200 expert-checked examples span four domains, and scoring checks conservation laws and expert assertions straight from the engine state. Their multi-agent correction method nearly triples the best proprietary baseline's pass rate.

Catalog record

CREATORTianyidan Xie, Peiyu Wang, Yuyi Qian, Yuxuan Wang, Rui Ma, Ying Tai, Song Wu, Qian Wang, Lanjun Wang, Zili Yi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75249
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.23580

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PhysCodeBench: Benchmarking Physics-Aware Symbolic Simulation of 3D Scenes via Self-Corrective Multi-Agent Refinement Tianyidan Xie, Peiyu Wang, Yuyi Qian, Yuxuan Wang, Rui Ma, Ying Tai, Song Wu, Qian Wang, Lanjun Wang, Zili Yi Abstract Physics-aware symbolic simulation of 3D scenes is critical for robotics, embodied AI, and scientific computing, requiring models to understand natural language descriptions of physical phenomena and translate them into executable simulation environments. While large language models (LLMs) excel at general code generation, they struggle with the semantic gap between physical descriptions and simulation implementation. We introduce PhysCodeBench, the first comprehensive benchmark for evaluating physics-aware symbolic simulation, comprising 700 manually-crafted diverse samples across mechanics, fluid dynamics, and soft-body physics with expert annotations. Our evaluation framework measures both code executability and physical accuracy through automated and visual assessment. Building on this, we propose a Self-Corrective Multi-Agent Refinement Framework (SMRF) with three specialized agents (simulation generator, error corrector, and simulation refiner) that collaborate iteratively with domain-specific validation to produce physically accurate simulations. SMRF achieves 67.7 points overall performance compared to 36.3 points for the best baseline among evaluated SOTA models, representing a 31.4-point improvement. Our analysis demonstrates that error correction is critical for accurate physics-aware symbolic simulation and that specialized multi-agent approaches significantly outperform single-agent methods across the tested physical domains. 1 Introduction Physics-aware symbolic simulation of 3D scenes has become a cornerstone of modern robotics, embodied AI, and scientific computing. From simulating robot dynamics for motion planning to modeling fluid flows for climate prediction, the ability to accurately encode physical laws in executable code is increasingly critical. However, creating such code remains a formidable challenge that requires both deep programming expertise and sophisticated understanding of physical laws. By symbolic simulation, we refer to code generation that creates symbolic representations of physical 3D scenes through executable simulation code, enabling computational modeling of complex physical phenomena. The complexity of physics-aware symbolic simulation stems from multiple interacting challenges that single-agent approaches struggle to address simultaneously. First, models must translate natural language descriptions of physical phenomena into syntactically correct code that executes without runtime errors. Second, they must ensure that the generated code accurately implements the intended physical laws and parameters. Third, they must handle simulation boundary conditions and numerical stability constraints that are crucial for realistic behavior. These challenges are fundamentally different in nature (syntactic correctness, physical accuracy, and numerical stability), making it difficult for single models to optimize across all dimensions effectively. Consider seemingly simple tasks like creating a cube sliding down an incline, simulating a ball bouncing on a trampoline, or generating raindrops falling on a surface. These scenarios require precise implementation of gravity, elasticity, surface tension, and collision dynamics. As illustrated in Fig. 1 , vanilla LLMs struggle with these tasks, producing failed simulations, implementation bugs, inaccurate results, and incorrect physics parameters. Our preliminary evaluation reveals that state-of-the-art models can only produce executable code at relatively low success rates. Even when the code executes successfully, GPT-4o [ 15 ] and Claude-3.5-Sonnet [ 1 ] frequently generate simulations where object interactions deviate from the instruction descriptions, producing inappropriate physics parameters, inaccurate boundary conditions, and incorrect object positioning that result in behaviors inconsistent with the intended physical scenarios. These models demonstrate a fundamental challenge: achieving basic code execution is only the first hurdle, as the generated simulations often fail to match the physical phenomena described in the natural language instructions. This failure points to a fundamental limitation: existing code generation models excel at syntactic correctness but struggle with the semantic requirements of physical accuracy and the practical constraints of numerical implementation. While general code generation has been extensively studied [ 6 , 25 ] , the specialized domain of physics-aware symbolic simulation presents unique challenges that remain largely unaddressed. Current benchmarks focus on general programming tasks and lack the domain-specific evaluation criteria necessary to assess physics-aware symbolic simulation quality. Figure 1 : PhysCodeBench and SMRF enable accurate physics-aware symbolic simulation of 3D scenes. Our Self-Corrective Multi-Agent Refinement Framework (SMRF) achieves 67.7 points overall performance, significantly outperforming state-of-the-art models (best baseline: 36.3 points). The framework correctly implements complex physics like conservation laws, surface tension, and collision dynamics, resulting in a remarkable 31.4-point improvement over existing approaches. The multi-faceted nature of this challenge requiring expertise in scene composition, physics understanding, and spatial reasoning motivates a decomposition approach where specialized agents can focus on their respective strengths. However, the lack of comprehensive benchmarks for physics-aware symbolic simulation has hindered progress in this critical area. We address this gap with three key contributions: • PhysCodeBench : The first comprehensive benchmark specifically designed for physics-aware symbolic simulation of 3D scenes, featuring manually-crafted examples across solid mechanics, fluid dynamics, and soft-body physics domains with expert annotations. • Self-Corrective Multi-Agent Refinement Framework (SMRF) : A novel multi-agent system that decomposes physics-aware symbolic simulation of 3D scenes into specialized components: a simulation generator for initial implementation, an error corrector for diagnosing and fixing errors, and a simulation refiner for optimization and preference alignment. This achieves a remarkable 31.4-point improvement over state-of-the-art models. • Comprehensive Evaluation and Analysis : Through extensive experiments and user studies, we demonstrate that SMRF not only generates more physically accurate simulations but also produces code that domain experts rate significantly higher in quality and practical utility. Our experiments demonstrate that decomposing physics-aware symbolic simulation into specialized agents (each focusing on code generation, error correction, and code refinement) enables more accurate implementation of complex physical phenomena. This work establishes a new foundation for AI systems that can bridge the gap between physical understanding and executable implementation, a capability that will be crucial as autonomous systems become more prevalent in physical environments. 2 Related Work Our work bridges physical simulation understanding, code generation, and multi-agent systems. We review key advances and identify gaps that PhysCodeBench and SMRF address. 2.1 Physics Simulation and Symbolic Code Generation Physical reasoning has been studied through video prediction tasks like Physion [ 4 ] and CLEVRER [ 31 ] , which focus on understanding phenomena rather than code implementation. Domain-specific benchmarks include PDEBench [ 26 ] for differential equations and PHYBench [ 22 ] for physics problem solving, but none address physics-aware symbolic simulation. Recent work explores LLM-based simulation generation. Mind’s Eye [ 17 ] uses physics simulation to enhance language model reasoning through bilevel optimization with MuJoCo [ 28 ] . Robotics applications like SimGen [ 32 ] , VoxPoser [ 13 ] , and Code as Policies [ 2 ] generate simulation environments for robot task planning. However, these focus on high-level task planning within constrained domains, generating sequences of pre-existing components. Our work addresses generating fundamental physics simulation code that symbolically represents physical laws accurately across broader 3D scene domains. Code generation has advanced with models like Codex [ 6 ] , CodeLlama [ 25 ] , DeepSeek-R1 [ 11 ] , and Qwen-2.5 [ 30 ] . These are evaluated on general programming benchmarks (HumanEval [ 6 ] , MBPP [ 3 ] ) but lack domain-specific evaluation for physics simulation, where syntactic correctness must couple with physical accuracy. Fine-tuning approaches like WizardCoder [ 19 ] show specialization improves domain performance, motivating our multi-agent approach. 2.2 Multi-Agent Systems and Preference Alignment Multi-agent frameworks like AutoGen [ 29 ]

Read the complete text