SR-Platform An Agentic Pipeline for Natural Language-Driven Robot Simulation Environment Synthesis strikerobot.ai

Ben Wei Lim, Minh Duc Le, Thang Truong, Thanh Nguyen Canh, [4pt] Strike Robotics [email protected] · 2026

Read SR-Platform An Agentic Pipeline for Natural Language-Driven Robot Simulation Environment Synthesis strikerobot.ai by Ben Wei Lim, Minh Duc Le, Thang Truong, Thanh Nguyen Canh, [4pt] Strike Robotics [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14700.txt.

Introduction

Lim, Le, Truong and Canh present SR-Platform, an agentic system that turns plain-language descriptions into executable, physically valid MuJoCo robot simulation scenes. An orchestrator plans the scene. An asset forge fetches or generates 3D geometry. A layout architect places objects and checks constraints. A bridge layer assembles the final scene with the chosen robot model. In thirty days of production use, five-object scenes took a median of about fifty seconds end to end.

Catalog record

CREATORBen Wei Lim, Minh Duc Le, Thang Truong, Thanh Nguyen Canh, [4pt] Strike Robotics [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74704
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.14700

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SR-Platform An Agentic Pipeline for Natural Language-Driven Robot Simulation Environment Synthesis strikerobot.ai Ben Wei Lim, Minh Duc Le, Thang Truong, Thanh Nguyen Canh, [4pt] Strike Robotics [email protected] Abstract Generating robot simulation environments remains a major bottleneck in simulation-based robot learning. Constructing a training-ready MuJoCo scene typically requires expertise in 3D asset modeling, MJCF specification, spatial layout, collision avoidance, and robot-model integration. We present SR-Platform, a production-deployed agentic system that converts free-form natural language descriptions into executable, physically valid MuJoCo environments. SR-Platform decomposes scene synthesis into four stages: an LLM-based orchestrator that converts user intent into a structured scene plan; an asset forge that retrieves cached assets or generates new 3D geometry through LLM-to-CadQuery synthesis; a layout architect that assigns object poses and verifies industrial constraints; and a bridge layer that assembles the final MJCF scene and merges the selected robot model. The system is deployed as a nine-service Docker stack with WebSocket progress streaming, MinIO-backed mesh storage, Qdrant-based semantic asset retrieval, Redis job state, and InfluxDB telemetry. Using 30 days of production telemetry covering 611 successful LLM calls, SR-Platform generates five-object scenes with a median end-to-end latency of approximately 50 s, while cache-accelerated scenes complete in approximately 30–40 s. The asset forge shows an 11.3% first-attempt retry rate with automatic recovery, and cached asset retrieval removes per-object LLM calls for previously generated object types. These results show that agentic scene synthesis can reduce the manual effort required to create diverse robot training environments, enabling users to produce executable MuJoCo scenes from plain English prompts in under one minute. Figure 1 : SR-Platform web interface generating a robot simulation environment from a natural-language scene prompt. The interface combines prompt refinement, robot selection, real-time generation progress, and browser-based MuJoCo scene visualization. 1 Introduction Robot learning at scale requires large collections of diverse, physically plausible simulation environments. Reinforcement learning, imitation learning, manipulation benchmarking, and sim-to-real evaluation all depend on environments that reflect the geometry, object diversity, and spatial constraints of the target deployment domain ( 11 ; 20 ) . In practice, however, constructing these environments remains a major bottleneck. A robotics engineer must locate or author 3D assets, convert them into simulator-compatible mesh formats, write a valid MJCF scene description, position objects without collisions, satisfy workspace and clearance constraints, and merge a robot model with an appropriate spawn pose. This process requires expertise in robotics, 3D modeling, and simulator-specific file formats, making it difficult to scale environment generation to the diversity required for robust policy training. MuJoCo has become a widely used simulator for robot learning because it provides efficient rigid-body dynamics, articulated-body simulation, and a compact XML-based modeling format ( 16 ) . However, the MJCF representation is designed for precision and simulator control, not for natural authoring by non-specialist users. A user may be able to describe a desired workspace in plain English, such as an assembly cell, warehouse aisle, laboratory bench, or kitchen-like manipulation environment, but this description must still be transformed into structured objects, geometric assets, valid spatial layouts, and simulator-executable XML. The gap between a natural-language scene description and a runnable robot simulation therefore, remains substantial. Recent advances in large language models (LLMs) suggest a promising path toward automating parts of this workflow. LLMs have demonstrated strong capabilities in structured output generation ( 2 ) , code synthesis ( 4 ) , and embodied or spatial reasoning ( 5 ) . In robotics, LLMs have been used to generate reward functions ( 10 ) , infer task plans ( 1 ) , produce robot-control programs ( 9 ) , and synthesize task-environment descriptions ( 18 ) . These systems show that language models can provide useful semantic reasoning for robotics workflows. However, most prior approaches either assume a fixed object library, stop at symbolic scene or task descriptions, or require manual intervention before the result can be executed in a physics simulator. As a result, the end-to-end problem of converting an unconstrained language prompt into a physically valid, simulator-ready robot scene remains insufficiently addressed. To address these challenges, this paper presents SR-Platform, a production-deployed agentic system for natural-language-driven robot simulation environment synthesis. Given a free-form workspace description, SR-Platform generates an executable MuJoCo scene with retrieved or synthesized assets, spatially assigned object poses, an integrated robot model, and spawn coordinates. Rather than using an LLM as a monolithic scene generator, SR-Platform embeds LLM calls within a typed, auditable, and cache-aware pipeline that separates semantic planning, asset generation, layout reasoning, and simulator assembly. Requested objects are first matched against a vector database of previously generated assets; if no suitable match is found, an LLM synthesizes CadQuery code to generate a new simulator-compatible mesh, which is then indexed for future reuse. Industrial validity is also checked during layout generation through constraints such as electrical clearance, fire-safety egress, machinery safety distance, and ventilation clearance. As shown in Fig. 1 , a user describes an industrial workspace through a web interface and receives a generated scene that can be visualized directly in the browser. The system is evaluated using 30 days of production telemetry covering 611 successful LLM calls, with additional object-generation benchmarks and model-routing results reported in the appendix. The main contributions of this paper can be organized as follows: 1. An end-to-end agentic pipeline is introduced for converting natural-language workspace descriptions into executable MuJoCo simulation scenes with robot models, mesh assets, object poses, and spawn coordinates. 2. A cache-aware asset forge is presented that combines semantic retrieval with LLM-to-CadQuery generation, reducing redundant LLM calls for previously generated object types. 3. Industrial layout verification is integrated into the scene synthesis process, enabling generated layouts to be checked against clearance and safety constraints before MJCF assembly. 4. Production telemetry from 611 successful LLM calls is reported, including per-stage latency, retry behavior, throughput limits, and cache-accelerated end-to-end generation estimates. 2 Related Work 2.1 Simulation Environment Generation Simulation environment generation has long been studied as a way to reduce the cost of robot learning and evaluation. Existing platforms provide either high-fidelity simulation backends or curated environment collections, but they often require substantial manual effort to construct new scenes. ThreeDWorld provides an interactive physical simulation environment with rich multimodal perception, but scene authoring remains largely manual ( 6 ) . BEHAVIOR-1K introduces a large set of household activities and corresponding environments for embodied AI, but its scope is tied to a predefined benchmark and simulator-specific assets ( 8 ) . RoboGen uses generative models to create task-environment pairs for robot learning, but still relies on curated object resources and does not focus on production deployment for industrial scene synthesis ( 18 ) . In contrast, SR-Platform targets on-demand generation of industrial robot workspaces from natural language and produces directly executable MuJoCo scenes rather than only task descriptions or benchmark environments. 2.2 Language Models for Robotics Large language models have increasingly been used as semantic reasoning components in robotics systems. SayCan grounds language instructions in robot affordances to select feasible actions ( 1 ) , while Code as Policies uses language models to generate robot-control programs from natural-language instructions ( 9 ) . Eureka demonstrates that LLMs can synthesize reward functions for reinforcement learning ( 10 ) , and Voyager shows how an LLM-based agent can

Read the complete text