Verti-Bench : A General and Scalable Off-Road Mobility Benchmark for Vertically Challenging Terrain

Tong Xu, Chenhui Pan, Madhan B. Rao, Aniket Datar, Anuj Pokhrel, Yuanjie Lu, Xuesu Xiao · 2025

Read Verti-Bench : A General and Scalable Off-Road Mobility Benchmark for Vertically Challenging Terrain by Tong Xu, Chenhui Pan, Madhan B. Rao, Aniket Datar, Anuj Pokhrel, Yuanjie Lu, Xuesu Xiao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11426.txt.

Introduction

Tong Xu and co-authors build a proving ground for robots that must climb brutally rough ground. Off-road machines have improved fast, but judging them fairly is hard: vehicles differ, soils differ, and driving skill only shows live, not in stored data. Their answer is Verti-Bench, a benchmark set in simulation on terrain that tilts steeply and fights back.

The suite holds one hundred off-road worlds and a thousand driving tasks across rock, mud-like deformable soil, and large natural obstacles. It fits vehicles of many sizes and drive styles, and ships with expert runs, random runs, and failure clips of rollovers and bog-downs. A gym-style interface invites learning-based drivers to train inside it. The team ran ten current systems through the course to map where the field stands.

Catalog record

CREATORTong Xu, Chenhui Pan, Madhan B. Rao, Aniket Datar, Anuj Pokhrel, Yuanjie Lu, Xuesu Xiao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-11426

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Verti-Bench : A General and Scalable Off-Road Mobility Benchmark for Vertically Challenging Terrain Tong Xu, Chenhui Pan, Madhan B. Rao, Aniket Datar, Anuj Pokhrel, Yuanjie Lu, Xuesu Xiao Abstract Recent advancement in off-road autonomy has shown promises in deploying autonomous mobile robots in outdoor off-road environments. Encouraging results have been reported from both simulated and real-world experiments. However, unlike evaluating off-road perception tasks on static datasets, benchmarking off-road mobility still faces significant challenges due to a variety of factors, including variations in vehicle platforms and terrain properties. Furthermore, different vehicle-terrain interactions need to be unfolded during mobility evaluation, which requires the mobility systems to interact with the environments instead of comparing against a pre-collected dataset. In this paper, we present Verti-Bench, a mobility benchmark that focuses on extremely rugged, vertically challenging off-road environments. 100 unique off-road environments and 1000 distinct navigation tasks with millions of off-road terrain properties, including a variety of geometry and semantics, rigid and deformable surfaces, and large natural obstacles, provide standardized and objective evaluation in high-fidelity multi-physics simulation. Verti-Bench is also scalable to various vehicle platforms with different scales and actuation mechanisms. We also provide datasets from expert demonstration, random exploration, failure cases (rolling over and getting stuck), as well as a gym-like interface for reinforcement learning. We use Verti-Bench to benchmark ten off-road mobility systems, present our findings, and identify future off-road mobility research directions. Verti-Bench project website can be found at https://cs.gmu.edu/ ~ xiao/Research/Verti-Bench . Fig. 1: Based on high-fidelity multi-physics simulation, Verti-Bench encapsulates a variety of off-road features, i.e., geometry, semantics, and obstacles, and vehicles and can be scaled to different sizes. 100 off-road environments and 1000 navigation tasks with millions of off-road features can objectively and quantitatively evaluate off-road vehicle mobility on extremely rugged, vertically challenging terrain. I Introduction Off-road autonomous mobile robots present unique opportunities in search and rescue, environment monitoring, scientific exploration, and other application domains. However, off-road autonomy presents unique challenges to both robot perception and vehicle mobility that distinguish from structured on-road environments [ 1 , 2 , 3 ] , such as variable geometry, deformable surfaces, and natural obstacles. Considering recent advancement, standardized benchmarks for off-road autonomy are necessary to objectively and quantitatively evaluate and compare the progress of the off-road robotics community. Unlike off-road perception, evaluating off-road mobility is more difficult. A plethora of off-road perception datasets [ 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 ] are available to provide static ground truth labels to evaluate against perception systems’ outputs, since the robot actions can be recorded and fed into the perception systems, not generated by them (if actions are necessary at all). However, mobility systems produce new actions to drive robots to different states than those collected in the dataset, i.e., distribution shift, and thus cannot be evaluated by comparing against a static dataset. Therefore, vehicle-terrain interactions need to be unfolded during off-road mobility evaluation, which creates difficulty in standardization across research groups. Considering a lack of standard off-road mobility evaluation, researchers currently develop their own benchmarks to evaluate their mobility systems and re-implement previous systems to compare against. Such a practice, however, leads to ad-hoc evaluation in a few aspects: The evaluation platforms, either a physical or a simulated robot, vary in terms of robot size, weight, actuation mechanism, and/or levels of off-road physics simulation fidelity; The evaluation environments range from handcrafted off-road simulation features [ 12 , 13 , 14 , 15 , 16 , 17 ] , small-scale indoor testbeds [ 18 , 19 , 20 , 21 ] , enclosed outdoor tracks [ 22 , 23 , 24 , 25 , 26 ] , and large-scale real-world testing facilities [ 6 , 27 , 5 ] ; Re-implementation of previous approaches on one’s own robot is not only laborious, but also subject to misinterpretation of implementation details. Therefore, a standard off-road mobility benchmark which is general and scalable to all these aspects is desired for the off-road mobility research community. In this work, we present Verti-Bench (Fig. 1 ), a general and scalable off-road mobility benchmark that focuses on extremely rugged, vertically challenging terrain with a variety of unstructured off-road features. The main goal of Verti-Bench is to directly compare the performances of different off-road mobility systems. Based on a high-fidelity multi-physics dynamics simulator, Chrono [ 28 ] , Verti-Bench encapsulates variations in four orthogonal dimensions: Using the Sliced Wasserstein Autoencoder (SWAE) [ 29 ] and real-world off-road terrain data, off-road geometry is represented as a diverse set of 2.5D elevation maps; Ten terrain semantics classes, including seven rigid and three deformable, are designed with different distributions of physics parameters, e.g., friction coefficient, cohesive effect, and soil stiffness; Different types of natural obstacles, e.g., boulders and vegetations, are randomly distributed based on different densities; A set of off-road vehicles with different scales (from 1/10th to full scale) and actuation mechanisms (4-, 6-, and 8-wheeled and 2-tracked chassis, single- and double-wishbone, multilink, toebar leaf-spring, and special tensioning suspensions, as well as pitman-arm, rack-and-pinion, toebar, bellcrank/rotary arm, and differential steering) are provided, with the possibility of adding customized vehicles. Using Verti-Bench, we also provide datasets from expert demonstration, random exploration, failure cases (rolling over and getting stuck), as well as a gym-like interface for Reinforcement Learning (RL). We use Verti-Bench to benchmark ten off-road mobility systems, present our findings, and identify future off-road mobility research directions. To summarize, our contributions are: • A general off-road mobility benchmark on vertically challenging terrain with 100 off-road environments and 1000 navigation tasks scalable to various vehicle types; • Millions of off-road terrain features including geometry, semantics (rigid and deformable), and obstacles; • Various datasets and a RL interface to facilitate data-driven off-road mobility; • Findings and future research directions based on benchmark results of various off-road mobility systems. II Related Work In this section, we review the field of off-road autonomy and current evaluation practices to motivate Verti-Bench. II-A Off-Road Autonomy Starting from the DARPA LAGR program [ 1 ] , roboticists have started developing autonomous systems to operate in off-road enviornments. Compared to indoor or on-road operations, off-road environments pose significant challenges during the entire sense-plan-act loop: Robot state estimation systems, such as visual inertial odometry [ 30 ] and simultaneous localization and mapping [ 31 ] , are easily affected by the unstructured visual features from the off-road environments as well as noisy inertial signals caused by the extensive vehicle vibrations; Perception is more than a geometric problem, i.e., free vs. obstacle spaces, as in indoor or on-road settings, and requires the consideration of semantics, e.g., gravel vs. grass. Both terrain geometry and semantics need to be represented [ 32 , 33 , 34 , 35 ] for downstream planning and control tasks; Off-road planners and controllers need to reason beyond collision avoidance and consider factors such as vehicle stability [ 36 , 37 ] , wheel slippage [ 38 , 39 , 40 ] , and terrain traversability [ 41 , 42 , 43 , 44 ] , oftentimes in \mathbb{SE}(3) instead of \mathbb{SE}(2) [ 19 ] due to the uneven off-road terrain surfaces [ 45 , 19 ] . The recent increase in interest

Read the complete text