Panoptic-CUDAL Technical Report: Rural Australia Point Cloud Dataset in Rainy Conditions

Tzu-Yun Tseng 1† , Alexey Nekrasov 2† , Malcolm Burdorf 2†, Bastian Leibe 2 , Julie Stephany Berrio 1 , Mao Shan 1 , Stewart Worrall 1 · 2025

Read Panoptic-CUDAL Technical Report: Rural Australia Point Cloud Dataset in Rainy Conditions by Tzu-Yun Tseng 1† , Alexey Nekrasov 2† , Malcolm Burdorf 2†, Bastian Leibe 2 , Julie Stephany Berrio 1 , Mao Shan 1 , Stewart Worrall 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-16378.txt.

Introduction

Self-driving research trains on sunny city streets, leaving muddy farm tracks in the rain nearly unmapped. Tzu-Yun Tseng and co-authors release Panoptic-CUDAL, a dataset recorded in rural Australia during rainy weather to fill that gap. High-resolution laser scans, camera images, and vehicle poses were captured together, targeting the task of labelling every point in a three-dimensional scan by object and instance.

Rain spatters sensors with noise and phantom reflections, exactly the stress test the dataset supplies. The team analyses the recordings and reports baseline scores for panoptic and semantic segmentation plus three-dimensional occupancy prediction. They offer the data so perception systems can learn the countryside's wet, unstructured reality before meeting it on the road.

Catalog record

CREATORTzu-Yun Tseng 1† , Alexey Nekrasov 2† , Malcolm Burdorf 2†, Bastian Leibe 2 , Julie Stephany Berrio 1 , Mao Shan 1 , Stewart Worrall 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-16378

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Panoptic-CUDAL Technical Report: Rural Australia Point Cloud Dataset in Rainy Conditions Thanks: † equal contribution Thanks: 1 The Australian Centre for Field Robotics (ACFR) at the University of Sydney (NSW, Australia) Thanks: 2 RWTH Aachen University (Aachen, Germany) Thanks: This research was conducted as part of the project titled ”Data analytics tools for developing and testing autonomous urban road vehicle capabilities” and was partially supported by the Australian Government through the Australian Research Council’s ARC Training Centre funding scheme for Automated Vehicles in Rural and Remote Regions (project IC230100001) Tzu-Yun Tseng 1† , Alexey Nekrasov 2† , Malcolm Burdorf 2†, Bastian Leibe 2 , Julie Stephany Berrio 1 , Mao Shan 1 , Stewart Worrall 1 Abstract Existing autonomous driving datasets are predominantly oriented towards well-structured urban settings and favorable weather conditions, leaving the complexities of rural environments and adverse weather conditions largely unaddressed. Although some datasets encompass variations in weather and lighting, bad weather scenarios do not appear often. Rainfall can significantly impair sensor functionality, introducing noise and reflections in LiDAR and camera data and reducing the system’s capabilities for reliable environmental perception and safe navigation. We introduce the Panoptic-CUDAL dataset, a novel dataset purpose-built for panoptic segmentation in rural areas subject to rain. By recording high-resolution LiDAR, camera, and pose data, Panoptic-CUDAL offers a diverse, information-rich dataset in a challenging scenario. We present analysis of the recorded data and provide baseline results for panoptic and semantic segmentation methods on LiDAR point clouds. The dataset can be found here: https://robotics.sydney.edu.au/our-research/intelligent-transportation-systems/ I Introduction Autonomous driving has received significant attention in recent years, with multiple proposed datasets [ 1 , 2 , 3 , 4 , 5 ] and methods [ 6 , 7 , 8 , 9 , 10 , 11 ] . Although high-quality data have contributed significantly to model improvement [ 5 ] , there is still a need for challenging training and evaluation data under real-world conditions. We aim to address the need for modern automotive datasets in different environments and under challenging weather conditions. Fig. 1 : Front-view image from the Panoptic-CUDAL dataset, with projected LiDAR point clouds. Data collection and annotation faces significant limitations due to high costs, particularly for datasets that consist of various data distributions [ 2 ] . While urban driving scenarios are well represented in existing datasets, remote environments remain understudied despite the important role of rural roads in transportation networks. Recent community interest in thoroughly annotated rural driving datasets [ 12 , 13 ] highlights the need for more research in this direction. Especially given the unique challenges posed by specific characteristics of non-urban settings. Rural and urban environments pose different challenges to autonomous vehicles. Urban datasets introduce multiple object instances in the same scene, multiple long-tail classes, and complex object interactions on a large scale [ 5 ] . However, in contrast to urban spaces, rural datasets introduce other challenges, such as vast open spaces, different road infrastructure, and sometimes reduced levels of road maintenance. This difference becomes particularly pronounced in adverse weather conditions, where model performance typically degrades significantly [ 14 ] This includes rain, which can create large puddles on rural roads, snow, or fog. Addressing these challenges requires comprehensive datasets that capture the long-tail distribution of real-world driving scenarios while enabling model evaluation. To address the need for a challenging dataset, we present Panoptic-CUDAL, a new LiDAR dataset captured in a rural area during rain. We record six driving sequences in Cudal, a town in the state of New South Wales in Australia (seen on Figure 1 ). Our dataset features synchronized sensor data, including surround-view camera images, high-resolution LiDAR point clouds, and odometry information. The data collected has diverse, challenging scenarios inherent to rural environments. We annotate the dataset with panoptic labels following the SemanticKITTI [ 3 ] labels format. TABLE I : Comparison of Automotive Datasets with Panoptic Labels on the LiDAR Level. Existing works focus on urban scenes with no rainfall. Dataset Year LiDAR Beams LiDAR Annotations Annotated Scans Surround-View Cameras Rural Rain SemanticKITTI [ 3 ] 2019 64 Panoptic 15k ✗ ✗ ✗ Waymo [ 5 ] 2019 64 Semantic 230k ✓ ✗ ✓ Panoptic nuScenes [ 15 ] 2021 32 Panoptic 40k ✓ ✗ ✗ Navya3DSeg [ 13 ] 2023 64 Semantic 50k ✗ ✓ ✗ R2D2 [ 12 ] 2023 128 Semantic 10.5k ✗ ✓ ✗ Cudal (ours) 2025 128 Panoptic 14.7k ✓ ✓ ✓ The combination of high-resolution LiDAR (128-channel), surround-view camera images, large dataset size, and data intentionally collected under rainfall conditions in a rural area addresses the need for a dataset to train and validate models that operate in these challenging environments. Accurately annotated data allows for the evaluation of semantic segmentation [ 9 , 16 , 8 ] as well as panoptic [ 7 , 6 ] methods for static and dynamic objects. We analyze the dataset labels, qualitatively show challenging scenarios, train and evaluate a set of models for semantic segmentation and panoptic segmentation on the proposed dataset. Our models achieve acceptable performance on the proposed dataset, and we hope that recorded data will be used by the community to train and evaluate segmentation methods to address challenges posed by rural roads and adverse weather conditions. II Related Work Automotive Datasets. The autonomous driving community has witnessed substantial progress through dataset development over recent years. Early efforts focused primarily on camera-based solutions, often employing a single front-facing monocular [ 17 , 18 ] or stereo vision system [ 1 ] , with a low-resolution LiDAR sensor [ 19 ] . Further works introduced multimodal sensor setups [ 20 , 4 , 3 , 21 ] incorporating high-resolution LiDAR, radars, and surround-view camera setups. Parallel advancements emerged in geographical diversity, with datasets expanding coverage to global regions and with dense labeling schemes [ 22 , 2 , 23 ] . Notably, most existing works do no explicitly identify weather conditions or complexity of a scenery in their recordings. Adverse Datasets. Recent years have seen increased focus on adverse weather perception, with image-based datasets addressing various visual conditions such as nighttime [ 24 , 25 ] , hazards environments [ 26 ] , or focus on seasonal changes [ 14 , 27 ] . However, rural environment introduce challenges, such as ambiguous road boundaries and sparse infrastructure, which remain understudied in the autonomous driving context. The ORFD dataset [ 28 ] partially addresses this gap but limits its scope only to drivable area segmentation. While RUGD [ 29 ] provides multimodal rural data, it is collected using a small robotics platform that makes it unsuitable for full-size vehicle perception. Recent efforts like Navya3DSeg [ 13 ] explicitly include limited rural and residential areas in their data collection. The R2D2 [ 12 ] dataset specifically targets rural road challenges through dense LiDAR semantic annotations. However, recent works lack comprehensive coverage of scenarios in rural contexts under rain conditions, and do not provide panoptic annotations or extensive sensor setup. LiDAR Segmentation Methods. Due to the challenging nature of point cloud data, several segmentation approaches have emerged in recent years to segment LiDAR point clouds using different underlining representations. Recent LiDAR segmentation methods have attempted to solve the task using

Read the complete text