SALON: Self-supervised Adaptive Learning for Off-road Navigation

Matthew Sivaprakasam, Samuel Triest, Cherie Ho, Shubhra Aich, Jeric Lew, Isaiah Adu, Wenshan Wang, Sebastian Scherer · 2024

Read SALON: Self-supervised Adaptive Learning for Off-road Navigation by Matthew Sivaprakasam, Samuel Triest, Cherie Ho, Shubhra Aich, Jeric Lew, Isaiah Adu, Wenshan Wang, Sebastian Scherer (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07826.txt.

Introduction

Matthew Sivaprakasam and seven co-workers wrote a 2024 paper on a robot system that learns off-road passability on the fly with almost no human input. Off-road ground lacks structure, so hand rules fail across terrains. Prior learning methods gulp data or demand minutes of human driving per terrain type. Their framework updates its terrain cost and speed maps from seconds of its own driving.

It steers clear of ground types unlike anything it has met, keeping risk in check. Over kilometer-long courses in varied terrain, it matched methods trained on vastly more data. Trials on differing robots and places also looked strong. Code is shared openly.

Catalog record

CREATORMatthew Sivaprakasam, Samuel Triest, Cherie Ho, Shubhra Aich, Jeric Lew, Isaiah Adu, Wenshan Wang, Sebastian Scherer
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-07826

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SALON: Self-supervised Adaptive Learning for Off-road Navigation Matthew Sivaprakasam, Samuel Triest, Cherie Ho, Shubhra Aich, Jeric Lew, Isaiah Adu, Wenshan Wang, Sebastian Scherer Abstract Autonomous robot navigation in off-road environments presents a number of challenges due to its lack of structure, making it difficult to handcraft robust heuristics for diverse scenarios. While learned methods using hand labels or self-supervised data improve generalizability, they often require a tremendous amount of data and can be vulnerable to domain shifts. To improve generalization in novel environments, recent works have incorporated adaptation and self-supervision to develop autonomous systems that can learn from their own experiences online. However, current works often rely on significant prior data, for example minutes of human teleoperation data for each terrain type, which is difficult to scale with more environments and robots. To address these limitations, we propose SALON , a perception-action framework for fast adaptation of traversability estimates with minimal human input. SALON rapidly learns online from experience while avoiding out of distribution terrains to produce adaptive and risk-aware cost and speed maps. Within seconds of collected experience, our results demonstrate comparable navigation performance over kilometer-scale courses in diverse off-road terrain as methods trained on 100-1000x more data. We additionally show promising results on significantly different robots in different environments. Our code is available at https://theairlab.org/SALON I Introduction Off-road autonomous driving is becoming an increasingly researched topic due to its wide range of applications. Robots are already being deployed in fields such as agriculture [ 1 ] , infrastructure monitoring [ 2 ] , and defense [ 3 ] , where they must operate in unstructured and diverse environments. To perform reliably, they must reason about terrain lacking clear structures and markings to navigate from one goal to another without crashing or getting stuck. Fig. 1: We present SALON, a framework for off-road navigation with no prior experience. With one prior hand-label, our system running SALON learns from its own experience in the real-world to predict where and how fast to drive. Recent research has focused on improving navigation by generating costmaps with fine details, which captures the complexities of off-road terrain. For instance, detecting obstacles like rocks and trees without confusing them with traversable terrain such as small bushes and tall grass illustrates the level of detail required for effective navigation. Costmap generation methods, such as height-thresholding [ 4 ] and semantic segmentation [ 5 , 6 ] often struggle in off-road domain due to basic assumptions. More expressive geometric analyses [ 7 , 8 , 9 , 10 ] reduce this limitation, but can require extensive hand-tuning and still fail to distinguish different terrain with similar geometry. Learned methods have shown potential to address these issues [ 11 , 12 , 13 , 14 , 15 ] but are trained offline and struggle when deployed in new environments. To improve generalization in novel environments, several works incorporate adaptation in a self-supervised manner, developing proprioceptive signals that enable the system to learn from its own experiences online [ 16 , 17 , 18 ] . ALTER [ 17 ] adapts its visual traversability model using explicitly generated cost (e.g. ideal to lethal) from LiDAR-built geometric maps, but is limited to accurate range sensors, and does not link to autonomy. Most relevant to our work, WVN [ 16 ] is a concurrent perception-action framework that learns traversability online by observing expert demonstration, then autonomously exploring within familiar terrain to refine its predictions. However, their anomaly detection severely penalizes areas where it has not traversed before, requiring teleoperation in all terrain types the robot should visit. This reliance on human teleoperation is not scalable when deploying in unknown environments, as the process may need to be repeated for different environments and robots. Such methods are insufficient for achieving fast adaptation in new environments with minimal human input . We therefore propose SALON , a perception-action framework for fast adaptation of traversability estimates with minimal human input (only one click on an image, instead of minutes of expert teleoperation in all terrains). SALON achieves this with the following design choices: • Mapping and learning in map space : We use a mapping pipeline to project visual features from the camera into a Birds-Eye-View (BEV) map and associate the traversed cells with a cost and speed experienced by the robot. This results in cleaner, more distinctive maps. • One-shot cost augmentation : a user simply clicks areas to avoid on prior images to initalize the system • Explicit Out-of-distribution (OOD) detection : allows cleaner detection of anomalous objects, without long prior teleoperation in all safe terrain • Adaptive and risk-aware cost and speed maps : Allows the robot to optimize mission-relevant metrics with novel experience Overall we present three contributions: 1. SALON , a novel adaptive perception-action framework to generate adaptive costmaps and speedmaps , allowing autonomous navigation with minimal interventions, given as few as one hand-label. 2. Real-world experiments demonstrating performance at a similar level as state-of-the-art methods trained offline on 100-1000x more data. 3. Qualitative results on multiple heterogeneous robots differing in terms of dynamics, sensors, cost functions, visual back-ends, and environments. Our code is open source, leveraging an existing mapping framework in order to facilitate deployment on other robots. II Related Work II-A Costmap Generation for Off-road Driving There exists a large number of existing works on learning self-supervised costmaps, both on and off-road. Some approaches leverage privileged information to supervise neural networks that predict map information at a given timestep [ 13 , 14 , 19 , 17 ] , but this information consists of semantic segmentation or comes from handcrafted cost functions, both of which require copious amount of hand labels and tuning. Some methods aim to circumvent this explicit hand-labeling requirement by using expert demonstration data [ 11 , 20 ] , and while the supervision comes from the data collection process itself new challenges arise with ensuring demonstration quality and adapting to novel stimuli without a human in the loop. Recent works, taking inspiration from older methods [ 21 , 22 , 7 ] , have explored the potential for proprioception as supervision as it allows for a strong robot-specific relationship between experience and cost. Some of these methods leverage signals such as residuals between planned and expected trajectories [ 23 , 24 ] or IMU-based roughness score [ 12 , 25 , 26 ] . However, above methods require a significant amount of training data. While recent works have leveraged pre-trained models or visual foundation models (VFMs) to reduce the amount of labeled inputs [ 27 , 15 ] , costmap generation methods that rely on statically-trained models may fail when deployed in out-of-distribution environments. II-B Adaptive Methods for Costmap Generation To improve performance when deployed in new environments, many works incorporate adaptation in their autonomy stack to learn from online experience. However, previous works do not sufficiently achieve fast adaptation in new environments with minimal human input due to insufficiently expressive features, sensing restrictions, and need for minutes of pretraining data before effective adaptation. We find works that adapt online with LiDAR geometry measurements [ 17 , 28 , 29 , 30 , 31 ] , but still struggle to reason about complex

Read the complete text