A Contrastive Few-shot RGB-D Traversability Segmentation Framework for Indoor Robotic Navigation

Qiyuan An, Tuan Dang, Fillia Makedon · 2026

Read A Contrastive Few-shot RGB-D Traversability Segmentation Framework for Indoor Robotic Navigation by Qiyuan An, Tuan Dang, Fillia Makedon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06927.txt.

Introduction

An and colleagues build an RGB-D system that marks safe free space for robots. Vision models often miss thin obstacles like chair legs, a safety risk. Their system pairs RGB images with 1D laser depth to capture interactions and improve detection of obstacles. Few-shot learning generalizes from few examples, cutting reliance on datasets. Older methods use only positive prototypes, risking overfitting. The team adds a contrastive branch using obstacle prototypes to refine free-space predictions, plus a two-stage attention depth module aligning depth vectors.

Catalog record

CREATORQiyuan An, Tuan Dang, Fillia Makedon
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33783
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.06927

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Contrastive Few-shot RGB-D Traversability Segmentation Framework for Indoor Robotic Navigation Qiyuan An, Tuan Dang, Fillia Makedon Abstract Indoor traversability segmentation aims to identify safe, navigable free space for autonomous agents, which is critical for robotic navigation. Pure vision-based models often fail to detect thin obstacles, such as chair legs, which can pose serious safety risks. We propose a multi-modal segmentation framework that leverages RGB images and sparse 1D laser depth information to capture geometric interactions and improve the detection of challenging obstacles. To reduce the reliance on large labeled datasets, we adopt the few-shot segmentation (FSS) paradigm, enabling the model to generalize from limited annotated examples. Traditional FSS methods focus solely on positive prototypes, often leading to overfitting to the support set and poor generalization. To address this, we introduce a negative contrastive learning (NCL) branch that leverages negative prototypes (obstacles) to refine free-space predictions. Additionally, we design a two-stage attention depth module to align 1D depth vectors with RGB images both horizontally and vertically. Extensive experiments on our custom-collected indoor RGB-D traversability dataset demonstrate that our method outperforms state-of-the-art FSS and RGB-D segmentation baselines, achieving up to 9% higher mIoU under both 1-shot and 5-shot settings. These results highlight the effectiveness of leveraging negative prototypes and sparse depth for robust and efficient traversability segmentation. I Introduction Traversability segmentation is a fundamental task in robotic navigation, aiming to identify freespace that is safe for autonomous agents to traverse. While most existing work focuses on outdoor scenarios such as self-driving cars [ 10 , 18 ] , indoor environments remain relatively underexplored despite their practical importance in warehouse automation, hotel service, and hospital robotics [ 2 , 1 ] . Compared to outdoor scenes that often feature structured roads and lane markings, indoor traversability segmentation poses unique challenges due to varying lighting conditions, complex floor textures, cluttered layouts, and the presence of arbitrarily moving humans or objects [ 17 ] . Most prior work adopts purely vision-based solutions [ 8 , 15 , 27 ] . However, our analysis shows that even state-of-the-art segmentation models, such as Deeplabv3+ [ 5 ] and SegFormer [ 28 ] , struggle to detect thin obstacles such as chair legs. Although these objects occupy only a small fraction of image pixels and have minimal influence on global segmentation metrics, failing to detect them can pose significant safety risks to robots. Fig. 1 : Illustration of our RGB-D few-shot segmentation framework. The support and query inputs consist of RGB images ( S_{RGB} , Q_{RGB} ) and depth vectors ( S_{D} , Q_{D} ), encoded separately and fused via a multi-modal fusion block to produce support and query features ( s , q ). The support mask S_{mask} is used to mask-pool on s , yielding both positive and negative prototypes ( s^{+} , s^{-} ). The query feature q is then refined into free-space and obstacle representations ( q^{+} , q^{-} ), which are concatenated and digested by a lightweight decoder to generate the final query mask ( q_{mask} ). Unlike conventional RGB-D datasets such as NYUv2 [ 21 ] or SUN RGB-D [ 23 ] , which provide dense 2D depth maps registered with corresponding RGB images, our custom-collected dataset consists of paired RGB images and 1D laser scans. We adopt this sensing setup for several reasons. First, many commercial indoor robots (e.g., cleaning, delivery, and assistive robots) are equipped with lightweight, low-cost 1D LiDARs rather than expensive 2D or 3D depth cameras, making our dataset more representative of real-world deployment scenarios. Second, 1D laser sensors enable large-scale, long-duration data collection at a fraction of the cost of dense depth sensors, thereby allowing us to build a large-scale dataset of thousands of RGB–1D depth pairs. Finally, the sparse and partial nature of 1D scans introduces a unique technical challenge: the depth backbone must learn to effectively encode 1D inputs and align them with 2D RGB features, which differs substantially from prior multimodal segmentation frameworks such as DFormer [ 30 ] and CANet [ 33 ] . Therefore, our dataset is inherently more challenging than conventional RGB-D datasets: the 1D depth signals are vertically degenerated and often unregistered with respect to the RGB images. This setting reflects real-world conditions in which robot-mounted sensors are limited in resolution, field of view, or calibration accuracy. Consequently, models trained on our dataset must be robust to sensor imperfections, unregistration, and sparse depth information—properties that are critical for practical indoor robotic navigation. Another challenge comes from the unregistered depth vectors, which do not align well with the vertical beams of the corresponding RGB images in our custom-collected dataset. To address this, we introduce a two-stage attention depth module that dynamically maps the 1D depth to its paired RGB image along both vertical and horizontal dimensions. This design not only eliminates the need for explicit registration but also enables the model to capture dynamic geometric interactions between RGB and depth features. Training a reliable traversability segmentation model also faces several practical challenges. Chief of them is acquiring large-scale, fine-grained annotations, which is often expensive, time-consuming, and labor-intensive [ 4 ] . Few-shot segmentation (FSS) addresses this challenge by enabling models to learn from a limited number of labeled examples (the support set) and generalize to new, unseen instances (the query set) [ 13 , 14 , 31 ] . We adopt the meta-learning paradigm in which the query set is matched to support prototypes at the feature level, allowing the model to generalize with minimal supervision. In the context of indoor traversability segmentation, traditional prototype-matching methods first learn a positive prototype from the support set, representing traversable freespace. Query pixels that closely match this prototype in the feature space are then classified as freespace. However, this strategy is prone to misclassification when regions share similar textures or colors—for example, confusing white walls with white ceramic floor tiles. Such reliance on positive prototypes alone leads to overfitting to the support set and poor generalization to unseen scenarios. To address this limitation, we propose to explicitly leverage negative prototypes —representations of obstacles that are typically ignored in conventional FSS methods. Our negative contrastive learning (NCL) branch first extracts obstacle prototypes from the support set and then identifies corresponding obstacle regions in the query set. These negative regions are used to refine the free-space masks by explicitly expelling obstacles from potential traversable areas. Finally, a lightweight decoder fuses the outputs of both the positive and negative branches to produce the final segmentation mask. To summarize, we propose a multi-modal traversability segmentation framework with a novel few-shot training paradigm. Our main contributions are: 1. Multi-modal RGB-D segmentation: Integrating RGB images and 1D depth vectors captures geometric interactions and improves detection of thin obstacles. 2. Two-stage attention depth module: Dynamically aligning depth vectors with RGB images along horizontal and vertical dimensions addresses unregistered depth. 3. Negative contrastive learning: Leveraging neglected negative prototypes enhances generalization and reduces overfitting in free-space prediction. 4. Dataset contribution: We collect and release a large-scale indoor RGB-D traversability dataset with sparse 1D depth annotations, providing a new benchmark for future research in indoor navigation. Our implementation will be publicly available at https://github.com/qiyuan53/NCL . II Related Work Traversability segmentation is a critical

Read the complete text