EGSA-PT:Edge-Guided Spatial Attention with Progressive Training for Monocular Depth Estimation and Segmentation of Transparent Objects

Gbenga Omotara, Ramy Farag, Seyed Mohamad Ali Tousi, G.N. DeSouza · 2025

Read EGSA-PT:Edge-Guided Spatial Attention with Progressive Training for Monocular Depth Estimation and Segmentation of Transparent Objects by Gbenga Omotara, Ramy Farag, Seyed Mohamad Ali Tousi, G.N. DeSouza (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-14970.txt.

Introduction

Gbenga Omotara and three co-authors present edge guided focus for depth and segmentation of transparent objects. Transparency confuses both geometry and semantic perception in vision systems. Multi task learning can help, yet tasks often interfere destructively. Their fusion mechanism injects boundary information between semantic and geometric features to reduce that harm.

On two benchmarks for transparent scenes, the method improves depth accuracy over the prior best system. Segmentation performance stays competitive at the same time. Gains are largest inside transparent regions, where help is most needed. A progressive training plan shifts edge sources from image edges to predicted depth edges. The staged plan further supports robust learning.

Catalog record

CREATORGbenga Omotara, Ramy Farag, Seyed Mohamad Ali Tousi, G.N. DeSouza
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-14970

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EGSA-PT:Edge-Guided Spatial Attention with Progressive Training for Monocular Depth Estimation and Segmentation of Transparent Objects Gbenga Omotara, Ramy Farag, Seyed Mohamad Ali Tousi, G.N. DeSouza Abstract Transparent object perception remains a major challenge in computer vision research, as transparency confounds both depth estimation and semantic segmentation. Recent work has explored multi-task learning frameworks to improve robustness, yet negative cross-task interactions often hinder performance. In this work, we introduce Edge-Guided Spatial Attention (EGSA), a fusion mechanism designed to mitigate destructive interactions by incorporating boundary information into the fusion between semantic and geometric features. On both Syn-TODD and ClearPose benchmarks, EGSA consistently improved depth accuracy over the current state of the art method (MODEST), while preserving competitive segmentation performance, with the largest improvements appearing in transparent regions. Besides our fusion design, our second contribution is a multi-modal progressive training strategy, where learning transitions from edges derived from RGB images to edges derived from predicted depth images. This approach allows the system to bootstrap learning from the rich textures contained in RGB images, and then switch to more relevant geometric content in depth maps, while it eliminates the need for ground-truth depth at training time. Together, these contributions highlight edge-guided fusion as a robust approach capable of improving transparent object perception. August 24, 2026 1 Introduction Transparent object perception remains a challenging research problem in computer vision [ 1 ] , both in terms of recovering the 3D geometry of scenes with transparent objects [ 2 ] or in understanding the semantics [ 3 ] of these scenes. Devising robust solutions to these low-level vision tasks is essential for real world applications such as in robotics [ 4 ] and autonomous driving [ 5 ] , [ 6 ] . In practice, failure to accurately perceive transparency can have serious consequences: a mobile robot may collide with a glass wall, or a manipulator may misjudge a transparent container during grasping. The difficulty arises because transparent objects violate the Lambertian assumption of light reflection. As a result, object boundaries become blurred and the continuity of surfaces is distorted and reflections are often misinterpreted by models. Conventional depth sensors also struggle to capture these objects reliably since their design likewise assumes Lambertian reflectance, often leading to reliance upon specialized sensors. Furthermore, the scarcity of high-quality annotated datasets for non-Lambertian objects makes training robust and reliable models for transparent object perception even more challenging. Recent approaches have sought to address the challenges of transparent object perception primarily through data-oriented strategies. For example, [ 7 ] , [ 8 ] , [ 9 ] curate large-scale datasets that aim to cover all possibilities and employ mixed dataset training to increase robustness across diverse scenarios including transparent ones. Other works tackle the problem by modifying the input data itself. The work done in [ 10 ] uses inpainting on transparent regions while [ 11 ] uses diffusion models to generate matte-equivalent training data. In addition, some researchers have explored specialized sensors [ 3 ] , multi-view [ 12 ] , or stereo setups [ 13 ] . While these methods show improvements and push the field forward, they primarily focus on improving the quality of the input data or extending modalities rather than crafting network modules that directly reason about transparent objects. In parallel to these data-oriented approaches, another line of work focuses on designing network architectures that explicitly model the challenges of transparent object perception. For example, earlier works such as MVTrans [ 12 ] and SimNet [ 13 ] adopted multi-task frameworks in multi-view and stereo settings, showing the value of simultaneously learning multiple perception tasks. However, their reliance on multi-view inputs limits their applicability in monocular scenarios. Building on this direction, MODEST [ 14 ] introduced a monocular multi-tasking framework with a semantic-geometric fusion module which surpassed state-of-the art results on two challenging synthetic and real-world datasets. Their results demonstrate the benefits of the complementary cues across perception tasks for transparent objects. While multi-task frameworks benefit from the complementary cues between segmentation and depth, these cross-task interactions could also be detrimental. For example, segmentation may bias depth toward predicting nearly uniform values across an entire object, even when true depth varies significantly within that object. Similarly, depth may bias segmentation to group spatially adjacent regions with similar depth values into a single semantic category, even when they belong to different objects. Explicit boundary cues are therefore essential to prevent such misinterpretations, motivating our use of edge-guided fusion. Beyond this cross-talk limitation, existing designs such as MODEST also rely heavily on channel attention, which we found can suppress a significant portion of the feature maps. In practice, this may either down-weight channels containing useful information or reveal a redundancy across many channels, leading to inefficient use of model capacity. This further reinforces the need for a redesigned fusion mechanism that incorporates boundary information that supports more reliable cross-task interaction. To this end, we propose an edge-guided redesign of the fusion mechanism that avoids the drawbacks of channel suppression. Our method introduces a gating mechanism for spatial attention by using edges as a guidance signal to incorporate boundary information. During training, these edges progressively shift from being generated from the input RGB images to being extracted from the model’s predicted depth maps in a self-training manner, producing accurate predictions as in 1 . In summary, our main contributions are: • We introduce an edge-guided fusion mechanism that replaces channel attention with boundary aware spatial gating, allowing the model to better understand the bounds of objects. • We propose a progressive training strategy that transitions from RGB-derived edges to depth-derived edges in a self-training manner, which improves stability and consistency. Figure 1: EGSA-PT predicts accurate depth and segmentation masks in the presence of transparent materials. 2 Related Work A growing body of work tackles transparent object perception by introducing high-quality synthetic and real-world datasets, since annotations for transparent objects are notoriously difficult to obtain in real-world contexts. Datasets such as Syn-TODD [ 12 ] , ClearGrasp [ 2 ] , ClearPose [ 15 ] , Booster [ 16 , 17 ] and Trans10k [ 18 ] provide complementary resources across synthetic and real-world domains with diverse input modalities. For instance, Syn-TODD offers RGB-D, stereo and multi-view inputs with precise ground-truth annotations for depth, segmentation and bounding boxes under controlled rendering conditions. ClearGrasp likewise supplies RGB-D data with ground-truth depth, semantic segmentation, surface normals along with camera and object poses. These datasets make it possible to developing and benchmarking models tailored to the unique challenges presented by transparent objects. Another line of work pursues data preprocessing strategies, where raw RGB inputs are transformed before being passed into a model. For example, [ 10 ] applies inpainting to transparent regions in images and generates pseudo-labels using a pre-trained monocular depth model [ 19 ] . These pseudo-lables are then used to fine-tune existing monocular [ 19 ] [ 9 ] or stereo networks [ 20 ] . More recently, a diffusion-based method [ 11 ] generates images of transparent objects from their "matte" counterparts along with a depth map generated using an existing depth model [ 21 ] to ensure consistent scene geometry. This creates a dataset with "easy" (matte)

Read the complete text