HGACNet: Hierarchical Graph Attention Network for Cross-Modal Point Cloud Completion

Yadan Zeng, Jiadong Zhou, Xiaohan Li, I-Ming Chen · 2025

Read HGACNet: Hierarchical Graph Attention Network for Cross-Modal Point Cloud Completion by Yadan Zeng, Jiadong Zhou, Xiaohan Li, I-Ming Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-13692.txt.

Introduction

Yadan Zeng, Jiadong Zhou, Xiaohan Li, and I-Ming Chen build HGACNet in a 2025 robotics vision paper. Robots see objects as partial point clouds because surfaces hide themselves and sensors miss regions, and those holes hurt grasp planning and obstacle avoidance. Completing the missing geometry is the task.

HGACNet encodes the partial cloud hierarchically: a graph attention encoder picks out the most informative local points and refines features level by level to preserve structure. A multi-scale fusion module then aligns those features with guidance from a single ordinary photograph, while a contrastive loss pulls the two data types into agreement. On the ShapeNet-ViPC benchmark and a real-object dataset, the system set the best reported completion results.

Catalog record

CREATORYadan Zeng, Jiadong Zhou, Xiaohan Li, I-Ming Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-13692

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HGACNet: Hierarchical Graph Attention Network for Cross-Modal Point Cloud Completion Yadan Zeng, Jiadong Zhou, Xiaohan Li, I-Ming Chen Abstract Point cloud completion is essential for robotic perception, object reconstruction and supporting downstream tasks like grasp planning, obstacle avoidance, and manipulation. However, incomplete geometry caused by self-occlusion and sensor limitations can significantly degrade downstream reasoning and interaction. To address these challenges, we propose HGACNet, a novel framework that reconstructs complete point clouds of individual objects by hierarchically encoding 3D geometric features and fusing them with image-guided priors from a single-view RGB image. At the core of our approach, the Hierarchical Graph Attention (HGA) encoder adaptively selects critical local points through graph attention-based downsampling and progressively refines hierarchical geometric features to better capture structural continuity and spatial relationships. To strengthen cross-modal interaction, we further design a Multi-Scale Cross-Modal Fusion (MSCF) module that performs attention-based feature alignment between hierarchical geometric features and structured visual representations, enabling fine-grained semantic guidance for completion. In addition, we proposed the contrastive loss (C-Loss) to explicitly align the feature distributions across modalities, improving completion fidelity under modality discrepancy. Finally, extensive experiments conducted on both the ShapeNet-ViPC benchmark and the YCB-Complete dataset confirm the effectiveness of HGACNet, demonstrating state-of-the-art performance as well as strong applicability in real-world robotic manipulation tasks. I Introduction Point clouds, as a fundamental representation of 3D geometry, are widely used in robotics, autonomous driving, and scene reconstruction due to their ability to capture precise spatial and structural details [ 1 , 2 , 3 ] . However, real-world point clouds obtained from LiDAR, depth cameras, and other 3D sensors often suffer from self-occlusion, sensor noise, limited resolution, and restricted field of view, leading to fragmented, sparse, or incomplete representations. Such incomplete geometry can significantly impair object recognition, pose estimation, and obstacle avoidance, ultimately degrading the reliability of downstream tasks such as robotic grasping and motion planning [ 4 , 5 , 6 , 7 ] , where accurate 3D structure is critical for safe and effective interaction. To address this challenge, object-level point cloud completion aims to reconstruct complete and structurally consistent 3D shapes of individual objects by inferring missing geometry. Unlike denoising or upsampling, which primarily refine existing points [ 8 ] , completion explicitly recovers unobserved structures by leveraging data-driven geometric priors. While traditional single-modal methods [ 9 , 10 , 11 ] adopt encoder-decoder architectures to predict shapes from partial inputs, they often struggle with occlusions and lack the ability to infer fine-grained structures, especially in symmetric or complex objects. Recent cross-modal approaches incorporate RGB images to provide semantic guidance [ 12 , 13 ] . However, they often fail to fully exploit cross-modal relationships, relying on early-stage fusion or computationally intensive attention modules, which limits semantic alignment and fine-grained reconstruction quality. To overcome these limitations, we propose HGACNet, a cross-modal point cloud completion framework that hierarchically encodes 3D geometric features and fuses them with image-guided priors, as illustrated in Fig. 1 . Our architecture consists of two core components: the Hierarchical Graph Attention (HGA) encoder and the Multi-Scale Cross-Modal Fusion (MSCF) module. The HGA encoder applies graph attention-based downsampling to adaptively select structurally important local points, while progressively refining hierarchical geometric features to capture both fine-grained structures and global context. Parallel to the geometric branch, the image branch incorporates Swin Transformer [ 14 ] for visual feature extraction. The architecture excels at capturing both fine-grained textures and scene-level context via its shifted window-based self-attention, while its multi-stage design naturally complements the hierarchical structure of our point cloud encoder. Fig. 1: Overview of HGACNet. Given a partial point cloud and an RGB image, HGACNet fuses geometric and visual features to reconstruct complete 3D object shapes, and works effectively on both synthetic and real-world data. Subsequently, the MSCF module performs both intra-modal fusion to enhance geometric consistency within the point cloud domain, and inter-modal fusion to align geometric features with structured visual representations via cross-attention, enabling instance-level alignment and fine-grained semantic guidance for completion. To further reduce the modality gap, we propose C-Loss, a novel contrastive loss function that enhances cross-modal feature alignment by enforcing consistency between point cloud and image modalities. Specifically, C-Loss encourages semantically corresponding features to be closer in the latent space while separating unrelated features, facilitating more coherent and accurate reconstruction. Finally, we conduct comprehensive experiments on ShapeNet-ViPC benchmark, where HGACNet outperforms the state-of-the-art (SOTA) method EGIINet [ 15 ] by 17% in L2-CD. Additionally, evaluation on the real-world YCB-Complete dataset highlights its practical applicability in robotic perception scenarios. The key contributions of this work are summarized as follows: • We propose HGACNet, a hierarchical cross-modal framework that integrates hierarchical graph attention encoding with multi-scale cross-modal fusion, effectively leveraging both point cloud geometry and image priors for high-fidelity completion. • We design a HGA encoder that adaptively selects critical local points and refines geometric features to preserve structural continuity and fine-grained details. • We develop a MSCF module that enables instance-level alignment between geometric and visual representations, complemented by our proposed C-Loss to reduce modality discrepancies and improves reconstruction accuracy. II Related Work II-A Geometry-only Point Cloud Completion Point-based completion methods operate directly on unordered point sets, avoiding the quantization artifacts and computational overhead of voxel-based approaches [ 9 ] . A pioneering method, PCN [ 16 ] introduced a coarse-to-fine framework, where a rough shape was first reconstructed and then refined. This paradigm has since been widely adopted and extended in various studies [ 17 , 18 , 19 , 20 ] , with subsequent works focusing on improving both the shape representation and refinement mechanisms. Among these advancements, FoldingNet [ 10 ] introduced a folding-based decoder that maps 2D grids to 3D surfaces, capturing shape deformations effectively. TopNet [ 21 ] adopted a hierarchical rooted tree structure for multi-level shape generation, while MSN [ 22 ] improved refinement through residual learning. Further improvements were introduced by PF-Net [ 23 ] , which enhanced completion by integrating multi-scale feature encoding with adversarial training, and GRNet [ 9 ] , which utilized a 3D grid-based representation to leverage CNNs while preserving structural information. Recently, transformer-based approaches have demonstrated promising advancements. PoinTr [ 11 ] and SnowflakeNet [ 24 ] , reformulated point cloud completion as a set-to-set translation problem, iteratively refining point distributions to recover fine details. While these methods achieve promising results, they rely solely on geometric reasoning from partial inputs, limiting their ability to recover missing details effectively. To address this, cross-modal approaches have been explored, leveraging additional image information to improve point cloud completion. II-B Cross-modal Point Cloud Completion Cross-modal point cloud completion aims to integrate visual cues from RGB images to compensate for missing geometric information in partial point clouds. Early approaches such as ViPC [ 25 ] and CSDN [ 12 ] explored shallow fusion strategies. ViPC generated a coarse shape from the image and refined it by directly concatenating image and point cloud features. CSDN enhanced this idea by proposing IPAdaIN to modulate point cloud features with image priors, followed by pixel-level alignment. While these methods provided initial improvements, their limited feature interaction often failed to capture rich cross-modal correspondences and struggled with structural inconsistencies in complex shapes. To enable deeper multi-modal interaction,

Read the complete text