NFR: Neural Feature-Guided Non-Rigid Shape Registration

Zhangquan Chen 1 , Puhua Jiang 1 , Mingze Sun 1 and Ruqi Huang 1 1 Tsinghua Shenzhen International Graduate School, Shenzhen, China · 2025

Read NFR: Neural Feature-Guided Non-Rigid Shape Registration by Zhangquan Chen 1 , Puhua Jiang 1 , Mingze Sun 1 and Ruqi Huang 1 1 Tsinghua Shenzhen International Graduate School, Shenzhen, China (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22445.txt.

Introduction

Zhangquan Chen, Puhua Jiang, Mingze Sun, and Ruqi Huang register bent and partial 3D shapes by letting a neural network whisper correspondences to a classical algorithm. Their 2025 paper addresses a hard pairing problem: two shapes that deform heavily and overlap only in part. Hand-built geometric features fail there, and learned matchers alone can be brittle.

Their pipeline feeds neural features, learned by shape-matching networks, into an iterative geometric registration loop. The correspondences are refreshed as the alignment improves and filtered for consistency, which hardens the whole process. Training needed no hand-labeled correspondences and only a few dozen shapes of modest variety. On standard benchmarks for non-rigid and partial matching, the method reached state-of-the-art accuracy, and it paired difficult unseen shapes that defeat both classical and intrinsic approaches. The code is public.

Catalog record

CREATORZhangquan Chen 1 , Puhua Jiang 1 , Mingze Sun 1 and Ruqi Huang 1 1 Tsinghua Shenzhen International Graduate School, Shenzhen, China
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-22445

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

NFR: Neural Feature-Guided Non-Rigid Shape Registration Zhangquan Chen 1 , Puhua Jiang 1 , Mingze Sun 1 and Ruqi Huang 1 1 Tsinghua Shenzhen International Graduate School, Shenzhen, China Abstract In this paper, we propose a novel learning-based framework for 3D shape registration, which overcomes the challenges of significant non-rigid deformation and partiality undergoing among input shapes, and, remarkably, requires no correspondence annotation during training. Our key insight is to incorporate neural features learned by deep learning-based shape matching networks into an iterative, geometric shape registration pipeline. The advantage of our approach is two-fold – On one hand, neural features provide more accurate and semantically meaningful correspondence estimation than spatial features ( e.g., coordinates), which is critical in the presence of large non-rigid deformations; On the other hand, the correspondences are dynamically updated according to the intermediate registrations and filtered by consistency prior, which prominently robustify the overall pipeline. Empirical results show that, with as few as dozens of training shapes of limited variability, our pipeline achieves state-of-the-art results on several benchmarks of non-rigid point cloud matching and partial shape matching across varying settings, but also delivers high-quality correspondences between unseen challenging shape pairs that undergo both significant extrinsic and intrinsic deformations, in which case neither traditional registration methods nor intrinsic methods work. Our code is available at https://github.com/rqhuang88/NFR . \JournalSubmission \copyrightTextTitPag Preprint. \copyrightTextRunPag Preprint. ccs Computing methodologies Computer vision ccs Computing methodologies Computer graphics 1 Introduction Estimating dense correspondences between 3D human shapes is pivotal for a multitude of human-centric computer vision applications, including 3D reconstruction [ YZG∗18 ] , 3D pose estimation [ WTZ∗21 ] , and animation [ PLG∗16 ] among others. In this paper, we tackle the challenging task of estimating correspondences between unstructured point clouds sampled from surfaces undergoing significant non-rigid deformations, irrespective of whether the point clouds are full or partial . In fact, the shape matching task has garnered increasing attention within the community [ ZQZ∗21 , LGAR21 , MRMO20 , JSH23a , CB23 ] . These methods predominantly adopt a data-driven approach to learn embedding schemes that project point clouds into high-dimensional spaces. By elevating the point clouds to these high-dimensional spaces, non-rigid deformations are more effectively characterized compared to the ambient space, \mathbb{R}^{3} . The optimal transformations, and consequently the correspondences, are then estimated within these embedded spaces. Despite the great progresses, such approaches are hindered by several limitations: 1) Their performance on previously unseen shapes is largely unpredictable; 2) The learned high-dimensional embeddings lack intuitive geometric interpretation, making the resulting correspondences challenging to evaluate and analyze in the absence of ground-truth labels; 3) There is still a large room for improvement in estimating dense correspondences between human shapes with partiality . On the other hand, shape registration itself does not serve as a direct rescue to our problem of interest, as the previous approaches typically rely on the premise that the undergoing non-rigid deformation can be approximated by a set of local, small-to-moderate, rigid deformations, which severely hinders their performance in the presence of large deformations (see Fig. 1 ) and heterogeneity. Motivated by the above observations, we propose a neural feature-guided framework (NFR) that synergistically combines the strengths of classic shape registration and learning-based embedding techniques. In a nutshell, we leverage the estimated correspondence from the latter to guide shape deformation via the former iteratively. Our key insight is to enforce similarity between the deformed source and target in both the ambient space and the learned high-dimensional space . Intuitively, in the presence of large deformations, the correspondences learned in the high-dimensional space are more reliable than those based solely on proximity in the ambient space. Conversely, accurately deforming the source mesh to align with the target point cloud enhances spatial similarity in the ambient space, which in turn increases similarity in the embedded space, leading to more precise correspondences. Ultimately, this approach enables us to compute correspondences between raw point clouds using a shared source mesh as a central hub. As demonstrated in Sec. 4 , our method allows for the selection of the source shape during inference, which can be independent of the training data. While conceptually straightforward, we aim to enhance the performance, robustness, and practical utility of our pipeline through several tailored designs. First and foremost, the key component of our pipeline is an embedding scheme for accurately and efficiently estimating correspondences between the deformed source shape and the target point cloud . Specifically, to leverage the advantages of Deep Functional Maps (DFM) [ LDO22 , CB22 , SMJ∗23 ] , the current state-of-the-art approach for matching triangular meshes , we pre-train an unsupervised DFM [ SMJ∗23 ] on meshes as a teacher network. We then train a point-based feature extractor (i.e., an embedding scheme for points) as a student network on the corresponding vertex sets using natural self-supervision. Unlike the approach in [ CB23 ] , which heavily relies on DiffusionNet [ SACO22 ] to extract intricate structural details (e.g., Laplace-Beltrami operators) from both mesh and point cloud inputs, the teacher-student paradigm allows us to utilize a more streamlined backbone—DGCNN [ WSL∗19 ] . Secondly, in contrast to prior works that either implicitly [ LH22b , YDXZ23a ] or explicitly [ SO20 , CB23 , JSH23a ] require rigidly aligned shapes for input/initialization, or demand dense correspondence labels [ MRMO20 ] , we train an orientation regressor on a large set of synthetic, rigidly aligned shapes to automatically pre-process input point clouds in arbitrary orientations. Lastly, we introduce several innovative components in our pipeline, including a dynamic correspondence updating scheme, bijectivity-based correspondence filtering, and a two-stage registration process. Moreover, we supplement the above framework for estimating correspondences between partial point clouds and complete source shapes . In particular, we assume to be given a set of complete meshes as training data. During training, we randomly pick a pair of meshes \mathcal{S},\mathcal{T} , and sample on \mathcal{T} for a partial point cloud \mathcal{T}_{p} , without preserving any mesh structure. Recall that, in DPFM [ APO21 ] , one first constructs a graph-like structure on \mathcal{T}_{p} and then computes spectral embedding on it, which is both slow and unstable. In contrast to that, we consider to assign the spectral embedding of \mathcal{T} to its subset \mathcal{T}_{p} via the natural inclusion map, forming a novel spatially truncated spectral embedding with respect to \mathcal{T}_{p} and \mathcal{T} . As shown in Prop. 1 in Sec. 3 , under certain conditions, we prove that the transformation between the spectral embedding of \mathcal{S} and that of \mathcal{T}_{p} can always be set as the functional map from \mathcal{S} to \mathcal{T} , regardless of how \mathcal{T}_{p} is generated. The above procedure leads to a simple yet effective training scheme. Namely, we can use arbitary point-based feature extractor to obtain per-point feature on both \mathcal{T}_{p} and \mathcal{S} , compute point-wise map T:\mathcal{T}_{p}\rightarrow\mathcal{S} via proximities in the feature space, and encode T into transformation between the spectral embedding of \mathcal{T}_{p} and \mathcal{S} . Finally, the functional map from \mathcal{S} to \mathcal{T} can be used as the supervision signal on the obtained transformation. One last missing puzzle from the above pipeline is the source of functional maps among the full shapes. In fact, such can be

Read the complete text