MultiPull: Detailing Signed Distance Functions by Pulling Multi-Level Queries at Multi-Step

Takeshi Noda, Chao Chen, Weiqi Zhang, Xinhai Liu, Yu-Shen Liu, Zhizhong Han · 2024

Read MultiPull: Detailing Signed Distance Functions by Pulling Multi-Level Queries at Multi-Step by Takeshi Noda, Chao Chen, Weiqi Zhang, Xinhai Liu, Yu-Shen Liu, Zhizhong Han (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01208.txt.

Description

A 2024 arXiv research paper by Takeshi Noda, Chao Chen, Weiqi Zhang, Xinhai Liu, Yu-Shen Liu, Zhizhong Han. Reconstructing a continuous surface from a raw 3D point cloud is a challenging task. Recent methods usually train neural networks to overfit on single point clouds to infer signed distance functions (SDFs). However, neural networks tend to smooth local details due to the lack of ground truth signed distances or normals, which limits the performance of overfitting-based methods in reconstruction tasks.

Catalog record

CREATORTakeshi Noda, Chao Chen, Weiqi Zhang, Xinhai Liu, Yu-Shen Liu, Zhizhong Han
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-01208

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MultiPull: Detailing Signed Distance Functions by Pulling Multi-Level Queries at Multi-Step Takeshi Noda, Chao Chen, Weiqi Zhang, Xinhai Liu, Yu-Shen Liu, Zhizhong Han Abstract Reconstructing a continuous surface from a raw 3D point cloud is a challenging task. Recent methods usually train neural networks to overfit on single point clouds to infer signed distance functions (SDFs). However, neural networks tend to smooth local details due to the lack of ground truth signed distances or normals, which limits the performance of overfitting-based methods in reconstruction tasks. To resolve this issue, we propose a novel method, named MultiPull, to learn multi-scale implicit fields from raw point clouds by optimizing accurate SDFs from coarse to fine. We achieve this by mapping 3D query points into a set of frequency features, which makes it possible to leverage multi-level features during optimization. Meanwhile, we introduce optimization constraints from the perspective of spatial distance and normal consistency, which play a key role in point cloud reconstruction based on multi-scale optimization strategies. Our experiments on widely used object and scene benchmarks demonstrate that our method outperforms the state-of-the-art methods in surface reconstruction. Project page: https://takeshie.github.io/MultiPull 1 Introduction Reconstructing surfaces from 3D point clouds is an important task in computer vision. It is widely used in various real-world scenarios such as autonomous driving, 3D scanning and other downstream applications. Recently, using neural networks to learn signed distance functions from 3D point clouds has made huge progress [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 ] . An SDF represents a surface as the zero-level set of a 3D continuous field, and the surface can be further extracted using the marching cubes algorithm [ 9 ] . In supervised methods [ 10 , 11 , 12 , 13 ] , a continuous field is learned using signed distance supervision. Some methods employ multi-level representations [ 14 , 15 ] , such as Fourier layers and level of detail (LOD) [ 16 , 17 ] , to learn detailed geometry. However, these methods require 3D supervision, including ground truth signed distances or point normals, calculated on a watertight manifold. To address this issue, several unsupervised methods [ 18 , 19 , 20 , 21 , 22 ] were proposed to directly infer an SDF by overfitting neural networks on a single point cloud without requiring ground truth signed distances and point normals. They usually need various strategies, such as geometric constraints [ 18 , 19 , 20 ] and consistency constraints [ 22 , 23 ] , for smoother and more completed signed distance field. However, the raw point cloud is a highly discrete approximation of the surface, learning SDFs directly from the point cloud is often inaccurate and highly ambiguous. This makes it hard for the network to learn accurate SDFs on local details, resulting in over-smooth reconstruction. To address this issue, we propose MultiPull , to learn an accurate SDF with multi-scale frequency features. It enables network to predict SDF from coarse to fine, significantly enhancing the accuracy of the predictions. Furthermore, to optimize the SDF at different scales simultaneously, we introduce constraints on the pulling process. Specifically, given query points sampled around 3D space as input, we use a Fourier transform network to represent them as a set of Fourier features. Next, we design a network that can leverage multi-scale Fourier features to learn an SDF fields from coarse to fine. To optimize the signed distance fields with multi-scale features, we introduce a loss function based on gradient consistency and distance awareness. Compared with Level of Detail (LOD) methods [ 16 , 17 , 24 ] , we can optimize the signed distance fields effectively without a need of signed distance supervision, recovering more accurate geometric details. Evaluations on widely used benchmarks show that our method outperforms the state-of-the-art methods. Our contribution can be summarized as follows. • We propose a novel framework that can directly learn SDFs with details from raw point clouds, progressing from coarse to fine. This provides a new perspective for recovering 3D geometry details. • We introduce a multi-level loss function based on gradient consistency and distance awareness, enabling the network to geometry details. • Our method outperforms state-of-the-art methods in surface reconstruction in terms of accuracy under widely used benchmarks. Figure 1: Visualization of the 3D shape reconstruction. In (a), (b) and (c), SDFs are learned from a point cloud by optimizing multi-level query points at multi-step. At each step, we optimize query points at one level with frequency features at this specific level as conditions. This enables the network to progressively recover coarse-to-fine geometry details. 2 Related Work Classic methods for geometric modeling [ 25 , 26 , 27 , 28 ] have attempted to analyze the geometric modeling of objects, which do not require large-scale datasets. With the advent of extensive and intricate 3D datasets like ShapeNet [ 29 ] and ABC [ 30 ] , learning-based methods have achieved significant advancements [ 18 , 12 , 31 , 32 , 22 , 33 , 34 , 35 , 36 , 37 , 38 ] . These approaches learn implicit representations from various inputs, including multi-view images [ 39 , 40 , 41 , 42 , 43 , 44 ] , point clouds [ 45 , 46 , 47 ] , and voxels [ 48 , 49 , 50 ] . Learning Implicit Functions with Supervision . Supervised methods have made significant progress in recent years. These methods leverage deep learning networks to learn priors from datasets or use real data for supervision [ 10 , 11 , 51 , 52 , 53 , 54 ] to improve surface reconstruction performance. Some supervised approaches use signed distances and point normals as supervision, or leverage occupancy labels to guide the network’s learning process. In order to improve the generalization ability of neural networks and learn more geometric details, some studies learn geometry prior of shapes through supervised learning. Learning Implicit Functions with Conditions . To alleviate the dependence on supervised information, recent studies focus on unsupervised implicit reconstruction methods. These methods do not require pretrained priors during optimization. For example, NeuralPull (NP) [ 21 ] learns SDF by pulling query points in nearby space onto the underlying surface, which relies on the gradient field of the network. CAP [ 55 ] further complements this by forming a dense surface by additionally sampling dense query points. GridPull [ 23 ] generalizes this learning method to the grid, by pulling the query point using interpolated signed distances on the grid. In addition, some studies explore surface reconstruction more deeply and propose innovative methods, such as utilizing differentiable Poisson solutions [ 56 ] , or learning signed [ 57 , 58 , 19 , 51 ] or unsigned functions [ 59 , 55 ] with priors. However, inferring implicit functions without 3D supervision requires a lengthy convergence process, which limits the performance of unsupervised methods on large-scale point cloud datasets. To address this, we propose a fitting-based frequency feature learning strategy that efficiently learns implicit fields without the need for additional supervision. Learning Implicit Functions with LOD . Level-Of-Detail (LOD) models [ 16 , 17

Read the complete text