PLATYPUS: Progressive Local Surface Estimator for Arbitrary-Scale Point Cloud Upsampling
Donghyun Kim, Hyeonkyeong Kwon, Yumin Kim, Seong Jae Hwang · 2024
Read PLATYPUS: Progressive Local Surface Estimator for Arbitrary-Scale Point Cloud Upsampling by Donghyun Kim, Hyeonkyeong Kwon, Yumin Kim, Seong Jae Hwang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00432.txt.
Description
A 2024 arXiv research paper by Donghyun Kim, Hyeonkyeong Kwon, Yumin Kim, Seong Jae Hwang. 3D point clouds are increasingly vital for applications like autonomous driving and robotics, yet the raw data captured by sensors often suffer from noise and sparsity, creating challenges for downstream tasks. Consequently, point cloud upsampling becomes essential for improving density and uniformity, with recent approaches showing promise by projecting randomly generated query points onto the underlying surface of sparse point clouds.
Catalog record
| CREATOR | Donghyun Kim, Hyeonkyeong Kwon, Yumin Kim, Seong Jae Hwang |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-00432
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PLATYPUS: Progressive Local Surface Estimator for Arbitrary-Scale Point Cloud Upsampling Donghyun Kim, Hyeonkyeong Kwon, Yumin Kim, Seong Jae Hwang Abstract 3D point clouds are increasingly vital for applications like autonomous driving and robotics, yet the raw data captured by sensors often suffer from noise and sparsity, creating challenges for downstream tasks. Consequently, point cloud upsampling becomes essential for improving density and uniformity, with recent approaches showing promise by projecting randomly generated query points onto the underlying surface of sparse point clouds. However, these methods often result in outliers, non-uniformity, and difficulties in handling regions with high curvature and intricate structures. In this work, we address these challenges by introducing the Progressive Local Surface Estimator (PLSE), which more effectively captures local features in complex regions through a curvature-based sampling technique that selectively targets high-curvature areas. Additionally, we incorporate a curriculum learning strategy that leverages the curvature distribution within the point cloud to naturally assess the sample difficulty, enabling curriculum learning on point cloud data for the first time. The experimental results demonstrate that our approach significantly outperforms existing methods, achieving high-quality, dense point clouds with superior accuracy and detail. Introduction Recently, autonomous driving, robotics, and other technologies that utilize 3D data have attracted significant interest, leading to the growing popularity of 3D point clouds as a representation of 3D data. However, the raw point clouds captured by sensors such as LiDAR, depth cameras often contain significant noise, and the distribution becomes particularly sparse for points that are farther from the sensor. Therefore, point cloud upsampling , the task of increasing the density points of a sparse 3D point cloud to be of dense (e.g., Fig. 1 sparse Input to dense GT) is vital for effectively using raw data in tasks like classification and segmentation, leading to various methods to tackle this challenge. Starting with early works ( Alexa et al. 2003 ; Lipman et al. 2007 ; Huang et al. 2009 ) that employed optimization-based methods, the rise of deep learning has led to the proposal of various learning-based methods ( Yu et al. 2018 ; Li et al. 2019 ; Qian et al. 2021a ) , for training point cloud upsampling networks. Existing learning-based methods have demonstrated excellent performance but they have several drawbacks. These methods typically split the sparse input point cloud into multiple patches, upsample each patch, and then recombine them. This approach (i.e., split-and-combine process), which fails to consider the relationships between patches during upsampling, often results in issues such as holes, outliers, and non-uniformity, especially at the boundaries where the patches are combined. Figure 1: Comparative visualization of 4 \times point cloud upsampling results on PU1K. Point cloud upsampling is the task of generating a denser point cloud (i.e., rightmost column) that accurately reflects the underlying geometry of a sparse point cloud (i.e., leftmost column). Our method successfully upsamples intricate areas where existing methods struggle to perform well. To address the issues arising from the split-and-combine process, recent studies ( He et al. 2023 ; Li et al. 2024 ) have proposed a pipeline that moves randomly generated points (i.e., query points) onto the surface that the point cloud inherently represents. In the upsampling process of this pipeline, initial query points are generated around the sparse point cloud (i.e., input point cloud). Next, these query points are projected onto the underlying surface of the sparse point cloud. To accurately determine the underlying surface, it is necessary to know the unsigned distance field of the ground-truth point cloud, but during upsampling, the ground-truth point cloud is not available. Therefore, in the training phase, the network is trained to predict the unsigned distance from randomly generated query points around the sparse point cloud to its underlying surface. This approach enables the network to infer the underlying surface of the dense point cloud using only the sparse point cloud. Figure 2: An analysis of the differences between FPS (Farthest Point Sampling) and our newly proposed curvature-based sampling. The comparison shows which points remain as the point cloud is progressively sampled down to fewer points using each sampling method. FPS uniformly samples points across the entire point cloud, whereas curvature-based sampling selectively samples points from regions with intricate structures and high curvature values. While this pipeline has addressed many of the issues in previous methods, there are still some significant problems that remain unresolved. As seen in Figure 1 , areas within the point cloud that require high curvature and locally complex structures (e.g., animal paws and ears, object edges) are often poorly upsampled. To better capture local features in complex regions that existing methods struggle to upsample, our work employs a novel approach using the concept of curvature value within the distance-estimating network of this pipeline. Curvature value represents the degree of curvature in a specific area, calculated by considering the geometric relationships with surrounding points. This metric allows us to quantitatively identify regions where existing methods fail to upsample effectively. We then utilize our newly proposed curvature-based sampling technique within the network to explicitly sample these regions with high curvature values ( Figure 2 ). We refer to this network as Progressive Local Surface Estimator (PLSE), which progressively retains regions with high curvature and intricate structures, enabling the network to focus on extracting features from these critical areas. Using the curvature value, which effectively highlights regions in the point cloud where upsampling is challenging, we further enhance the learning process of PLSE by implementing a new curriculum learning strategy that calculates the difficulty of a point cloud based on the distribution of curvature values. As shown in Figure 1 , the network’s feature extractor generally struggles more with capturing features in local regions with complex structures and high curvature compared to simpler, flatter areas. Based on this observation, if a point cloud has fewer complex structures—indicated by a curvature value distribution skewed toward lower values and a lower mean—we classify it as an easy sample from a learning perspective ( Figure 3 ). Conversely, if the point cloud’s curvature value distribution is skewed toward higher values, indicating more complex structures, it is classified as a hard sample. Following the curriculum learning strategy, easy samples were used during the early epochs, while hard samples were introduced in the later epochs, helping the network to more effectively learn the unsigned distance field. Contributions. In this work, we introduce PLATYPUS ( P rogressive L ocal Surface Estimator for A rbi T rar Y -Scale P oint Cloud U p S ampling) for point cloud upsampling. Specifically, we make the following contributions: • We propose a novel network, Progressive Local Surface Estimator (PLSE), to learn the unsigned distance field from sparse point clouds. PLSE employs a curvature-based sampling method, which allows our network to explicitly focus on extracting features from critical areas. • To improve the learning of the unsigned distance field, we implement a curriculum learning strategy, which classifies training samples into easy and hard based on the skewness of curvature value distribution. • The results from diverse experiments demonstrate that our approach achieves state-of-the-art performance. We provide the code in the supplementary material which will be released upon publication. Figure 3: An analysis of the easy samples and hard samples