Next Best View For Point-Cloud Model Acquisition: Bayesian Approximation and Uncertainty Analysis
Madalena Pombinho Caeiro Caldeira, Plinio Moreno · 2024
Read Next Best View For Point-Cloud Model Acquisition: Bayesian Approximation and Uncertainty Analysis by Madalena Pombinho Caeiro Caldeira, Plinio Moreno (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01734.txt.
Description
Abstract The Next Best View problem is a computer vision problem widely studied in robotics. To solve it, several methodologies have been proposed over the years. Some, more recently, propose the use of deep learning models. Predictions obtained with the help of deep learning models naturally have some uncertainty associated with them.
Catalog record
| CREATOR | Madalena Pombinho Caeiro Caldeira, Plinio Moreno |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-01734
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Next Best View For Point-Cloud Model Acquisition: Bayesian Approximation and Uncertainty Analysis Madalena Pombinho Caeiro Caldeira, Plinio Moreno Abstract The Next Best View problem is a computer vision problem widely studied in robotics. To solve it, several methodologies have been proposed over the years. Some, more recently, propose the use of deep learning models. Predictions obtained with the help of deep learning models naturally have some uncertainty associated with them. Despite this, the standard models do not allow for their quantification. However, Bayesian estimation theory contributed to the demonstration that dropout layers allow to estimate prediction uncertainty in neural networks. This work adapts the point-net-based neural network for Next-Best-View (PC-NBV). It incorporates dropout layers into the model’s architecture, thus allowing the computation of the uncertainty estimate associated with its predictions. The aim of the work is to improve the network’s accuracy in correctly predicting the next best viewpoint, proposing a way to make the 3D reconstruction process more efficient. Two uncertainty measurements capable of reflecting the prediction’s error and accuracy, respectively, were obtained. These enabled the reduction of the model’s error and the increase in its accuracy from 30% to 80% by identifying and disregarding predictions with high values of uncertainty. Another method that directly uses these uncertainty metrics to improve the final prediction was also proposed. However, it showed very residual improvements. I Introduction Developments in the fields of robotics and artificial intelligence have been allowing several improvements in some scientific and non-scientific areas, revolutionizing everyday tasks in the process. This influence can also be visible in the domains of archaeology, conservation and restoration, where their strategic integration has become an invaluable asset, aiding men in tasks that require lots of precision and time investment. The RePair project [ 1 ] is proof of this. It focuses on the use of intelligent robotics in this area, with the goal of making the reconstruction process of fragmented Pompeii frescoes more efficient. Some methodologies for reconstruction rely on computational help - it is possible to use computer algorithms that, just like solving a jigsaw puzzle, are able to perform the reconstructions. For this, however, they require the 3D models of the fragments, which is the task that inspired this work. We study an algorithm for the efficient 3D model acquisition of the fresco fragments. The 3D data collection method must be as efficient as possible. This means that the algorithm needs to find a way of getting the most information of the fragments surface with as few scans as possible. To achieve this goal the algorithm has to find the camera viewpoints (i.e. camera pose with respect to the object being scanned) that will get the most new information on the object being scanned. This is known as the Next Best View (NBV) problem, which the algorithm must adeptly solve to achieve its intended outcome. Recent approaches are based on Deep Learning architectures such as [ 2 ] , [ 3 ] , [ 5 ] and [ 4 ] , showed very good results in performing 3D model reconstructions in an efficient manner. However, they do not account for the uncertainty of their predictions which, in real world scenarios, can be vital to guarantee a good reconstruction performance. Hence, in this work, a NBV that predicts the coverage of the viewpoints - PC-NBV [ 5 ] - is analyzed in a more detailed way, and modifications are made to its architecture to make it account for uncertainty. The uncertainty of the estimated coverage scores is calculated using the idea of Monte Carlo sampling to estimate the uncertainty of the output of a Neural Network. To do this, the deterministic output becomes probabilistic through the application of dropout layers. The goal is for the model to output a prediction along with an uncertainty measurement that reflects how sure the model is of its predictions, in an attempt of enhancing precision with the information given by this measurement. This can subsequently improve the efficiency of the 3D model acquisition process. Two types of metrics that give two different kinds of uncertainty measurements were obtained. One is error related, as it estimates the prediction uncertainty of the coverage estimation. Another one is accuracy related, as it estimates the prediction uncertainty related to the ability of the network on choosing the right NBV. These metrics allow to improve the performance of the NBV selection. II State of the Art II-A Deep Learning for the NBV Problem Resolution The vast majority of deep learning-based methods for NBV, usually approximate the continuous space by creating a discrete set of possible camera viewpoints, the candidate viewpoints. Each method then has its own set of evaluation criteria to measure the quality of each candidate viewpoint. Deep learning enters this process by learning how to approximate the function that measures their quality, being then able to predict it. Amongst previous works [ 2 ] , [ 3 ] , [ 5 ] and [ 4 ] , we selected the Point Cloud Based Next-Best-View Network (PC-NBV) [ 5 ] that relies on point cloud for the object representation [ 6 , 7 ] . Point cloud representation is less computationally costly compared to other common representation methods like voxels and triangular meshes. The network receives a raw point cloud and selects what is the NBV out of a set of 33 candidate views. The candidate views are defined by uniformly sampling points on a sphere around the object. On each view, the coverage score (that measures the amount of new information about the object each view can capture) is computed as: C(P)=\frac{1}{|P_{o}|}\sum_{p\in P}U(min_{p_{0}\in P_{0}}||p-p_{0}||_{2}+\varepsilon), (1) where U is the Heaviside step function, \epsilon is a distance threshold, P_{0} is the complete point cloud of the object and P is the object’s partial point cloud - the point cloud of the object’s reconstruction. Essentially this score quantifies the amount of new points a viewpoint captures in relation to the object partial point cloud (points of the object’s total point cloud that have not yet been included in the partial point cloud). It presents the value as a ratio between the number of new points and the total number of points of the object’s full point cloud. The network’s architecture is represented in the Figure 1 . It consists firstly of a feature extraction unit proposed in [ 8 ] , used to extract structure-aware features. The unit is then followed by a max pooling layer and a self attention unit proposed in [ 9 ] that enables the network to learn how to allocate attention according with similarity of color and texture. This unit is then followed by a shared multilayer perceptron, a max pooling layer and another multilayer perceptron. Fig. 1: PC-NBV Units Architecture. Convolutional Layers: Orange; Activation Layers: Green; Inputs: Gray; Features: Pink The total loss of the model is given by \mathcal{L}=\frac{1}{n_{s}}\sum_{i=1}^{n_{s}}(y_{i}-\hat{y}_{i})^{2}+\lambda\sum_{j=1}^{M}w_{j}^{2}, (2) which is the sum of the MSE between the coverage score prediction ( \hat{y} ) and the groundtruth ( y ) with the L2 loss of the weights ( w_{j} ). The MSE between the coverage score prediction and the groundtruth measures how accurate the model is. The L2 regularization loss measures how well