MATTER: Multiscale Attention for Registration Error Regression

Shipeng Liu, Ziliang Xiong, Khac-Hoang Ngo, Per-Erik Forssén · 2025

Read MATTER: Multiscale Attention for Registration Error Regression by Shipeng Liu, Ziliang Xiong, Khac-Hoang Ngo, Per-Erik Forssén (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-12924.txt.

Introduction

Shipeng Liu, Ziliang Xiong and two co-authors present MATTER in 2025, a method to grade how well two point clouds line up. Good grades matter for mapping and tracking, but past methods sorted grades into a few coarse classes.

MATTER instead predicts the error as a smooth score, which gives finer detail. It pulls traits at many scales and joins them with a focus step. Tests on varied sets are strong, most of all where point density shifts across a scene. Used to guide mapping, it raised map quality for the same repair budget versus the best class-based rival.

Catalog record

CREATORShipeng Liu, Ziliang Xiong, Khac-Hoang Ngo, Per-Erik Forssén
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-12924

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MATTER: Multiscale Attention for Registration Error Regression Shipeng Liu, Ziliang Xiong, Khac-Hoang Ngo, Per-Erik Forssén Abstract Point cloud registration (PCR) is crucial for many downstream tasks, such as simultaneous localization and mapping (SLAM) and object tracking. This makes detecting and quantifying registration misalignment, i.e., PCR quality validation , an important task. All existing methods treat validation as a classification task, aiming to assign the PCR quality to a few classes. In this work, we instead use regression for PCR validation, allowing for a more fine-grained quantification of the registration quality. We also extend previously used misalignment-related features by using multiscale extraction and attention-based aggregation. This leads to accurate and robust registration error estimation on diverse datasets, especially for point clouds with heterogeneous spatial densities. Furthermore, when used to guide a mapping downstream task, our method significantly improves the mapping quality for a given amount of re-registered frames, compared to the state-of-the-art classification-based method. † † address: Department of Electrical Engineering, Linköping University, Linköping, Sweden 1 Introduction Point cloud registration (PCR) is a fundamental task in three-dimensional (3D) computer vision, aiming to estimate the rigid transform between two overlapping point clouds. It serves as a backbone for numerous downstream applications, including simultaneous localization and mapping (SLAM), 3D reconstruction, and robotic navigation. Despite recent advances in PCR, registration error is still significant in challenging scenarios, e.g., when optimization of the loss function gets trapped in local minima [ 1 ] , under motion distortion [ 2 ] or measurement noise, or when the problem is geometrically under-constrained [ 3 , 4 ] . Errors in PCR can propagate to downstream modules, resulting in map distortion and navigation failure. This motivates the need for registration misalignment detection, i.e., predicting, after registration, whether the registered point clouds deviate from the well-aligned ground truth. This prediction can then allow downstream tasks to take corrective action. Figure 1 : Overview of the proposed multiscale attention mechanism for geometric features in registered point clouds. The objective functions used in existing registration methods, such as closest-point residual [ 5 ] , score function [ 6 ] , and entropy gain [ 7 ] , do not reflect the alignment error, as they are not evaluated using the ground truth pose. Recent works [ 8 , 9 , 10 , 11 , 12 , 13 ] proposed learning-based approaches for point cloud misalignment classification (PCMC), where the registration result is categorized into discrete quality levels. Specifically, in [ 12 , 8 ] , logistic regression is used with manually designed features for binary classification. CorAl, introduced in [ 13 ] , fits the neighborhood of each point to a 3D Gaussian distribution and uses the differential entropy of this distribution as the feature. FACT [ 11 ] —the state-of-the-art method—extends binary classification to multiple classes, which allows evaluation at a finer level. However, all these works are limited to classification, i.e., coarsely categorizing the misalignment into a few levels, without providing continuous predictions of the actual alignment error. In this paper, we cast registration error estimation as a continuous prediction problem, regressing a scalar alignment error from multiscale geometric features fused by attention. Our contributions are threefold. First, we extend PCMC to point cloud misalignment regression (PCMR) for fine-grained assessment. Second, we propose multiscale attention for registration error regression (MATTER), which extracts features at multiple geometric scales and fuses them using attention. We depict the proposed attention architecture in Fig. 1 , and detail it in Section 2 . This significantly improves robustness over single-scale features. Finally, we adapt prior PCMC methods to regression and show, via numerical experiments, that MATTER consistently outperforms them on three datasets. We also consider a mapping downstream task and use MATTER to detect misaligned frames that need correction, i.e., to be re-registered. Notably, MATTER achieves a better map quality than the state-of-the-art classification approach for the same amount of re-registration. 2 Methods In this section, we first define the PCMR task, introduce the overall architecture of MATTER and describe the proposed multiscale attention depicted in Fig 1 . 2.1 Task Formulation Consider a pair of source and reference point clouds (P^{\mathrm{S}},P^{\mathrm{R}}) , which are sets of points in \mathbb{R}^{3} that have been registered by an estimated rigid transform \hat{T}\in\mathbb{SE}(3) . We define their alignment error as the point-wise mean distance between the source point cloud aligned by \hat{T} and the same source aligned by the ground-truth transform, T^{\ast} [ 11 ] , E_{\text{align}}\;=\;\,\frac{1}{|P^{\mathrm{S}}|}\sum_{p\in P^{\mathrm{S}}}\big\|\,\hat{T}(p)\,-\,T^{*}(p)\,\big\|_{2}\;.\vskip-5.69054pt (1) We aim to regress this error to obtain a fine-grained estimate of the misalignment. 2.2 Model Architecture We adapt the model architecture in FACT [ 11 ] to our PCMR task by using multiscale attention to fuse alignment-related features. See the illustration in Fig. 1 . First, we follow [ 11 ] to extract global features and local features in local neighborhoods at multiple scales. Afterwards, both types of features are concatenated and cross-attended to extract scale-wise attention score. The attention score is then used as a weighting factor for multiscale features before passing into a point transformer and regression head for the final prediction. Our model is then trained with an l_{2} loss. 2.3 Feature Extraction We first transform both P^{\mathrm{S}} and P^{\mathrm{R}} point clouds into a common frame and apply the farthest point sampling [ 14 ] to select the same number of anchor points from each of them. In the rest of the section, we focus on a generic anchor point. Let \mathcal{N}_{P^{\mathrm{S}}} and \mathcal{N}_{P^{\mathrm{R}}} denote the points from the source and reference point clouds, respectively, within the neighborhood of this anchor point. We use \mathcal{N}_{\mathrm{sep}} to refer to \mathcal{N}_{P^{\mathrm{S}}} if the anchor point belong to P^{\mathrm{S}} , and to \mathcal{N}_{P^{\mathrm{R}}} if it belongs to P^{\mathrm{R}} . Define the joint neighborhood \mathcal{N}_{\mathrm{joint}}=\mathcal{N}_{P^{\mathrm{S}}}\cup\mathcal{N}_{P^{\mathrm{R}}} . Following [ 11 ] , we extract local features around each anchor, namely the separate and joint differential entropy [ 13 ] H(\mathcal{N}{\mathrm{sep}}),H(\mathcal{N}{\mathrm{joint}}) , and the Sinkhorn divergence D_{\lambda} . They measure the discrepancy between the distributions of points in the aligned source with respect to reference neighborhoods. Additional details on the feature computation are provided in Appendix. A . Unreliable neighborhoods are handled explicitly: if one point cloud contributes no points, the differential entropy and Sinkhorn divergence are set to a bounded default. The next local features are coverage ratios \rho_{\mathrm{joint}}=\big|\mathcal{N}_{\mathrm{joint}}\big|/(\big|P^{\mathrm{S}}\big|+\big|P^{\mathrm{R}}\big|) and \rho_{\mathrm{sep}}=\big|\mathcal{N}_{\mathrm{sep}}\big|/\big|P_{\mathrm{sep}}\big| , where \big|P_{\mathrm{sep}}\big| is given by \big|P^{\mathrm{{S}}}\big| or \big|P^{\mathrm{{R}}}\big| if the anchor point belongs to P^{\mathrm{{S}}} or P^{\mathrm{{R}}} , respectively. Global features that are computed on the whole point cloud include a co-visibility score c\!\in[0,1] computed with a visibility operator [ 15 ] , the distance d from the point to the LiDAR sensor, and a binary source flag b\in\{0,1\} designating which point cloud the point originates from. These features serve to down-weight anchor points with non-covisible or distant points, while up-weighting well-observed ones. 2.4 Multiscale Cross-Attention Multiscale Features. In FACT [ 11 ] , the radius of \mathcal{N}_{\mathrm{joint}} is chosen as a single value, adaptively adjusted according to d . There is a tradeoff between using small neighborhoods and large ones. A small radius may fail when the initial alignment is poor, since the corresponding structure can fall outside the neighborhoods; in contrast, a large neighborhood contains

Read the complete text