Cross-Modal Mapping and Dual-Branch Reconstruction for 2D–3D Multimodal Industrial Anomaly Detection Journal: Information Fusion

Radia Daci, Vito Renó, Cosimo Patruno, Angelo Cardellicchio, Abdelmalik Taleb-Ahmed, Marco Leo, Cosimo Distante · 2026

Read Cross-Modal Mapping and Dual-Branch Reconstruction for 2D–3D Multimodal Industrial Anomaly Detection Journal: Information Fusion by Radia Daci, Vito Renó, Cosimo Patruno, Angelo Cardellicchio, Abdelmalik Taleb-Ahmed, Marco Leo, Cosimo Distante (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03939.txt.

Introduction

Radia Daci and others present CMDR-IAD, an unsupervised method for multimodal industrial anomaly finding. The method pairs bidirectional 2D-3D cross-modal mapping, models appearance-shape consistency, with dual-branch reconstruction capturing texture and shape structure. Fusion then combines a reliability-gated mapping anomaly with a confidence-weighted reconstruction anomaly for stable precise localization. On the MVTec 3D-AD benchmark it reaches leading results without memory banks, including 97.3% image-level AUROC, and its 3D-only variant proves strongly effective on a real polyurethane cutting dataset.

Catalog record

CREATORRadia Daci, Vito Renó, Cosimo Patruno, Angelo Cardellicchio, Abdelmalik Taleb-Ahmed, Marco Leo, Cosimo Distante
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66722
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.03939

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Cross-Modal Mapping and Dual-Branch Reconstruction for 2D–3D Multimodal Industrial Anomaly Detection Journal: Information Fusion Radia Daci, Vito Renó, Cosimo Patruno, Angelo Cardellicchio, Abdelmalik Taleb-Ahmed, Marco Leo, Cosimo Distante Abstract Multimodal industrial anomaly detection benefits from integrating RGB appearance with 3D surface geometry, yet existing unsupervised approaches commonly rely on memory banks, teacher–student architectures, or fragile fusion schemes, limiting robustness under noisy depth, weak texture, or missing modalities. This paper introduces CMDR–IAD , a lightweight and modality-flexible unsupervised framework for reliable anomaly detection in 2D+3D multimodal as well as single-modality (2D-only or 3D-only) settings. CMDR–IAD combines bidirectional 2D ↔ \leftrightarrow 3D cross-modal mapping to model appearance–geometry consistency with dual-branch reconstruction that independently captures normal texture and geometric structure. A two-part fusion strategy integrates these cues: a reliability-gated mapping anomaly highlights spatially consistent texture–geometry discrepancies, while a confidence-weighted reconstruction anomaly adaptively balances appearance and geometric deviations, yielding stable and precise anomaly localization even in depth-sparse or low-texture regions. On the MVTec 3D-AD benchmark, CMDR–IAD achieves state-of-the-art performance while operating without memory banks, reaching 97.3% image-level AUROC (I-AUROC), 99.6% pixel-level AUROC (P-AUROC), and 97.6% AUPRO. On a real-world polyurethane cutting dataset, the 3D-only variant attains 92.6% I-AUROC and 92.5% P-AUROC, demonstrating strong effectiveness under practical industrial conditions. These results highlight the framework’s robustness, modality flexibility, and the effectiveness of the proposed fusion strategies for industrial visual inspection. Our source code is available at https://github.com/ECGAI-Research/CMDR-IAD/ . 1 Introduction Industrial anomaly detection (IAD) aims to identify defective or atypical products during or after manufacturing, and is therefore a key component of modern quality control pipelines. In practice, collecting representative defective samples is difficult due to their rarity, variability, and the cost of labeling. As a result, most recent methods adopt an unsupervised or one-class setting, where models are trained only on normal (nominal) data and anomalies are detected as deviations from learned regular patterns. While deep learning has significantly advanced anomaly detection in two-dimensional (2D) images, many industrial defects remain challenging to recognize from RGB appearance alone. Variations in illumination, specular reflections, or sensor noise can degrade the reliability of purely image-based approaches, and some defects manifest predominantly as subtle geometric deviations rather than as obvious color or texture changes. To address these challenges, recent research has increasingly turned to multimodal anomaly detection that combines RGB images with three-dimensional (3D) information such as point clouds or depth maps. 3D measurements are largely invariant to lighting and provide explicit shape and surface information, making them highly complementary to RGB signals. The introduction of benchmarks such as MVTec 3D–AD [ 3 ] has accelerated this line of work by offering pixel-registered RGB and 3D data, fostering a variety of multimodal anomaly detection methods. Early multimodal industrial anomaly detection methods, such as BTF and M3DM, extend successful 2D approaches by modeling normality through memory banks of multimodal features , achieving strong performance at the cost of high memory usage and limited inference speed [ 15 , 38 ] . To improve efficiency, AST adopts a teacher–student architecture, but treats 3D information indirectly, which limits the explicit exploitation of geometric structure [ 33 ] . CFM explicitly models cross-modal feature consistency between RGB and 3D data, highlighting the importance of appearance–geometry relationships, yet remains constrained by its multimodal-only assumption [ 10 ] . More recent approaches address robustness through cross-modal distillation, joint teacher–student memory mechanisms, and fusion architecture analysis [ 35 , 24 , 25 ] , but often introduce architectural complexity or rely on fixed fusion strategies. Existing methods exhibit clear trade-offs between segmentation performance, inference speed, and memory consumption, motivating a lightweight and modality-flexible framework with adaptive fusion. In this work, we propose a different perspective on multimodal industrial anomaly detection based on explicit cross-modal feature mapping and dual-branch reconstruction. We introduce CMDR–IAD (Cross-Modal Mapping and Dual-Branch Reconstruction for 2D–3D Multimodal Industrial Anomaly Detection), an unsupervised and modality-flexible framework that directly models relationships between RGB-based and 3D-based features while also learning modality-specific reconstructions of normal appearance and geometry. CMDR–IAD employs frozen pretrained encoders for RGB images and point clouds to extract pixel-aligned feature representations, and learns two lightweight mapping networks that project 2D features into the 3D feature space and vice versa. These mappings are trained exclusively on nominal data, capturing cross-modal correspondences characteristic of defect-free samples. In parallel, two independent reconstruction decoders are trained to reproduce normal 2D and 3D features, providing modality-specific cues about deviations from normal patterns. At inference time, CMDR–IAD combines four complementary signals: 2D reconstruction error, 3D reconstruction error, and two cross-modal mapping discrepancies. An adaptive fusion mechanism converts reconstruction errors into spatial reliability weights and modulates cross-modal discrepancies through a spatially varying gating function. This design allows the model to emphasize the most informative modality at each pixel and suppress noisy responses in regions with unreliable depth or weak visual texture, producing accurate and stable anomaly maps. Importantly, the use of frozen encoders and lightweight mapping and decoding networks results in competitive memory consumption and inference speed, making CMDR–IAD suitable for deployment in real-world industrial inspection systems. Moreover, the 3D reconstruction branch can operate independently in scenarios where only point clouds are available, as demonstrated on a real-world polyurethane cutting dataset. The contributions presented in this paper can be summarized as follows: 1. We propose CMDR–IAD , a modality-flexible cross-modal mapping and dual-branch reconstruction framework for multimodal industrial anomaly detection, which explicitly models 2D–3D feature relationships while preserving modality-specific normal patterns. 2. We introduce an adaptive fusion strategy that integrates cross-modal mapping inconsistencies with modality-aware reconstruction deviations through spatial reliability weighting and contrast-based gating, enabling robust anomaly localization under noisy or incomplete 3D measurements. 3. We demonstrate that CMDR–IAD achieves state-of-the-art anomaly detection and localization performance on the MVTec 3D–AD benchmark while maintaining competitive inference speed and memory footprint (Fig. 1 ), and we show strong performance of the 3D-only variant on a real polyurethane cutting dataset. Figure 1: Performance, speed and memory occupancy of Multimodal Anomaly Detection methods. The chart reports defect segmentation performance (AUPRO@30%) vs inference speed. 2 Related Work 2.1 Unsupervised 2D Industrial Anomaly Detection Unsupervised 2D industrial anomaly detection methods [ 23 ] can be broadly grouped into reconstruction-based and feature-embedding-based approaches. Reconstruction-based models rely on autoencoders [ 37 , 1 , 4 , 16 , 31 , 42 ] , GANs [ 6 , 40 ] , inpainting networks [ 29 ] , or diffusion models [ 39 , 36 ] to reproduce nominal samples and identify anomalies through reconstruction discrepancies. While intuitive, these methods often struggle due to their tendency to partially reconstruct abnormal regions. Feature-embedding methods instead operate in the latent space [ 20 , 9 , 12 , 34 , 44 ] of deep networks. Some approaches, such as Deep Feature Reconstruction (DFR) [ 41 ] , train an autoencoder to reconstruct features extracted from normal samples and detect anomalies via feature-level reconstruction errors. Another popular line of work relies on frozen feature extractors [ 14 , 27 , 7 , 32 , 2 ] , features from normal images are stored in a memory bank during training [ 45 , 19 , 32 ] , and test features are compared against this bank to identify deviations. These memory-based approaches achieve strong performance

Read the complete text