A Semantically Disentangled Unified Model for Multi-category 3D Anomaly Detection

SuYeon KimWongyu Lee, MyeongAh ChoKyung Hee University, Republic of Korea {spoiuy3, fovert, maycho}@khu.ac.kr · 2026

Read A Semantically Disentangled Unified Model for Multi-category 3D Anomaly Detection by SuYeon KimWongyu Lee, MyeongAh ChoKyung Hee University, Republic of Korea {spoiuy3, fovert, maycho}@khu.ac.kr (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25159.txt.

Introduction

Kim, Lee, and Cho build one model that spots defects in 3D scans across many object types. Training uses only normal point clouds. A single shared model tends to blur types together, so rebuilt shapes borrow the wrong traits and scores turn shaky. Their design fights that blur in three ways. Tokens give each item a semantic name tag. Contrast training pulls type meanings apart. A geometry-guided decoder then rebuilds each scan true to its type. On two benchmarks it tops both shared and per-type models.

Catalog record

CREATORSuYeon KimWongyu Lee, MyeongAh ChoKyung Hee University, Republic of Korea {spoiuy3, fovert, maycho}@khu.ac.kr
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61724
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.25159

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Semantically Disentangled Unified Model for Multi-category 3D Anomaly Detection SuYeon KimWongyu Lee, MyeongAh ChoKyung Hee University, Republic of Korea {spoiuy3, fovert, maycho}@khu.ac.kr Abstract 3D anomaly detection targets the detection and localization of defects in 3D point clouds trained solely on normal data. While a unified model improves scalability by learning across multiple categories, it often suffers from Inter-Category Entanglement (ICE) —where latent features from different categories overlap, causing the model to adopt incorrect semantic priors during reconstruction and ultimately yielding unreliable anomaly scores. To address this issue, we propose the Semantically Disentangled Unified Model for 3D Anomaly Detection, which reconstructs features conditioned on disentangled semantic representations. Our framework consists of three key components: (i) Coarse-to-Fine Global Tokenization for forming instance-level semantic identity, (ii) Category-Conditioned Contrastive Learning for disentangling category semantics, and (iii) a Geometry-Guided Decoder for semantically consistent reconstruction. Extensive experiments on Real3D-AD and Anomaly-ShapeNet demonstrate that our method achieves state-of-the-art for both unified and category-specific models, improving object-level AUROC by 2.8% and 9.1%, respectively, while enhancing the reliability of unified 3D anomaly detection. Project page: https://spoiuy3.github.io/SeDiR/ Figure 1 : Overview of different paradigms for 3D anomaly detection. (a) Category-specific models trained on each object class separately. (b) A unified model handles multiple categories with local features but often suffers from inter-category feature entanglement. (c) Our method achieves semantically consistent results by aggregating coarse-to-fine geometric cues into category-aware global features, disentangling them via C3L, and guiding reconstruction with input geometry. † † footnotetext: † Corresponding author 1 Introduction 3D anomaly detection (3D-AD) aims to identify defective objects and localize abnormal regions within 3D point clouds, typically trained using only normal data. Conventional methods 19 ; 46 ; 16 ; 45 are category-specific, where a separate model is trained for each object type ( Fig. 1 (a)). These models—most commonly based on encoder–decoder reconstruction or memory matching—learn the normal pattern distribution of a single category. During inference, reconstruction-based methods detect anomalies when the model fails to reconstruct an abnormal sample, whereas memory-based approaches identify anomalies when the feature representation of an abnormal sample deviates significantly from the stored normal patterns. Although effective under controlled conditions, this paradigm has two major limitations. First, it suffers from transfer collapse —the model becomes overly specialized to a specific geometric pattern of normal training samples and fails to generalize over shape or texture variations. Second, it is impractical in real industrial scenarios involving multiple object categories, as it requires duplicating training, storage, and maintenance pipelines for each class. To address these constraints, a recent study 3 has explored unified 3D-AD framework, where a single model is jointly trained across multiple categories ( Fig. 1 (b)). Such unified design reduces system redundancy and improves deployment efficiency, emerging as a promising direction for practical 3D anomaly detection. A unified model offers improved practicality and even higher accuracy, yet a critical question remains— can a model truly detect anomalies without first establishing an object’s identity? In unified 3D-AD, the model is expected to learn normal shape priors across multiple categories and detect anomalies through reconstruction errors. However, our observations reveal a deeper issue: even when trained only on normal data, the model sometimes reconstructs objects using the wrong category prior ( e.g . , a chair partially reconstructed with table-like geometry). This does not indicate a failure to detect anomalies, but rather a failure to establish the object’s identity before reconstruction. We define this phenomenon as Inter-Category Entanglement ( ICE ) —a state in which features from different categories become entangled in the latent space, preventing the formation of distinct, category-specific representations. When the ICE problem occurs, the model loses semantic identity, and reconstruction is guided by an incorrect category prior. This semantic misalignment propagates through the entire pipeline, leading to geometrically inconsistent outputs and degraded performance in both anomaly detection and localization. Figure 2 : T-SNE visualization and quantitative analysis of semantic disentanglement. MC3D-AD exhibits entangled feature clusters across similar categories ( e.g . , chicken, duck, gemstone ), where samples with low category classification scores are reconstructed under uncertain semantic priors, yielding high reconstruction errors on normal data and increased false positives. Our model instead forms well-separated, semantically aligned manifolds with higher classification scores and lower reconstruction errors, enabling reliable category-conditioned reconstruction. Figure 3 : The overview of the proposed method. Our method consists of two main stages: Semantically Disentangled Representation Learning and Semantically Disentangled Reconstruction. Given an input point cloud, CFGT encodes multi-resolution geometric features into a category-aware global token, C3L disentangles the latent semantics, and GGD reconstructs the object conditioned on these disentangled semantics and geometric priors. To validate the presence of the ICE problem, in Fig. 2 , we perform both qualitative and quantitative analyses using MC3D-AD 3 . We first project encoder features into t-SNE space 23 and observe that categories such as chicken, duck, and gemstone form heavily overlapping clusters rather than distinct manifolds. Furthermore, to quantify the category classification score, we freeze the trained model and train a lightweight MLP on the reconstructed features. By analyzing the relationship between the classification score and the reconstruction error on normal test samples, we observe that samples with low semantic score tend to exhibit significantly higher reconstruction errors. These observations confirm that incorrect semantic understanding leads to incorrect reconstruction, demonstrating that the ICE problem is not a random side effect but a systematic bottleneck in unified 3D-AD. The above observations indicate that the fundamental limitation of unified 3D-AD is not the difficulty of reconstructing anomalies, but the attempt to reconstruct before understanding what is being reconstructed. In other words, reconstruction fails not because the object is anomalous, but because the model lacks a clear semantic identity of the object in the first place. This misalignment—where geometry is reconstructed under an incorrect category prior—leads to unreliable anomaly scores and mislocalized defects. Therefore, we reformulate unified 3D-AD as a problem of semantically conditioned reconstruction and propose the paradigm of Semantically Disentangled Reconstruction ( Fig. 1 (c)), where reconstruction is conditioned on correctly disentangled, category-level semantics rather than indiscriminately applied to all latent representations. To operationalize Semantically Disentangled Reconstruction, we design a semantically disentangled unified framework, named SeDiR , that explicitly separates semantic identity from geometry before attempting reconstruction. We propose the Coarse-to-Fine Global Tokenization (CFGT) module, which aggregates multi-resolution geometric cues into a global token that captures category-aware semantics rather than relying solely on local point features. This global representation enables the model to form a reliable semantic identity of the object before any decoding takes place. To further prevent different categories from collapsing into a shared latent space, we introduce Category-Conditioned Contrastive Learning (C3L) , which explicitly enforces intra-category compactness and inter-category separation to form distinct semantic manifolds, directly mitigating Inter-Category Entanglement. Finally, we propose a Geometry-Guided Decoder (GGD) that performs reconstruction guided by both the disentangled semantic prior and local geometric evidence. Instead of decoding blindly from latent features, the decoder is softly biased towards category-consistent reconstruction pathways, ensuring that geometry is reconstructed according to the correct semantic identity. Together, these components enable the model to understand what it is reconstructing before deciding how to reconstruct, establishing a new perspective and solution space for unified 3D anomaly detection. Extensive experiments validate the

Read the complete text