Hypergraph Normal World Models for Logical Visual Anomaly Detection
Weizhi Nie, Zibo Xu, Weijie Wang, Yuting Su · 2026
Read Hypergraph Normal World Models for Logical Visual Anomaly Detection by Weizhi Nie, Zibo Xu, Weijie Wang, Yuting Su (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25368.txt.
Introduction
Factory inspection must flag products that look fine piece by piece but are assembled wrong, like a breakfast box holding the wrong count of items. Weizhi Nie, Zibo Xu, Weijie Wang, and Yuting Su propose a detector in 2026 that learns only from normal images. Frozen features from a vision model are distilled into patch, relation, and hypergraph statistics, where hyperedges link groups of patches that belong together. A combined score then weighs local, relational, and group evidence separately.
On validation data for a breakfast box task, the full model lifted logical anomaly detection well above a nearest neighbour baseline using the same features. Removing the hypergraph parts dropped the score partway back. The detector also held up with very few normal training images, and scrambling the relations raised its alarm sharply, showing the score tracks real structure.
Catalog record
| CREATOR | Weizhi Nie, Zibo Xu, Weijie Wang, Yuting Su |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.25368
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hypergraph Normal World Models for Logical Visual Anomaly Detection Weizhi Nie, Zibo Xu, Weijie Wang, Yuting Su Abstract Visual anomaly detection is often deployed with only normal training images. Most one-class detectors map test patches or features to a normal reference distribution. This works well for local structural defects. Logical anomalies are different. Each visible part may look normal, while the whole image violates a normal count, co-occurrence, or spatial relation. This paper studies whether a model can learn such a category-specific normal world from nominal images alone. We propose the Hypergraph Normal World Model, a normal-only detector that distills frozen DINOv2 patch tokens into patch, relation, and hypergraph statistics. It builds spatial hyperedges over token groups. It then scores each test image with an information quotient that separates local, relational, hyperedge, and hyperedge-relation evidence. On the available MVTec LOCO breakfast-box validation data, the full hypergraph model improves logical anomaly AUROC from 0.8434 for DINOv2 patch-kNN to 0.9279. It also improves over the non-hypergraph variant, from 0.9013 to 0.9279. Few-shot experiments show that the model remains effective with very limited normal images. We also test whether the score reflects normal-world knowledge rather than a shallow mapping. t-SNE separates logical anomalies in the learned energy space. Relation counterfactuals increase the information quotient by 83.13 on average. Random hypergraphs reduce logical AUROC, and hyperedge attribution is much larger on logical anomalies. Qualitative examples show that high scores are driven by relation-bearing terms. These results suggest that logical visual anomaly detection should model normal relations, not only normal local patches. 1 Introduction Visual anomaly detection aims to decide whether a test image is consistent with the normal state of an object, product, or visual environment. In practical inspection systems, the data distribution is highly unbalanced. Normal samples are easy to collect because they are produced by the regular process. Abnormal samples are rare, diverse, and often unavailable before deployment. Even when a few abnormal examples exist, they cannot cover the space of possible failures. This reality has made one-class, few-shot, and zero-shot anomaly detection an important problem in computer vision. Most existing methods therefore avoid training a standard supervised classifier. They instead learn a mapping from an input image, patch, or feature to a normal reference. This reference may be a nearest-neighbor memory bank, a Gaussian patch distribution, a reconstruction target, a flow likelihood, a teacher-student discrepancy, or a vision-language prompt response [ 6 , 7 , 25 , 30 , 29 , 8 , 15 , 1 , 31 ] . This line of work has produced strong detectors, especially when the anomaly creates visible local evidence such as a scratch, stain, missing texture, or broken surface. However, the mapping view still has an important limitation. It asks whether a test image can be matched to, reconstructed from, or classified against a set of learned normal references. When the number of normal images is small, this reference set is incomplete. When the abnormal space is open ended, no mapping can cover all invalid cases. More importantly, some failures are not failures of local appearance. They are failures of the visual world’s internal rules. An image may be composed of normal-looking parts, but the parts may have a wrong count, a wrong co-occurrence pattern, an impossible order, or an inconsistent spatial relation. MVTec LOCO AD was designed to expose this issue by separating logical anomalies from structural anomalies [ 3 ] . In this setting, the key question is not only whether each patch has a familiar appearance. The key question is whether the image can be explained as a valid normal world. This paper addresses the problem of logical visual anomaly detection from normal images only. We study whether a model can learn the normal world of one visual category without abnormal examples. We also test whether the learned world can reject logically invalid images. The problem is difficult for two reasons. • Challenge 1. Few normal samples provide incomplete coverage. In a normal-only setting, the model cannot rely on abnormal labels. In a few-shot setting, it also lacks dense coverage of normal appearances. A direct mapping from test patches to stored normal patches is therefore fragile. It can memorize the observed normal samples, but it may fail to describe the latent rule that makes those samples normal. A detector needs a prior that is broader than the target training set, while still being calibrated to the target category. • Challenge 2. Single-category normality is relational, not only local. Logical anomalies are created by invalid configurations of normal parts. A detector must learn what is essential in the normal category. It must know which regions should appear, which parts should co-occur, which groups should be compatible, and which relations should remain stable. This knowledge cannot be evaluated only by a local distance to the nearest normal patch. The score should measure how much information is required to explain the whole image under the learned normal world. To address the first challenge, we use a visual foundation model as a source of world knowledge and distill a category-specific normal world from it. A large self-supervised model has already learned broad visual regularities from massive unlabeled data. Its patch tokens contain semantic cues about objects, parts, and layouts that cannot be learned from a few target images alone. We therefore do not train a visual encoder from scratch. Instead, we freeze the encoder and use the available normal images to estimate what the target category looks like inside this pretrained feature space. This gives the model a low-shot prior and keeps the normal-only protocol. To address the second challenge, we treat a normal image as a structured world rather than a bag of independent patches. The normal category is represented by local tokens, larger region groups, and relations among these groups. If a test image is normal, these elements should be easy to explain under the statistics learned from normal images. If the image is logically abnormal, the local tokens may still be familiar, but the relational structure should become inefficient to encode. We capture this idea with an information quotient. The quotient measures the relative explanation cost of a test image under the normal world. It is high when patch groups or cross-region relations deviate from the normal relational code. This can happen even when many individual patches remain visually plausible. Figure 1 illustrates this motivation. Figure 1: Motivation. Patch-memory baselines are strong local normality models. Logical anomalies can preserve local appearance while breaking relations among parts. We model the normal visual world with hyperedges over patch tokens and use an information quotient to measure relation violations. Based on these ideas, we propose the Hypergraph Normal World Model. The method is a normal-only anomaly detector built on frozen DINOv2 patch tokens [ 23 ] . It turns the conceptual goal of learning a normal world into a concrete scoring pipeline. • First, we distill general visual knowledge into target normal tokens. The model extracts a grid of semantic patch tokens from each image with a frozen DINOv2 encoder. The encoder is not updated on the target data. The target normal images are used only to calibrate normal-world statistics in the pretrained feature space.