Towards Efficient Pixel Labeling for Industrial Anomaly Detection and Localization
Jingqi Wu, Hanxi Li \sthanks Hanxi Li is the corresponding author. This project was partially supported by National Natural Science Foundation of China 62372150, Lin Wu, Hao Chen, Deyin Liu, Peng Wang · 2025
Read Towards Efficient Pixel Labeling for Industrial Anomaly Detection and Localization by Jingqi Wu, Hanxi Li \sthanks Hanxi Li is the corresponding author. This project was partially supported by National Natural Science Foundation of China 62372150, Lin Wu, Hao Chen, Deyin Liu, Peng Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-05034.txt.
Introduction
Labelling defective pixels on industrial products is slow, and that bottleneck limits anomaly detection training. Jingqi Wu, lead author with five co-authors, propose ADClick in a 2025 computer vision paper. Their interactive segmentation algorithm creates pixel-wise anomaly annotations from just a few clicks by the user, plus a short text description.
Those annotations improve anomaly model performance sharply on a standard benchmark in their tests. A companion framework, ADClick-Seg, aligns visual features with text prompts through prototypes. It reaches leading results on the hard multi-class anomaly task in the authors' evaluation, combining pixel priors with language guidance.
Catalog record
| CREATOR | Jingqi Wu, Hanxi Li \sthanks Hanxi Li is the corresponding author. This project was partially supported by National Natural Science Foundation of China 62372150, Lin Wu, Hao Chen, Deyin Liu, Peng Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-05034
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Efficient Pixel Labeling for Industrial Anomaly Detection and Localization Jingqi Wu, Hanxi Li \sthanks Hanxi Li is the corresponding author. This project was partially supported by National Natural Science Foundation of China 62372150, Lin Wu, Hao Chen, Deyin Liu, Peng Wang Abstract Industrial product inspection is often performed using Anomaly Detection (AD) frameworks trained solely on non-defective samples. Although defective samples can be collected during production, leveraging them usually requires pixel-level annotations, limiting scalability. To address this, we propose ADClick , an Interactive Image Segmentation (IIS) algorithm for industrial anomaly detection. ADClick generates pixel-wise anomaly annotations from only a few user clicks and a brief textual description, enabling precise and efficient labeling that significantly improves AD model performance (e.g., AP = 96.1% on MVTec AD). We further introduce ADClick-Seg , a cross-modal framework that aligns visual features and textual prompts via a prototype-based approach for anomaly detection and localization. By combining pixel-level priors with language-guided cues, ADClick-Seg achieves state-of-the-art results on the challenging “Multi-class” AD task (AP = 80.0%, PRO = 97.5%, Pixel-AUROC = 99.1% on MVTec AD). † † address: 1 Jiangxi Normal University, Jiangxi, China 2 Southern University of Science and Technology, GuangDong, China 3 Swansea University, Swansea, United Kingdom 4 Zhejiang University, Zhejiang, China 5 Anhui University, Jiangsu, China 6 Northwestern Polytechnical University, Shanxi, China 1 Introduction Automatic Optical Inspection (AOI) is essential in modern manufacturing. Most existing research adopts Anomaly Detection (AD) methods that avoid anomalous samples during training—not because such samples are unavailable, but because pixel-level annotation is prohibitively time-consuming. Since manual labeling is the primary bottleneck, models struggle to adapt quickly to newly observed defects. Some works generate synthetic anomalies for training [ 1 , 2 ] , while others show that real defect samples can improve accuracy [ 3 , 4 ] , though they still rely on costly pixel-level annotations. Recent studies attempt to reduce labeling costs with weak labels (e.g., bounding boxes) [ 3 ] , but the low precision leads to degraded performance. In this paper, we propose ADClick , a semi-automatic labeling tool that efficiently produces high-quality pixel-wise anomaly labels from a few user clicks and a short defect description. ADClick integrates multi-modal cues—image features, language prompts, and residual features—to generate dense anomaly masks at a cost comparable to weak labels. For example, annotating a scratch on a metal nut requires only 2–5 clicks plus a brief textual prompt. We further introduce ADClick-Seg , which embeds our multi-modal feature fusion module into anomaly detection, achieving state-of-the-art results under the challenging “Multi-class” setting. Extensive experiments confirm the effectiveness of our approach. Contributions: • We pioneer the use of Interactive Image Segmentation (IIS) for efficient anomaly labeling, enabling pixel-wise annotations at weak-label cost. • We enhance annotation quality via fusion of residual and linguistic features, yielding SOTA anomaly label generation. • We propose ADClick-Seg, a multi-modal detection framework that achieves new benchmarks in anomaly detection and localization. 2 Method Figure 1: The overview of the proposed ADClick. The model takes four input sources: the query image, the reference (defect-free) images, the language guidance, and the manual clicks. The dark-gray boxes labeled as ‘E” represent the linear embedding process. Better viewed in color. 2.1 Overview Recent raw-pixel-based Interactive Image Segmentation (IIS) methods have shown strong applicability in real-world scenarios [ 5 , 6 ] .Inspired by residual-based anomaly detection [ 7 , 3 ] , we extend this paradigm to interactive anomaly labeling by leveraging residual features, which are inherently more robust to noise and generalize better. Fig. 1 illustrates the architecture of ADClick. The model takes four inputs: the query image, reference (defect-free) images, language guidance, and user clicks. Residual features, termed “PosFAR” from WeakREST [ 3 ] , are fused with click embeddings and processed by a Swin Transformer backbone, where defect-specific linguistic features guide learning via cross-attention. In parallel, a ViT-based branch integrates clicks with raw-pixel information. The outputs of both branches are combined in a “Fusion & Decoder” module to predict dense anomaly masks, which are iteratively refined across clicks. Formally, given a normalized input image \text{I}_{\text{tst}}\in\mathbb{R}^{H_{I}\times W_{I}\times 3} and a sequence of clicks \mathcal{C}=\{{\bf C}_{t}=[x_{t},y_{t},\beta_{t}]^{\text{T}}\} with \beta_{t}\in\{0,1\} indicating positive (anomalous) or negative (normal) clicks, the anomaly mask \mathrm{M}_{t} at iteration t is updated as: \mathrm{M}_{t}=\Phi_{\text{Click}}(\mathfrak{R},\mathrm{M}_{t-1},{\bf C}_{t},\mathbf{l},\text{I}_{\text{tst}}), (1) where \mathfrak{R}\in\mathbb{R}^{h_{f}\times w_{f}\times d_{f}} denotes stored PosFAR features, \mathbf{l}\in\mathbb{R}^{Z} is the linguistic feature of the task description, and \mathrm{M}_{t-1} is the mask from the previous iteration. 2.2 Interactive Segmentation with Positional Fast Anomaly Residuals We adopt WeakREST [ 3 ] to generate residual features, termed “PosFAR,” which jointly capture global and local similarities between the test image and the reference bank. Specifically, we first extract a set of deep feature vectors from the test image \text{I}_{\text{tst}} : \{\mathbf{p}^{1}_{\text{tst}},\mathbf{p}^{2}_{\text{tst}},\cdots,\mathbf{p}^{M}_{\text{tst}}\}=\Psi_{\mathrm{PCF}}(\text{I}_{\text{tst}}), (2) where each \mathbf{p}^{j}_{\text{tst}}\in\mathbb{R}^{d_{f}} represents a Position Constrained Feature (PCF) [ 3 ] for the j -th patch. In the same way, anomaly-free references are extracted as \mathcal{P}_{\text{ref}}=\{\mathbf{p}^{1}_{\text{ref}},\mathbf{p}^{2}_{\text{ref}},\cdots,\mathbf{p}^{N}_{\text{ref}}\} . For each patch feature \mathbf{p}^{j}_{\text{tst}} , its Positional Fast Anomaly Residual (PosFAR) is computed as: \mathbf{r}_{j}=\lceil\text{ABS}(\mathbf{p}^{j}_{\text{tst}}-\mathbf{p}^{\ast}_{\text{ref}})\rceil^{\theta}\in\mathbb{R}^{d_{f}},\quad j=1,2,\dots,M, (3) where \text{ABS}(\cdot) denotes the element-wise absolute value, \lceil\cdot\rceil^{\theta} is the element-wise \theta -power operation, and \mathbf{p}^{\ast}_{\text{ref}} is the matched reference feature selected from \mathcal{P}_{\text{ref}} using the WeakREST strategy. Figure 2: The training of defect-specific linguistic features follows a simple yet effective workflow. Users only need to provide keywords to form templatized instructions, from which ChatGPT generates descriptive phrases. 2.3 Discriminative Linguistic Features Inspired by referring image segmentation [ 8 ] , we introduce defect-specific linguistic features to provide richer semantic cues beyond binary “defective/non-defective” labels, improving detection quality. As illustrated in Fig. 2 , candidate prompts are first generated using ChatGPT [ 9 ] with a templatized instruction, e.g., \text{``Give }U\text{ phrases describing the }\{def\}\text{ defect on a }\{obj\}." (4) where def and obj denote defect and object categories, and U controls the number of variants. Each prompt \phi is encoded with BERT [ 10 ] and further mapped into a compact linguistic representation \mathbf{l} via a trainable encoder: \phi\xrightarrow{\Psi_{\text{BERT}}}\mathbf{v}\xrightarrow{\Psi_{\text{En}}}\mathbf{l}\in\mathbb{R}^{Z}. (5) During inference, a randomly selected ChatGPT-generated description is used as input. The resulting linguistic feature \mathbf{l} is fused with visual residual features \mathbf{R}_{l} via cross-attention [ 11 ] : \mathbf{R}_{L}^{{}^{\prime}}=\text{softmax}\!\left(\tfrac{\Psi_{\text{Conv}}(\mathbf{R}_{l})^{T}\mathbf{W}_{K}\mathbf{l}}{\sqrt{d_{f}}}\right)\mathbf{W}_{V}\mathbf{l}, (6) where \Psi_{\text{Conv}}(\cdot) is a 1\times 1 convolution with normalization, and \mathbf{W}_{K} , \mathbf{W}_{V} are projection matrices. This integration injects fine-grained defect semantics into the AD process. 2.4 Dense Label Estimation We design a raw-pixel branch to complement residual features for dense label estimation.The proposed Fusion & Decoder module integrates cross-modal and image features through element-wise addition across multiple scales ( \frac{1}{32} – \frac{1}{4} of input resolution). Before fusion, cross-modal tensors are adapted using Zero-Convolution (ZC) layers [ 12 ] , effectively functioning as a ControlNet [ 12 ] for the raw-pixel branch. The fusion process is formalized as: \begin{split}\mathbf{F}^{img}_{i}&=\Psi^{img}_{\text{ConvNeck}_{i}}(\mathbf{F}_{N})\in\mathbb{R}^{d_{i}\times h_{i}\times w_{i}}~\forall i,\\ \mathbf{F}^{res}_{i}&=\Psi^{res}_{\text{ConvNeck}_{i}}(\mathbf{R}_{L}^{{}^{\prime}})\in\mathbb{R}^{d_{i}\times h_{i}\times w_{i}}~\forall i,\\ \mathbf{F}^{fus}_{i}&=\mathbf{F}^{img}_{i}+\Psi_{\text{ZC}_{i}}(\mathbf{F}^{res}_{i})\in\mathbb{R}^{d_{i}\times h_{i}\times w_{i}}~\forall i,\\ \end{split} (7) where \mathbf{F}_{N} is the output of the ViT image branch and \mathbf{R}_{L}^{{}^{\prime}} is the residual feature fused with linguistic cues (Eq. 6 ). \Psi_{\text{ConvNeck}i} extracts features at different scales, while \Psi_{\text{ZC}_{i}} adapts residual features for stable fusion. Multi-scale fused features {\mathbf{F}^{fus}_{i}} are then aggregated and decoded into dense label predictions. 2.5 Cross-modal Anomaly Detection and Localization We further enhance ADClick—termed