A Hybrid Framework for Statistical Feature Selection and Image-Based Noise-Defect Detection
Alejandro Garnung Menéndez, M.Sc., PhD Candidate · 2024
Read A Hybrid Framework for Statistical Feature Selection and Image-Based Noise-Defect Detection by Alejandro Garnung Menéndez, M.Sc., PhD Candidate (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-08800.txt.
Introduction
Alejandro Garnung Menéndez presents a 2024 framework for telling real surface flaws from noise in factory images. In harsh plant settings, specks and grain can mimic true defects, and false calls cost money. The framework scores each suspect region for how likely it is a flaw or mere noise. About 55 traits are pulled from each image region to feed that score.
Stats tests then pick the traits that split flaws from noise most cleanly. One classic split rule anchors the set for speed and trust. The module can run alone or sit on top of a learning-based sorter as a check layer. Its trait logic stays open to view, unlike black-box sorts. Plants gain a QC tool they can audit and tune.
Catalog record
| CREATOR | Alejandro Garnung Menéndez, M.Sc., PhD Candidate |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 104 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-08800
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Hybrid Framework for Statistical Feature Selection and Image-Based Noise-Defect Detection Alejandro Garnung Menéndez, M.Sc., PhD Candidate Abstract In industrial imaging, accurately detecting and distinguishing surface defects from noise is critical and challenging, particularly in complex environments with noisy data. This paper presents a hybrid framework that integrates both statistical feature selection and classification techniques to improve defect detection accuracy while minimizing false positives. The motivation of the system is based on the generation of scalar scores that represent the likelihood that a region of interest (ROI) is classified as a defect or noise. We present around 55 distinguished features that are extracted from industrial images, which are then analyzed using statistical methods such as Fisher separation, chi-squared test, and variance analysis. These techniques identify the most discriminative features, focusing on maximizing the separation between true defects and noise. Fisher’s criterion ensures robust, real-time performance for automated systems. This statistical framework opens up multiple avenues for application, functioning as a standalone assessment module or as an a posteriori enhancement to machine learning classifiers. The framework can be implemented as a black-box module that applies to existing classifiers, providing an adaptable layer of quality control and optimizing predictions by leveraging intuitive feature extraction strategies, emphasizing the rationale behind feature significance and the statistical rigor of feature selection. By integrating these methods with flexible machine learning applications, the proposed framework improves detection accuracy and reduces false positives and misclassifications, especially in complex, noisy environments. 1 Introduction The detection and classification of noise in images is crucial for numerous industrial applications, particularly in automated inspection systems. Image processing has always highlighted the need for effective noise classification methods that can distinguish between noise and actual defects. Many works address the intricate problem of differentiating between noise and actual surface defects in industrial parts, which frequently requires in-depth statistical analysis to uncover subtle differences that are not immediately apparent [ 1 ] . Moreover, feature selection plays a critical role in enhancing machine learning (ML) performance by reducing false positives. This article focuses on statistical methods for feature selection in image-based defect detection and the usage of these enhanced features that help build an effective discriminative method for defect and noise classification. These techniques aim to identify the most effective features that can distinguish between true positives or true defects (TP) and false positives or false defects (FP) in detection tasks. Specifically, the aim is to develop a dichotomous image classification methodology based on specific scalar features extracted from candidate images. The proposed hybrid framework acts as a black-box model that integrates multiple analytical methods which can be assembled on top of existing models to classify ROIs as noise or defects. For example, it enables the evaluation of the outputs of other models using simple threshold-based methods or by training a model such as a random forest (RF). On the one hand, this study presents a comprehensive methodology for feature selection aimed at optimizing the separation between TP and FP in a defect detection system, specialized in the morphology of highly noisy images degraded by random and impulse noise of defects captured with laser sensors. The analysis begins with a set of approximately 55 extracted features, seeking to identify those that best differentiate between the two classes while ensuring long-term reliability. The primary objective is to retain the features where the distributional means of TP and FP are maximally separated, to allow efficient and straightforward threshold-based classification. Using statistical methods, our aim is to develop a robust feature selection process that can be applied to various supervised, unsupervised, or hand-crafted classification methods, enhancing their precision and generalizability across different types of defects and environments. This integration, based on statistical intuition, enhances the system’s adaptability to complex real-world scenarios, ensuring reliability across diverse applications. On the other hand, we present a novel approach that leverages this wide range of analytical metrics, combining them into a simple yet ad hoc weighted scoring system. One could further enhance and generalize this system using machine learning models to optimize decision making. This capitalizes on inherent advantages: robustness to noise, ability to handle complex datasets, flexibility in managing various types of input characteristics, and feature selection based on its relative importance. In this context, our primary concern is that both noise and defects occupy a very small portion of the dynamic range, resulting in inherently low contrast, which makes classification challenging. However, we can capitalize on the fact that our type of noise is often accompanied by point losses, typically represented by values below the three-bit digital threshold. We aim to characterize and differentiate the structure and arrangement of these losses, as they seem to exhibit distinct patterns compared to the defects analyzed in this study. 2 Related work The objective of this article is to contribute to the literature by providing a repository of useful methods for the detection and classification of noise under particularly unfavorable scanning conditions where defect detection (such as cracks, dents, scratches, chips, etc.) is required. In these scenarios, the geometric structure of the noise and the defects themselves is often indistinguishable with the naked eye. In [ 2 ] the author reviews the evolution of image noise reduction techniques, focusing on detection, measurement, and removal, with an emphasis on noise measurement and denoising methods. The work explores measurement approaches such as filter-based, block-based, and wavelet-transform methods, along with other significant advancements. Noise removal techniques are discussed across four historical phases, from the 1970s to the early 2010s, transitioning through spatial and transform domains, local and non-local statistical methods, and recent innovations like total variation minimization, sparse coding, and deep learning. The article highlights a shift from spatial to transform domains, from local to non-local approaches, and from thresholding to optimization. Despite the evolution of techniques, the primary goal remains unchanged: smoothing images while maintaining their essential details. The authors in [ 3 ] introduce a new image statistic designed to detect random-valued impulse noise in images. This statistic enables the identification of most noisy pixels in corrupted images, and when combined with an edge-preserving regularization, it forms a powerful two-stage denoising method that effectively handles high noise levels. The effectiveness of the method has been shown to reduce false detection of noisy pixels while maintaining high image quality, but remains restricted to a denoising application. Recently, in [ 4 ] a method for detecting and classifying various types of image noise, such as Gaussian, log-normal, Rayleigh, salt and pepper and speckle [ 5 ] was proposed. The method uses a two-stage approach: first, a Convolutional Neural Network (CNN) detects the noise, and then machine learning classifiers, combined with a deep wavelet model, classify the noise. Moreover, [ 6 ] proposes a framework for estimating the noise level in natural images using statistical analysis, focusing on high kurtosis and scale invariance in the transform domain, examining the limitations of these statistics, especially for images with highly directional edges or large smooth areas. They combine wavelets, non-directional discrete cosine transform (DCT) and noise-injection rectification to estimate noise variance across a wide range of image content and noise levels. However, the scope of the method may be limited to enhancing denoising. In [ 7 ] ,