Probing Network Decisions: Capturing Uncertainties and Unveiling Vulnerabilities without Label Information

Youngju Joung, Sehyun Lee ⋆ \star, Jaesik Choi · 2025

Read Probing Network Decisions: Capturing Uncertainties and Unveiling Vulnerabilities without Label Information by Youngju Joung, Sehyun Lee ⋆ \star, Jaesik Choi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-09068.txt.

Introduction

Youngju Joung, Sehyun Lee, and Jaesik Choi present a 2025 way to probe a neural image classifier for weak spots without using the true labels. Checking single images one by one takes human effort and can bend with the checker's own bias, above all for images the model got wrong.

Their frame trains a second, small prober that learns to guess, from the model's internal signals, whether its call on an image is right or wrong. That prober then guides the crafting of altered images that slip past it. Tests on standard image sets show the prober spots bad calls well. Fooling the prober also exposed real flaws in the main model, shown on handwritten digit images, all without reading the answer key.

Catalog record

CREATORYoungju Joung, Sehyun Lee ⋆ \star, Jaesik Choi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-09068

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Probing Network Decisions: Capturing Uncertainties and Unveiling Vulnerabilities without Label Information Youngju Joung, Sehyun Lee ⋆ \star, Jaesik Choi Abstract To improve trust and transparency, it is crucial to be able to interpret the decisions of Deep Neural classifiers (DNNs). Instance-level examinations, such as attribution techniques, are commonly employed to interpret the model decisions. However, when interpreting misclassified decisions, human intervention may be required. Analyzing the attributions across each class within one instance can be particularly labor-intensive and influenced by the bias of the human interpreter. In this paper, we present a novel framework to uncover the weakness of the classifier via counterfactual examples. A prober is introduced to learn the correctness of the classifier’s decision in terms of binary code - hit or miss . It enables the creation of the counterfactual example concerning the prober’s decision. We test the performance of our prober’s misclassification detection and verify its effectiveness on the image classification benchmark datasets. Furthermore, by generating counterfactuals that penetrate the prober, we demonstrate that our framework effectively identifies vulnerabilities in the target classifier without relying on label information on the MNIST dataset. Interpretable machine learning is crucial for explaining the decisions of Deep Neural classifiers (DNNs). In particular, mission-critical systems in the real world, such as autonomous driving or AI-assisted medical diagnosis programs, should provide suitable reasons why the classifier makes such decisions. However, it is challenging to comprehend the decisions due to the complexity of these models. For instance-level explanation, the attribution techniques are commonly employed to highlight the influence of input features on the model decisions. By observing the gradient or activations of the classifiers, the decisions can be interpreted by calculating the saliency of input features. However, it requires human intervention to interpret the salient features since the techniques produce multiple attributions across the classes. Moreover, some cases are reported where the input attribution techniques show disagreement within the same classifier by issuing the robustness of explanations [ 14 , 20 ] Figure 1: Overview of the proposed framework to investigate the misclassified samples. In this example, the sample is classified as 9 while the true label is 4. Given the hidden representation of the layer of the instance, the prober predicts whether the classifier is hit or miss . Then, the counterfactual example is generated through the classifier and the prober. In this process, obstructive features are modified, contributing to the reduction of uncertainty in the classifier’s confidence. Our goal is to reduce human interference and produce more objective explanations in interpreting the classifier. The main idea involves introducing a prober to encode the classifier’s decision into binary code ( hit and miss ) so that it simplifies the number of target classes. In detail, hit represents cases where the classifier correctly predicts the answer, while miss indicates cases where the classifier fails to predict the correct answer. This binary encoding approach reduces the effort required for users to compare interpretations across various classes. In this paper, we propose a simple yet novel framework to discover vulnerabilities in the classifier at the instance-level. Our contributions are as follows. First, by introducing a shallow Feed Forward Network (FFN) called a prober, we conduct detection for misclassified samples. Additionally, we construct a Hit - Miss Dataset for the experiment and explore the evidence behind how the prober operates effectively. Second, through analyzing the prober, we indirectly investigate the classifier to identify obstructive features that confuse the classifier. Since our method employs a binary encoding approach, it is enable to access potential labels even in opaque scenarios where the true label is unknown. Moreover, the advantage of our framework demonstrates the possibility of facilitating auto-correction capabilities for the classifier. 1 Related Work Prober . Probers are widely applied for testing specific properties of the internal representations within a neural classifier. Exploring the internal workings of a neural classifier is typically challenging, so employing a relatively simple prober for indirect analysis is preferred. [ 1 , 3 , 21 ] . In the case of Natural Language Processing (NLP), probing techniques are extensively utilized to evaluate knowledge of Language Models (LM). For example, detecting true-false relationships can be employed to validate the model’s ability to handle hallucinations [ 2 ] . Predicting verb tense is used to assess the model’s understanding of sentence structure [ 24 ] . To prevent additional biases and ensure the simplicity of the prober, a simple feedforward classifier is typically employed to measure probing accuracy. Misclassification Detection . Misclassification detection aims to identify the misclassified samples from the given classifier to enhance the reliability of confidence estimation. It is closely related to out-of-distribution (OOD) detection, but misclassification is not constrained to whether the sample comes from in-distribution or out-of-distribution explicitly. A baseline of misclassification detection is using the maximum softmax probability [ 10 ] . OpenMix [ 26 ] proposed utilizing the outlier exposure method [ 11 ] for more reliable misclassification detection. However, extra actions are still required to interpret the misclassified samples. Counterfactual Explanations . The counterfactual-based explanation [ 16 , 23 ] method aims to explain the causal relationship between input and the model’s final decision. In the classification task, to understand why the input could not be classified into another class, a situation opposite to what is actually observed is assumed. As counterfactual examples aim to provide semantically different examples compared to adversarial examples, a generative model is employed to generate more semantically meaningful counterfactual examples [ 8 ] . Specifically, they propose the Approximated Diffeomorphic Counterfactuals (ADC) approach, utilizing normalizing flow to find optimal values in the latent space of the generator along the data manifold. We embrace the ADC method for discovering and analyzing vulnerabilities in the classifier. 2 Methodology In this section, we first suggest the proposed framework to provide an explanation through counterfactual examples representing a likely hit decision from the perspective of the classifier as illustrated in Figure 1 . Then, we present the following three core elements comprising this framework including a Hit - Miss datasets, a prober, and a counterfactual generator. 2.1 Probing Framework In this subsection, we present an overview of the probing framework along with relevant notations. The framework is designed to work with a classifier f of interest, initially trained for a specific task x\mapsto\hat{y} , where x represents the input and \hat{y} is the prediction. The primary focus of this paper is on the image classification task, but it should be noted that the framework is versatile and adaptable to diverse domains and tasks. The classifier evaluates an annotated dataset \mathcal{D}=\{x^{(i)},y^{(i)}\} and generates the intermediate representation of x at the layer l , denoted as f_{l}(x) . To encode the classifier’s decision into binary code, a probing label o is scored to indicate whether the classifier predicts the class correctly or incorrectly. Following this, a prober h:f_{l}(x)\mapsto\hat{o} is trained on the Hit - Miss dataset \mathcal{D}_{P}=\{f_{l}(x)^{(i)},o^{(i)}\} , which consists of representations and the corresponding probing label. The prober allows for the indirect analysis of the obstructive feature that straightforwardly hindered the prediction of the classifier. 2.2 The Hit - Miss Dataset In the context of our work, we leverage a Hit - Miss dataset \mathcal{D}_{P}=\{f_{l}(x)^{(i)},o^{(i)}\} encoded

Read the complete text