Set-Valued Sensitivity Analysis of Deep Neural Networks

Xin Wang, Feilong Wang, Xuegang (Jeff) Ban · 2024

Read Set-Valued Sensitivity Analysis of Deep Neural Networks by Xin Wang, Feilong Wang, Xuegang (Jeff) Ban (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-11057.txt.

Introduction

Training a neural network rarely lands on one unique answer: small changes in the data can push the weights to a different minimum. Wang, Wang, and Ban argue that sensitivity analysis should therefore track the whole set of solutions, not a single trained model. They build a set-valued framework that asks how that solution set grows or shrinks when the training data are nudged.

If the set's movement can be bounded by the size of the nudge, the model earns a Lipschitz-like stability property. The framework supplies set-level tools for distance, convergence, and derivatives, and it does not require the loss surface to be nicely curved at the minimum. They prove the property for fully connected networks. For general networks such as ResNet, they give a graphical-derivative method that estimates the new solution set after a data change without retraining.

Catalog record

CREATORXin Wang, Feilong Wang, Xuegang (Jeff) Ban
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-11057

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Set-Valued Sensitivity Analysis of Deep Neural Networks Thanks: This paper has been accepted to the 39th Annual AAAI Conference on Artificial Intelligence Xin Wang, Feilong Wang, Xuegang (Jeff) Ban Abstract This paper proposes a sensitivity analysis framework based on set-valued mapping for deep neural networks (DNN) to understand and compute how the solutions (model weights) of DNN respond to perturbations in the training data. As a DNN may not exhibit a unique solution (minima) and the algorithm of solving a DNN may lead to different solutions with minor perturbations to input data, we focus on the sensitivity of the solution set of DNN, instead of studying a single solution. In particular, we are interested in the expansion and contraction of the set in response to data perturbations. If the change of solution set can be bounded by the extent of the data perturbation, the model is said to exhibit the Lipschitz-like property. This ’set-to-set’ analysis approach provides a deeper understanding of the robustness and reliability of DNNs during training. Our framework incorporates both isolated and non-isolated minima, and critically, does not require the assumption that the Hessian of loss function is non-singular. By developing set-level metrics such as distance between sets, convergence of sets, derivatives of set-valued mapping, and stability across the solution set, we prove that the solution set of the Fully Connected Neural Network holds Lipschitz-like properties. For general neural networks (e.g. Resnet), we introduce a graphical-derivative-based method to estimate the new solution set following data perturbation without retraining. 1 Introduction Sensitivity analysis is a classical research topic that crosses optimization [ 8 ] , machine learning, and deep learning [ 22 , 10 , 3 ] . It studies how the solution of a model responds to minor perturbations in hyperparameters or input data. For instance, Christmann et al. [ 3 ] proved that the solution of the classification model with convex risk function is robust to the bias in data distribution. In this paper, we focus on deep neural networks (DNN), for which the solution is the weights of DNN trained (optimized) on input data. We are concerned about how the solution of DNN responds to perturbations in the input data in the training stage of the model. In the domain of deep learning (e.g., DNN), sensitivity analysis has started to draw attention due to its wide range of applications, such as designing effective data poisoning attacks [ 15 , 13 ] , evaluating the robustness of models [ 21 ] , and understanding the impact of important features in training data on the prediction [ 10 ] . Define a neural network f:\mathcal{X}\rightrightarrows\mathcal{Y} , where \mathcal{X} (e.g., images) is the input space and \mathcal{Y} is the (e.g., labels) output space. Given training data samples x=[x_{1},x_{2},\dots,x_{n}] and y=[y_{1},y_{2},\dots,y_{n}] , and the loss function L , the empirical risk minimizer is given by \hat{w}\stackrel{{\scriptstyle\text{ def }}}{{=}}\arg\min_{w\in\mathcal{W}}\frac{1}{n}\sum_{i=1}^{n}L\left({x}_{i},{y}_{i},w\right) . This paper assumes that we perturb only the features, keeping the label constant. The learning process from data to local minimizers thus can be formulated as a set-valued mapping S:\mathcal{X}^{n}\rightrightarrows\mathcal{W} 1 1 1 Here, \mathcal{X}^{n} denotes the n -fold Cartesian product of \mathcal{X} , i.e., \mathcal{X}^{n}=\mathcal{X}\times\mathcal{X}\times\cdots\times\mathcal{X} ( n times). , S({x})=\left\{\hat{w}|\hat{w}\stackrel{{\scriptstyle\text{ def }}}{{=}}\arg\min_{w\in\mathcal{W}}\frac{1}{n}\sum_{i=1}^{n}L\left({x}_{i},{y}_{i},w\right)\right\}. (1) For the unperturbed feature \bar{x}=[\bar{x}_{1},\bar{x}_{2},\dots,\bar{x}_{n}] and \bar{w}\in S(\bar{x}) , current sensitivity analysis [ 10 , 16 , 3 ] aims to study the change of S(\bar{x}) when an individual point is perturbed. For example, if the feature \bar{x} is perturbed to \hat{x}=[\bar{x}_{1},\bar{x}_{2},\dots,x_{p},\dots,\bar{x}_{n}] , the sensitivity analysis can be conducted by examining a limit: \lim_{x_{p}\rightarrow\bar{x}_{p}}\frac{S\left(\hat{x}\right)-S(\bar{x})}{\|x_{p}-\bar{x}_{p}\|}. (2) Most current sensitivity analysis methods for DNN suffer from the following two issues. First, to figure out the sensitivity of solution {w} w.r.t. data {x}_{p} , one of the most common approaches (e.g., the influence function [ 10 ] ) is to apply the Dini implicit function theorem [ 6 ] to the optimality condition of the model: \frac{1}{n}\sum_{i=1}^{n}\nabla_{w}L\left(\bar{x}_{i},\bar{y}_{i},\bar{w}\right)=0 , which leads to: \displaystyle\lim_{x_{p}\rightarrow\bar{x}_{p}}\frac{S\left(\hat{x}\right)-S(\bar{x})}{\|x_{p}-\bar{x}_{p}\|} \displaystyle=\nabla_{x_{p}}w\bigg|_{x_{p}=\bar{x}_{p}} (3) \displaystyle=-H_{\bar{w}}^{-1}\nabla_{x_{p}}\nabla_{\theta}L(\bar{x}_{p},\bar{y}_{p},\bar{w}), where H_{\bar{w}}^{-1}=\frac{1}{n}\sum_{i=1}^{n}\nabla_{w}^{2}L\left(\bar{x}_{i},\bar{y}_{i},\bar{w}\right) is the Hessian. However, the application of the Dini implicit function theorem is invalid when the Hessian H_{\bar{w}} is not invertible due to the often non-locally strong convex loss function L of DNN [ 12 ] . Second, the current approaches [ 10 , 16 ] assume S is a single-valued mapping, omitting the fact that S is often set-valued. Some findings have demonstrated that DNN may not exhibit a unique solution S(x) (even when L is convex). For example, it was noticed in [ 12 ] that the stochastic gradient descent (SGD) method can find flat minima (solutions located in the flat valley of the loss landscape); others found that all SGD solutions for DNN may form a manifold [ 2 , 4 ] . When an application (e.g., a data poisoning attack) is designed and evaluated based on only one of the solutions, it overlooks the fact that the learning algorithm may converge to other solutions during re-training. In this paper, we incorporate the fact that S is often set-valued into the sensitivity analysis framework for DNNs. This extends the scope of sensitivity analysis in DNNs from focusing on a single solution to a solution set, shifting from the traditional ’point-to-point’ analysis to a ’set-to-set’ paradigm. That is, we study how the solution set of a DNN expands and contracts in response to data perturbations. The proposed approach covers more general situations in risk minimization of DNN, including isolated local minima, non-isolated minima, and minima that consist of a connected manifold. More importantly, it directly deals with the solution sets without the assumption of non-singular Hessian matrix, offering a more complete understanding of DNN. Considering S as set-valued, with the isolated minima a special case where solution set S(x) contains only one element, the sensitivity analysis aims to study 1) whether the limit ( 2 ) exists, and 2) whether the limit ( 2 ) is bounded. However, directly studying the limit when mapping S is set-valued could be challenging. On the one hand, S(x) could be a large or even infinitely large set (e.g., a manifold), which makes it complicated to analyze. To address this, we instead focus on the change of the solution set S(x) within a small neighborhood. Given a pair (\bar{x},\bar{w}),\bar{w}\in S(\bar{x}) , we study the change of S(x)\cap U , where U is a neighborhood of \bar{w} , in response to the data perturbation within a neighborhood V of \bar{x} . On the other hand, as S(x) and S(\bar{x}) are sets rather than single solutions, the subtraction operation between sets does not exist (set operations only include union, intersection, and set difference), which invalidates the limit ( 2 ). Consequently, we leverage the Lipchitz-like property of S around (\bar{x},\bar{w}) , to measure the change of S(x)\cap U . We say that S holds Lipchitz-like property around \left(\bar{x},\bar{w}\right)\in\operatorname{gph}S:=\{(x,w)\mid w\in S(x)\} , if there exist neighborhoods U of \bar{w},V of \bar{x} and a positive real number \kappa such that [ 6 ] 2 2 2 Here, \left\|x-{x^{\prime}}\right\| represents the square root of the sum of the squares of the differences between corresponding elements, i.e., \left\|x-{x^{\prime}}\right\|=\sqrt{\sum_{i=1}^{n}\|x_{i}-{x^{\prime}}_{i}\|^{2}} . S(x^{\prime})\cap U\subset S(x)+\kappa\left\|x-x^{\prime}\right\|\mathbb{B},\hskip 10.00002pt\forall x^{\prime},x\in V, (4) where \mathbb{B} is a closed unit ball. The Lipschitz-like property is an extension of

Read the complete text