Distribution and Clusters Approximations as Abstract Domains in Probabilistic Abstract Interpretation to Neural Network Analysis

1 st Zhuofan Zhang, 2 nd Herbert Wiklicky · 2026

Read Distribution and Clusters Approximations as Abstract Domains in Probabilistic Abstract Interpretation to Neural Network Analysis by 1 st Zhuofan Zhang, 2 nd Herbert Wiklicky (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25273.txt.

Introduction

Zhang and Wiklicky add two new abstract domains to their probabilistic reading of neural networks. The framework tracks how input density flows through a net, and its grid domain chops space into boxes. One new domain fits smooth distributions to that flow. The other groups inputs into clusters. The pair explains how each domain works in theory. They give the matching abstract transformers and walk through small examples with pictures. The options widen the trade-offs between precision and cost in network analysis.

Catalog record

CREATOR1 st Zhuofan Zhang, 2 nd Herbert Wiklicky
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29379
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.25273

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Distribution and Clusters Approximations as Abstract Domains in Probabilistic Abstract Interpretation to Neural Network Analysis Thanks: 1 st Zhuofan Zhang, 2 nd Herbert Wiklicky Abstract The probabilistic abstract interpretation framework of neural network analysis analyzes a neural network by analyzing its density distribution flow of all possible inputs. The grids approximation is one of abstract domains the framework uses which abstracts concrete space into grids. In this paper, we introduce two novel approximation methods: distribution approximation and clusters approximation. We show how these two methods work in theory with corresponding abstract transformers with help of illustrations of some simple examples. I Introduction Different methods in the field of neural network verification and analysis are desired to help humans better understand how neural networks make their decisions and to ensure certain properties of neural networks, such as safety or robustness. One branch of this research has focused on using techniques of abstract interpretation theory. In 2018 Gehr et al. proposed an analyzer AI^{2} that uses abstract interpretation to over-approximate a range of concrete values into an abstract domain and checks whether this abstract domain holds the expected result and which will prove every single input over-approximated by this abstract domain also holds this result [ 1 ] . This analyzer makes use of the zonotope abstract domain which was originally proposed by Ghorbal et al. [ 2 ] . Later, Singh et al. [ 3 ] proposed a new abstract domain which combines floating point polyhedra with intervals, instead of the former zonotopes abstract domain. Its performance was compared with Reluplex, which is a former property verification technique that extends the simplex algorithm to support ReLU constraints [ 4 ] . Singh et al. also presented DeepZ based on abstract interpretation that handles ReLU, Tanh and Sigmoid activation functions and supports feedforward, convolutional, and residual architectures [ 5 ] . Abstract interpretation is also bridged with gradient-based optimization and can be applied to training neural networks [ 6 ] . Ryou et al. presented Prover, a scalable and precise verifier for recurrent neural networks using polyhedral abstractions [ 7 ] . Beyond classic verifiers, the question of whether certain neural network properties hold in probabilistic conditions is also considered. Techniques of the theory of probabilistic abstract interpretation have been then brought into the research. Probabilistic abstract interpretation is a framework that allows to systematically lift any classical analysis or verification method to the probabilistic setting by separating in the program semantics the probabilistic behavior from the (non-)deterministic behavior [ 8 ] . Di Pierro and Wiklicky also developed the framework of probabilistic abstract interpretation by using Concurrent Constraint Programming as a reference programming paradigm in probabilistic program analysis [ 9 ] , and by investigating relations between the operational semantics of probabilistic programming languages and Discrete Time Markov Chains (DTMCs) to introduce Linear Operator semantics(LOS) [ 10 ] , and used the framework to compare analysis on the basis of their expected exactness for a given program with a quantitative notion of precision [ 11 ] , and also extended the framework from finite state spaces to infinite state spaces [ 12 ] . In 2020 Steffen et al. introduced NetDice which is a scalable and accurate probabilistic network configuration analyzer that is based on an inference algorithm to efficiently explore the space of failure scenarios [ 13 ] . Mangal et al. verifies probabilistic robustness of neural networks with respect to the input distribution [ 14 ] . Pasareanu et al. investigated the use of symbolic analysis and constraint solution space quantification to precisely quantify probabilistic properties in neural networks [ 15 ] . In 2024 Zhang and Wiklicky proposed a method of grids approximation to analyze neural networks with techniques from probabilistic abstract interpretation [ 16 ] . In this paper, we introduce two novel approximation methods: distribution approximation and clusters approximation based on the probabilistic abstract interpretation framework with respect to neural network analysis. Different approximation approaches may be chosen with respect to different program circumstances and different properties concerned. Our approach aims to analyze the probability density distribution flow of all possible inputs of a trained neural network. This can be used to make sure that the system is robust and under controlled in context of probability, by showing that a majority of data flows through propagation of neural networks are bounded in desired domain. In addition, it can also offer a sensitivity analysis of the data from the input to each layer, helping to better understand how the data flow inside a neural network. We show how these two novel approximation methods work in theory and illustrate the corresponding abstract transformers used in the framework with help of some examples. II Abstract Domains II-A Distribution Approximation as Abstract Domain The first new approximation method we present is the distribution approximation. The distribution approximation approximates the concrete space in its distribution that includes all concrete data plus the corresponding probabilities. That is, if a concrete data point is in dimension n , then the approximated model is n+1 dimensional, the additional dimension being the probabilities of the concrete data. An abstract transformer in this case maps a distribution following propagations of neural network layers. In other words, we consider a distribution that propagates through the neural network in the abstract domain. In distribution approximation, different choices of function approximators can be selected with respect to problem circumstance. Polynomials, radial basis functions, or Fourier transform can be one of typical choices. II-A 1 Polynomial Regression Polynomials can be one of the simplest approximators in such a regression problem. A general polynomial model is in the form of \textbf{y}=\beta_{0}+\beta_{1}\textbf{x}+\beta_{2}\textbf{x}^{2}+...+\beta_{m}\textbf{x}^{m}+\epsilon (1) Or it can be written as a system of linear equations: \begin{bmatrix}y_{1}\\ y_{2}\\ \vdots\\ y_{n}\end{bmatrix}=\begin{bmatrix}1&x_{1}&x_{1}^{2}&...&x_{1}^{m}\\ 1&x_{2}&x_{2}^{2}&...&x_{2}^{m}\\ \vdots&\vdots&\vdots&\ddots&\vdots\\ 1&x_{n}&x_{n}^{2}&...&x_{n}^{m}\\ \end{bmatrix}\begin{bmatrix}\beta_{0}\\ \beta_{1}\\ \beta_{2}\\ \vdots\\ \beta_{m}\end{bmatrix}+\begin{bmatrix}\epsilon_{1}\\ \epsilon_{2}\\ \vdots\\ \epsilon_{n}\end{bmatrix} (2) which can also be written in matrix notation of \overrightarrow{y}=\textbf{X}\overrightarrow{\beta}+\overrightarrow{\epsilon} (3) and we can find the least square solution \hat{\overrightarrow{\beta}}=(\textbf{X}^{\intercal}\textbf{X})^{-1}\textbf{X}^{\intercal}\overrightarrow{y} (4) We can now have the polynomials that approximate the concrete space. By definition of probabilistic abstract interpretation, f in concrete space is an affine function representing the forward propagation of the neural network layer, therefore we can have the abstract transformer f^{\#} in abstract domain mapping a distribution to a distribution such that f^{\#}(\textbf{y}(\textbf{x}))=\textbf{y}(f^{-1}(\textbf{x})) (5) II-A 2 Radial Basis Functions Radial basis functions(RBFs) are real-valued functions whose value depends only on the distance from a central point, typically expressed as \phi(\textbf{x})=\phi(\|\textbf{x}-\textbf{c}\|) (6) where x is the input vector, c is the center, and \|\cdot\| denotes a norm(often Euclidean). Common RBFs include the Gaussian function \phi(\textbf{x},\textbf{c})=e^{-\frac{\|\textbf{x}-\textbf{c}\|^{2}}{2\sigma^{2}}} where \sigma is a parameter that controls the width of the basis function, multiquadric function \phi(\textbf{x},\textbf{c})=\sqrt{\|\textbf{x}-\textbf{c}\|^{2}+\sigma^{2}} , inverse multiquadric function \phi(\textbf{x},\textbf{c})=\frac{1}{\sqrt{\|\textbf{x}-\textbf{c}\|^{2}+\sigma^{2}}} , or thin plate spline function \phi(\textbf{x},\textbf{c})=\|\textbf{x}-\textbf{c}\|^{2}\ln(\|\textbf{x}-\textbf{c}\|) . More specifically, radial basis functions approximate the concrete space in a distribution with N centers. That is, if let original concrete space be x , and let the dimension of probabilities be y , the approximation can be represented in the form of \textbf{y}=\sum_{i=1}^{N}a_{i}\phi(\|\textbf{x}-\textbf{c}\|) (7) where a_{i} are the interpolation coefficients of the radial basis functions. Note that a_{i} can be computed once given a particular choice of N centers. To find the optimal choice of centers,

Read the complete text