Separating balls with partly random hyperplanes with a view to partly random neural networks

Olov Schavemaker, Utrecht University, the Netherlands · 2025

Read Separating balls with partly random hyperplanes with a view to partly random neural networks by Olov Schavemaker, Utrecht University, the Netherlands (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-10470.txt.

Introduction

Olov Schavemaker wrote a 2025 math paper on split planes that are only partly random. The paper asks how likely such a plane is to split two round balls in space. A fully random plane turns out to split the balls less often than some partly random ones, in set cases. The exact formulas give a firm base for that claim, and the author links the result to neural networks built with partly random parts, as a first step for work on them.

Catalog record

CREATOROlov Schavemaker, Utrecht University, the Netherlands
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-10470

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Separating balls with partly random hyperplanes with a view to partly random neural networks Olov Schavemaker, Utrecht University, the Netherlands Abstract We derive exact expressions for the probabilities that partly random hyperplanes separate two Euclidean balls. The probability that a fully random hyperplane separates two balls turns out to be significantly smaller than the corresponding probabilities for hyperplanes which are not fully random in certain cases. Our results motivate studying partially random neural networks and provide a first step in this direction. mathx"20 mathx"28 I Introduction In this short paper we deduce exact expressions for the probabilities that three types of partly random hyperplanes separate two Euclidean balls. Letting \displaystyle\mathsf{H}[w;b] be the zero locus of \displaystyle u\mapsto(w\mid u)-b, we consider the following three types of partly random hyperplanes: \displaystyle\mathsf{H}[w;{b}],\mathsf{H}[{w};b], and \displaystyle\mathsf{H}[{w};{b}], where \displaystyle w,b are deterministic and \displaystyle{w},{b} are random variates. The hyperplane parametrization \displaystyle\mathsf{H}[w;b] is redolent of neural networks by design; a “neuron” in a neural network consists of a nonlinear activation function being applied to a linear function of the form \displaystyle u\mapsto(w\mid u)-b. This paper was inspired by [ 2 ] , where the authors showed that random neural networks can make two fairly arbitrary subsets of Euclidean space linearly separable by covering those two sets with small enough balls. In [ 2 ] they consider an architecture with two layers. Their result relies on letting the first layer be wide enough for it to separate all the balls covering one set from all of the balls covering the other set with high probability. This boils down to drawing enough fully random hyperplanes to separate all the balls covering one set from all the balls covering the other set with high probability. Clearly the probability one random hyperplane separates one pair of balls is of interest if one wants to get insight into how many random hyperplanes are needed to separate all pairs of balls with high probability. Since sufficiently nice manifolds are known to admit nice covering number bounds [ 6 , Thm 4.2] , [ 7 , Lemma B.1] , [ 8 , §5] the result in [ 2 ] and possible improvements thereof could be of great interest to those interested in classifying manifolds; a commonly assumed situation. Random neural networks might sound void of practical significance at first as one generally wants to learn the weights and biases. Well, learning the weights and biases can be slow, sensitive to hyperparameters, and final values may be suboptimal [ 3 ] . One way to avoid these undesirabilities is to forgo learning some of the weights or/and biases; a popular architecture with this design philosophy is the so-called Random Vector Functional Link (RVFL) network [ 3 ] . In the case of RVFL networks, it has been known for quite a while that they are performant despite the random weights and biases [ 4 , 5 ] . An RVFL consists of two layers, the first one being fully random and the second one being learned. As far as the author knows, partially random layers have not yet been studied despite having the potential to be a best of both worlds golden mean. In § VI we shall discuss how our theorems hint at this indeed being the case. In § II we clarify the setting wherein we operate throughout the paper, including notation and assumptions. §§ III – V are each dedicated to a probability that a partly random hyper- plane separates two balls; \displaystyle\mathsf{H}[w;{b}],\mathsf{H}[{w};b], and \displaystyle\mathsf{H}[{w};{b}] in that order. Lastly, we discuss our results and their possible implications in § VI . II Setting Since we are interested in separating balls in \displaystyle\mathds{R}^{n} with hyperplanes, we introduce the following notation. The Euclidean inner product and norm on \displaystyle\mathds{R}^{n} will be denoted by \displaystyle(\cdot\!\mid\!\cdot) and \displaystyle\lvert\mkern 2.0mu\cdot\mkern 2.0mu\rvert resp. Let \displaystyle\mathsf{B}[c,r]=\{u\in\mathds{R}^{n}:\lvert c-u\rvert<r\}, where \displaystyle c\in\mathds{R}^{n} and \displaystyle r>0. Let \displaystyle\mathsf{H}[w;b] be the zero locus of \displaystyle u\mapsto(w\mid u)-b, where \displaystyle b\in\mathds{R} and \displaystyle w\in\mathbb{S}^{n-1}=\{u\in\mathds{R}^{n}:\lvert u\rvert=1\}. The notations \displaystyle\mathsf{B}[c,r] and \displaystyle\mathsf{H}[w;b] parametrize all balls and hyperplanes resp., albeit not quite uniquely, as \displaystyle\mathsf{H}[w;b]= \displaystyle\mathsf{H}[-w;-b]. Definition . We say \displaystyle\mathsf{H}[w;b] separates separates separates separates separates separates separates separates separates separates separates separates separates separates separates separates separates separates \displaystyle\mathsf{B}[c,r] and \displaystyle\mathsf{B}[x,p] if \displaystyle(w\mid u)-b\gtrless 0 for all \displaystyle u\in\mathsf{B}[c,r] whereas \displaystyle(w\mid u)-b\lessgtr 0 for all \displaystyle u\in\mathsf{B}[x,p]. The balls \displaystyle\mathsf{B}[c,r] and \displaystyle\mathsf{B}[x,p] clearly cannot be separated if \displaystyle\mathsf{B}[c,r]\cap\mathsf{B}[x,p]\neq\varnothing. Since we are interested in partially random hyperplanes, we assume \displaystyle\lvert c-x\rvert=p+r+\delta for some \displaystyle\delta>0. If \displaystyle\delta=0, only one hyperplane separates the balls \displaystyle\mathsf{B}[c,r] and \displaystyle\mathsf{B}[x,p]; namely, \displaystyle\{c-x\}^{\perp}+v, where \displaystyle\{v\}=\partial\mathsf{B}[c,r]\cap\partial\mathsf{B}[x,p]. When considering a hyperplane \displaystyle\mathsf{H}[w;b], we shall refer to \displaystyle w,b as the weight weight weight weight weight weight weight weight weight weight weight weight weight weight weight weight weight weight and bias bias bias bias bias bias bias bias bias bias bias bias bias bias bias bias bias bias resp., in keeping with neural networks. We account random weights uniformly on \displaystyle\mathbb{S}^{n-1} and random biases uniformly on \displaystyle[-k,k] with \displaystyle k>0. Henceforth we shall always consider \displaystyle\mathsf{B}[c,r] and \displaystyle\mathsf{B}[x,p] so that \displaystyle\lvert c-x\rvert=p+r+\delta for some \displaystyle\delta>0. We withal assume that \displaystyle n\geqslant 2 and that \displaystyle{w},{b} are uniformly distributed on \displaystyle\mathbb{S}^{n-1} and \displaystyle[-k,k] resp., where \displaystyle k\geqslant\lvert c\rvert\vee\lvert x\rvert>0~(\Leftarrow c\neq x). III Random biases The first and easiest probability we are interested in is \displaystyle\displaystyle\mathds{P}_{\!{b}}\{\exists w\in\mathbb{S}^{n-1}:\mathsf{H}[w;{b}]\text{ separates }\mathsf{B}[c,r]\text{ and }\mathsf{B}[x,p]\} To compact notation, we will shorten this to \displaystyle\displaystyle\mathds{P}_{\!{b}}\{\exists w:\mathsf{H}[w;{b}]\text{ separates}\} Before we evaluate this probability, however, we return to the aforementioned \displaystyle\mathsf{H}[w;b]=\mathsf{H}[-w;-b], which suggests that it may be more natural to consider \displaystyle w\in\mathbb{S}^{n-1}\!/\{\pm\}\cong \displaystyle\mathds{P}^{n-1} instead, where \displaystyle\mathds{P}^{n-1} comprises lines in \displaystyle\mathds{R}^{n} through the origin. Indeed, the orthogonal complement of a hyper- plane is a line through the origin. We therefore introduce a second parametrization of hyper- planes, namely \displaystyle\mathsf{H}[a,v]=\{a\}^{\perp}+v, where \displaystyle a\in\mathds{P}^{n-1} and \displaystyle v\in a is the unique point on the line \displaystyle a wherethrough the hyperplane passes. This parametrization is unique and we have \displaystyle\mathsf{H}[w;(w\mid v)]=\mathsf{H}[w\mathds{R},v] whenever \displaystyle w\in\mathbb{S}^{n-1} and \displaystyle v\in\mathds{R}w. Theorem III . Let \displaystyle{b} be as in § II . Then \displaystyle\displaystyle\mathds{P}_{\!{b}}\{\exists w:\mathsf{H}[w;{b}]\text{ separates}\}= \displaystyle\displaystyle\mathds{P}_{\!{b}}\{\mathsf{H}[\omega\mathds{R},\omega{b}]\text{ separates}\}=\delta/(2k) where \displaystyle\mathbb{S}^{n-1}\ni\omega is parallel to the line through \displaystyle c and \displaystyle x, which we will call \displaystyle a. Proof. Starting with the second equality, we can introduce coordinates onto \displaystyle a by considering \displaystyle\Omega:a\ni v\mapsto(\omega\mid v). Let the intervals \displaystyle(s,t) and \displaystyle(y,z) be the images of \displaystyle\mathsf{B}[c,r]\cap a & \displaystyle\mathsf{B}[x,p]\cap a under \displaystyle\Omega resp.; i.e., \displaystyle s is the coordinate of one of the endpoints of the line segment \displaystyle\mathsf{B}[c,r]\cap a, etc. WLOG assuming that \displaystyle t<y, we must have \displaystyle y-t=\delta. Now, \displaystyle\Omega(\omega{b})={b} so \displaystyle\displaystyle\mathds{P}_{\!{b}}\{\mathsf{H}[\omega\mathds{R},\omega{b}]\text{ separates}\}=\mathds{P}_{\!{b}}\{{b}\in[t,y]\}=\delta/(2k) if \displaystyle[t,y]\subseteq[-k,k]; indeed, since \displaystyle t,y\in(c,x), \displaystyle\displaystyle\lvert t\rvert\vee\lvert y\rvert\leqslant\lvert c\rvert\vee\lvert x\rvert\leqslant k As for the first equality, it is geometrically optimal to have \displaystyle\mathsf{H}[w;{b}]\perp a, which is achieved by letting \displaystyle w=\omega. Indeed, each separating hyperplane must intersect \displaystyle a with coordinate in the interval \displaystyle[t,y] and every coordinate is achieved by at most one realization of \displaystyle{b}. Only \displaystyle\mathsf{H}[\omega\mathds{R},\omega{b}] can achieve the coordinates \displaystyle t and \displaystyle y, and since we have already shown it in fact achieves the entire interval, \displaystyle w=\omega is optimal. Since \displaystyle(\omega\mid\omega{b})={b}\lvert\omega\rvert^{2}={b} the first equality follows. ∎ Henceforth \displaystyle a

Read the complete text