Interval Spacing

Greg Kreider · 2026

Read Interval Spacing by Greg Kreider (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22116.txt.

Introduction

Greg Kreider defines a new summary for spread-out data. Take sorted data points and look across a gap of some width. The interval spacing is the difference between the order statistics at the gap's two ends. Kreider derives its probability density, its expected value, and its variance.

He works out those results for uniform, exponential, and logistic random variables. He also shows interval spacing is equivalent to running a rectangular low-pass filter over the ordinary spacings, which simplifies the formulas greatly. His 2026 preprint gives statisticians a workable way to measure data density. It shows how data thin or thicken across intervals. That helps wherever the local density of points carries the signal.

Catalog record

CREATORGreg Kreider
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE13 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-22116

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Interval Spacing Greg Kreider Abstract We define interval spacing as the difference in the order statistics of data over a gap of some width. We derive its density, expected value, and variance for uniform, exponential, and logistic variates. We show that interval spacing is equivalent to running a rectangular low-pass filter over the spacing, which simplifies the expressions for the expected values and introduces correlations between overlapping intervals. Keywords: interval spacing ; logistic variate AMS Subject Classification: 62G30 The theory behind spacing, the difference between consecutive order statistics, is well-developed by now. Occasionally one sees the difference over larger gaps being used. [ 7 ] estimates the location of the mode as the midpoint of the interval with the most data points, where the size of the interval depends on the polynomial order of the data around the mode. [ 1 ] collects a test statistic over a range of gaps to determine if data has a non-zero slope. [ 2 ] and [ 4 ] study such test statistics in general for non-overlapping segments. Let us call “interval spacing” the difference in order statistics over distances greater than one. Extending the notation used in [ 5 ] with i the upper index and w the width of the interval such that w<i\leq n , the interval spacing D_{i,w} is D_{i,w}=T_{i}-T_{i-w} (1) where T_{i} is the i^{\rm th} order statistic. Alternative notations that have been used are a center point j=i-(w/2) and radius r_{n}=w/2 [ 7 , (2.1)] , or start index j=i-w+1 and endpoint k=i of the range [ 1 ] . In this notation the spacing D_{i}=D_{i,1} . The density of the interval spacing \displaystyle f_{D_{i,w}}(y) \displaystyle=\begin{aligned} S_{1}\int_{-\infty}^{\infty}&\left\{F_{x}(x)\right\}^{i-w-1}\left\{F_{x}(x+y)-F_{x}(x)\right\}^{w-1}\left\{1-F_{x}(x+y)\right\}^{n-i}\\ &\times f_{x}(x)~f_{x}(x+y)~dx\end{aligned} (2) \displaystyle S_{1} \displaystyle=\frac{n!}{(i-w-1)!~(w-1)!~(n-i)!} follows from the joint density of two order statistics [ 8 , (8) and (31) with r=i-w and r^{\prime}=i ] , where F_{x}(x) and f_{x}(x) are distribution and density functions. Expanding [ 5 , (2.4)] gives the same density, using k_{1}=i-w , t_{1}=x , k_{2}=i , t_{2}=x+y , k_{3}=n+1 , t_{3}=\infty , and t_{0}=-\infty . ( 2 ) simplifies to the spacing’s density function [ 5 , (2.7)] when w=1 ; notably, the factor of the distribution function raised to w-1 disappears. The expected value and variance follow normally from the first two moments of this density. \displaystyle E\Bigl\{D_{i,w}\Bigr\} \displaystyle=\int_{0}^{\infty}y~f_{D_{i,w}}(y)~dy (3) \displaystyle V\Bigl\{D_{i,w}\Bigr\} \displaystyle=E\Bigl\{D_{i,w}^{2}\Bigr\}-E\Bigl\{D_{i,w}\Bigr\}^{2}=\int_{0}^{\infty}y^{2}~f_{D_{i,w}}(y)~dy-E\Bigl\{D_{i,w}\Bigr\}^{2} (4) For uniform variates over the range a,b we have \displaystyle f_{D_{i,w,unif}}(y) \displaystyle=\frac{n!}{(w-1)!(n-w)!}~\left(\frac{1}{b-a}\right)^{n}y^{w-1}~(b-y-a)^{n-w} (5) \displaystyle E\Bigl\{D_{i,w,unif}\Bigr\} \displaystyle=w~\frac{b-a}{n+1} (6) \displaystyle V\Bigl\{D_{i,w,unif}\Bigr\} \displaystyle=w~\frac{(n+1-w)}{n+2}\left(\frac{b-a}{n+1}\right)^{2} (7) For exponential variates with rate parameter \lambda these are \displaystyle f_{D_{i,w,exp}}(y) \displaystyle=w~\binom{n-i+w}{n-i}~\lambda~\left\{e^{-\lambda y}\right\}^{n-i+1}\left\{1-e^{-\lambda y}\right\}^{w-1} (8) \displaystyle E\Bigl\{D_{i,w,exp}\Bigr\} \displaystyle=w~\binom{n-i+w}{n-i}~\frac{1}{\lambda}~(-1)^{w-1}\sum_{k=0}^{w-1}\binom{w-1}{k}\frac{(-1)^{k}}{(n-i+w-k)^{2}} (9) \displaystyle E\Bigl\{D_{i,w,exp}^{2}\Bigr\} \displaystyle=w~\binom{n-i+w}{n-i}~\frac{2}{\lambda^{2}}~\sum_{k=0}^{w-1}\binom{w-1}{k}\frac{(-1)^{k}}{(n-i+w-k)^{3}} (10) Use ( 4 ) and the last two equations to calculate the variance; squaring ( 9 ) and subtracting from ( 10 ) does not lead to a simpler equation. The pre-factor can also be written (n-i+w)!/(w-1)!(n-i)! , which will cancel a factor (w-1)! from the combinatorial inside the series. For logistic variates with mean \mu and standard deviation \sigma we find \displaystyle f_{D_{i,w,logis}}(y) \displaystyle=\frac{1}{\sigma}~S_{2}~e^{y/\sigma}\left\{e^{y/\sigma}-1\right\}^{w-1} \displaystyle\qquad\times\begin{aligned} {}_{2}F_{1}\left(i,n-i+w+1;n+w+1;1-e^{y/\sigma}\right)\end{aligned} (11) \displaystyle S_{2} \displaystyle=\frac{n!}{(n+w)!}~\frac{(n-i+w)!}{(n-i)!}~\frac{(i-1)!}{(i-w-1)!}~\frac{1}{(w-1)!} \displaystyle=w\prod_{j=1}^{w}\frac{(i-j)(n-i+j)}{j(n+j)} \displaystyle E\Bigl\{D_{i,w,logis}\Bigr\} \displaystyle=\sum_{k=0}^{w-1}\frac{n!}{(i-w-1)!(w-1-k)!(n-i+1+k)!}~\sigma~(-1)^{w-1-k} \displaystyle\qquad\times\left[\begin{aligned} \frac{1}{i-k-1}\left[\psi(i-k)-\psi(i-k-1)\right]\\ -\sum_{l=1}^{n-i+w-1}\frac{1}{n-i+w-l}~B(n-i+w+1-l,i-k-1)\\ -~\frac{w-1-k}{n-i+w-1}~B(n-i+w+1,i-k-2)\\ +\sum_{l=2}^{w-1-k}(-1)^{l}\sum_{j=0}^{l-1}S_{3}~B(n-i+w+1+j,i-k-2-j)\end{aligned}\right] (12) \displaystyle S_{3} \displaystyle=\frac{(w-1-k)!}{(w-1-k-l)!}~\frac{(n-i+w-l-1)!}{(n-i+w-l+j)!}~\frac{1}{l(l-1-j)!} The second moment would require an additional integration by parts of these terms, which we do not perform. B(a,b)=(a-1)!(b-1)!/(a+b-1)! is the beta function and \psi(x) the dilogarithm. The interval spacing introduces an extra factor F^{w-1} of the distribution function which requires an integration by parts to handle. This leads to more complicated expressions for the expected interval spacing than for the normal spacing, although all results reduce to the spacing versions found in [ 3 ] if w=1 . The uniform and exponential equations follow from known definite integrals (see Supplemental Materials for derivations). Rather than integrating the hypergeometric function in the logistic density function ( Interval Spacing ) to get the first moment, it is better to integrate first over y after combining ( 2 ) and ( 3 ), then over x . This must be done by parts, which involves a recursion down n that gives a series that must then be integrated term by term. High-precision math libraries must be used to evaluate the series. The factorial scaling factors in ( 5 ) – ( Interval Spacing ) reach n^{w}/w! while the spacing is of order one, so the series sum of terms of alternating sign of this size must nearly cancel. Figure 1 plots the density of the interval spacing for draws of n=50 data points. The function for uniform variates in the left graph is independent of the index i , but the others show two sets of curves. The densities with high, narrow peaks correspond to small values from the variate, found at the start of the exponential’s order statistics, drawn with i=w+2 , or the center of the logistic, drawn with i=n/2 . The lower, broader densities come from the tails of the distribution where the order statistics are largest. This occurs near the final indices of the expontial, drawn for i=n-2 , and the initial indices of the logistic, drawn for i=w+2 but also valid by symmetry at the other tail. All densities skew to the right, although this is much more noticeable in the broader curves. Figure 1: Density of the interval spacing at widths w of 2, 5, and 10 and indices i as noted. Figure 2 plots the expected interval spacing for exponential and logistic variates at three widths; the data size n=50 has been kept small for visibility. The grey bands are inter-quartile ranges based on ten thousand simulated draws. The expected values are not centered, lying closer to the upper quartile. This reflects the skewing of the density. Such shifting also happens for the medians, not plotted. Figure 2: Expected interval spacing for exponential and logistic variates at widths w of 2, 5, and 10. Bands are inter-quartile ranges from simulations. An interval can be broken into non-overlapping segments whose sub-spacing will add. For example, if w is even we can split the interval in half, \displaystyle D_{i,w/2}+D_{i-w/2,w/2} \displaystyle=T_{i}-T_{i-w/2}+T_{i-w/2}-T_{i-w}=T_{i}-T_{i-w} \displaystyle=D_{i,w} (13) The half intervals sum to the whole. Any sub-intervals need not have the same size, which would happen in this example if w were odd, but the endpoints must match and cover the whole range. Splitting into more pieces is also possible, a process which ultimately ends by breaking the interval into w single steps. The interval spacing is the sum of these spacings, D_{i,w}=\sum_{j=0}^{w-1}D_{i-j} (14) A sum over consecutive data points is just a rectangular low-pass filter. If the total were divided by w this would be a running mean. Because such normalization is not done — the filter kernel is a vector of w 1’s rather than 1/w ’s — larger intervals amplify differences. This must be taken into account when comparing different widths. The spacing is not constant over the interval so the interval spacing is not just the spacing scaled by w , although it is close, especially where the variate’s density is nearly constant, at the start of one-sided distributions or the center of two-sided. We can demonstrate that filtering occurs by taking the ratio of the Fourier transforms of D_{i,w} and D_{i} and comparing it to the impulse response \mathcal{R}

Read the complete text