Statistics of Min-max Normalized Eigenvalues in Random Matrices

Hyakka Nakada 1 and Shu Tanaka 1,2,3,4,5 · 2025

Read Statistics of Min-max Normalized Eigenvalues in Random Matrices by Hyakka Nakada 1 and Shu Tanaka 1,2,3,4,5 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-15427.txt.

Introduction

Hyakka Nakada and Shu Tanaka study random matrices in a short 2025 paper on how their eigenvalues spread out after a common scaling step. In data work, values are often rescaled so the smallest becomes zero and the largest becomes one. The authors ask what pattern the scaled eigenvalues follow in that case.

They apply a known formula for the scaled spread to derive a scaling law for the cumulative distribution. They also derive the leftover error that appears when a random matrix is split into simpler factors. Computer tests check both results. The work helps link a routine data step to the math behind it.

Catalog record

CREATORHyakka Nakada 1 and Shu Tanaka 1,2,3,4,5
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-15427

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Statistics of Min-max Normalized Eigenvalues in Random Matrices Hyakka Nakada 1 and Shu Tanaka 1,2,3,4,5 Abstract Random matrix theory has played an important role in various areas of pure mathematics, mathematical physics, and machine learning. From a practical perspective of data science, input data are usually normalized prior to processing. Thus, this study investigates the statistical properties of min-max normalized eigenvalues in random matrices. Previously, the effective distribution for such normalized eigenvalues has been proposed. In this study, we apply it to evaluate a scaling law of the cumulative distribution. Furthermore, we derive the residual error that arises during matrix factorization of random matrices. We conducted numerical experiments to verify these theoretical predictions. Practical data analysis pipelines often employ preprocessing steps. Among these techniques, feature scaling, which scales all features to a fixed range [0,1] , is widely adopted for purposes such as mitigating the influence of extremely large values and stabilizing machine learning models. Furthermore, this scaling facilitates interpretation as rates or probabilities. For example, in principal component analysis (PCA), the proportion of variance explained by each principal component is crucial for measuring its contribution, which corresponds to the relative magnitude of each eigenvalue [ 1 ] . Thus, normalized quantities or ratios can sometimes be better statistical measures than absolute values. Representative examples of such scaling methods include min-max normalization and the softmax activation function. Random matrix theory has played an important role in various areas of pure [ 2 , 3 ] and applied mathematics [ 4 , 5 , 6 , 7 ] . For example, in physics, the interaction coefficients in spin glass models are typically described by random matrices [ 8 ] . In computer science, because real-world signals contain a significant amount of noise, random matrices are sometimes used to model the data matrix [ 9 ] . A key advantage is that one can predict the asymptotic spectral properties even without knowing the detailed values of the matrix elements. In recent years, the size of data matrices has grown enormously, leading to active study on random matrix theory [ 10 , 11 , 12 ] . This paper investigates the theoretical properties of the distribution of min-max normalized eigenvalues in random matrices, \displaystyle\hat{\lambda}=\frac{\lambda-\lambda_{N}}{\lambda_{1}-\lambda_{N}}. (1) Here, \lambda_{1} and \lambda_{N} are the largest and smallest eigenvalues of a random matrix Q , which follows Gaussian distribution, \displaystyle Q_{ij}\overset{\textrm{iid}}{\sim}\mathcal{N}\left(\mu,\sigma^{2}\right) (2) where i<j . The mean of the coupling coefficient is denoted by \mu , the standard deviation by \sigma^{2} , and the input dimension is N . In addition, we assume that diagonal Q_{ii} follows N(\mu,2\sigma^{2}) . Thus, the matrix Q is an orthogonal random matrix with a finite mean. Hereinafter, \mu>0 is considered. In the previous study [ 13 ] , we proposed the cumulative distribution of the min-max normalized eigenvalues ( 1 ), during the process of analyzing Factorization Machines (FMs) [ 14 , 15 ] . FMs are a supervised learning model that can estimate pairwise feature interactions by factorizing the interaction matrix into low-rank latent vectors, making them effective for tasks like recommendation in sparse datasets. Recently, FMs have also been applied to black-box optimization [ 16 , 17 , 18 , 19 , 20 , 21 , 22 ] . Because these models are considered as Quadratic Unconstrained Binary Optimization (QUBO) forms, it has also garnered significant attention as an application of quantum annealing [ 23 , 24 , 25 , 26 ] . By approximating the eigenvalues by their expectations as follows, \displaystyle\lambda_{1}\approx E\left[\lambda_{1}\right]=N\mu+\sigma^{2}/\mu \displaystyle\lambda_{2}\approx E\left[\lambda_{2}\right]=2\sqrt{N}\sigma (3) \displaystyle\lambda_{N}\approx E\left[\lambda_{N}\right]=-2\sqrt{N}\sigma, we obtained the cumulative distribution [ 13 ] : For \sigma\leq\sqrt{N}\mu , \displaystyle P\left(\hat{\lambda}<x\right)=\begin{cases}1+\frac{x-1}{(1-r)N}&\text{for $x>r$,}\\ \frac{1}{N}+\frac{4(N-2)}{\pi Nr^{2}}\displaystyle\int_{0}^{x}\sqrt{r^{2}-(2t-r)^{2}}dt&\text{for $x\leq r$}.\end{cases} (4) For \sigma>\sqrt{N}\mu , \displaystyle P\left(\hat{\lambda}<x\right)=\frac{1}{N}+\frac{4(N-1)}{\pi N}\displaystyle\int_{0}^{x}\sqrt{1-(2t-1)^{2}}dt. (5) In Eq. ( Statistics of Min-max Normalized Eigenvalues in Random Matrices ), the expectation values are derived from Wigner’s semicircle law [ 27 , 28 ] and the theory of the largest eigenvalue distribution theory [ 29 , 30 ] . r denotes the deterministic value for the normalized second-largest eigenvalue \hat{\lambda}_{2} , \displaystyle r\equiv\frac{4\sqrt{N}\mu\sigma}{(\sqrt{N}\mu+\sigma)^{2}}. (6) In this study, starting from the approximate cumulative-distribution ( 4 ), ( 5 ) and ( 6 ), several statistics are derived and discussed. Firstly, we explain the nature of scaling laws. The behavior of normalized eigenvalues essentially depends solely on the ratio of standard deviation to mean, as detailed later. Secondly, the residual error is analytically derived when Q is decomposed by matrix factorization [ 31 ] . Matrix factorization has a wide variety of applications, such as dimensionality reduction [ 32 ] , recommender system [ 33 ] , and model compression [ 34 ] . This is also the foundation of FMs [ 15 ] . We perform experiments on the consistency between the predicted error and the empirical error, to verify our theory. Firstly, we discuss the scaling laws. According to the basic treatment used in spin glass [ 35 ] , the mean and standard deviation are rewritten as \displaystyle\mu=\frac{J_{0}}{N},\sigma=\frac{J_{1}}{\sqrt{N}}. (7) Here, the values of J_{0} and J_{1} are positive. Equation ( 6 ) is transformed to \displaystyle r=\frac{4J_{1}/J_{0}}{(1+J_{1}/J_{0})^{2}}. (8) Thus, r depends only on the value of J_{1}/J_{0} . Equation ( 8 ) has the value of 1 if and only if J_{0}=J_{1} . When the input dimension N is sufficiently large, Eqs. ( 4 ) and ( 5 ) converge to the following distributions. For J_{1}\leq J_{0} , \displaystyle P\left(\hat{\lambda}<x\right)=\begin{cases}1&\text{for $x>r$,}\\ \frac{4}{\pi r^{2}}\displaystyle\int_{0}^{x}\sqrt{r^{2}-(2t-r)^{2}}dt&\text{for $x\leq r$}.\end{cases} (9) For J_{1}>J_{0} , \displaystyle P\left(\hat{\lambda}<x\right)=\frac{4}{\pi}\displaystyle\int_{0}^{x}\sqrt{1-(2t-1)^{2}}dt. (10) Based on Eqs. ( 8 ), ( 9 ) and ( 10 ), the cumulative distribution of min-max normalized eigenvalues is expected to depend solely on the ratio of J_{1} to J_{0} . This is a property not observed in the distribution of unnormalized eigenvalues. Figure 1: (Color online) Plots of the cumulative distribution of min-max normalized eigenvalues with varying J_{1}/J_{0} . The black lines show our theoretical line computed from Eq. ( 4 ) or ( 5 ), for J_{1}/J_{0}=0.1,0.3,10.0 respectively. The green, blue, and red lines are experimental plots for J_{0}=0.1,1.0,10.0 , respectively. These lines were obtained with N=100 . Figure 2: (Color online) Plots of the cumulative distribution of min-max normalized eigenvalues with varing N . The yellow, green, blue, and red thick lines show our theoretical lines computed from Eq. ( 4 ), for N=10,25,100,500 respectively. The thin lines represent the experimental results. These lines were obtained with J_{0}=1 and J_{1}=0.3 . To verify the above scaling law, we performed the following numerical experiment. After sampling random matrices Q , eigenvalues were numerically calculated by eigenvalue decomposition. Then, the cumulative distribution was experimentally estimated. For J_{0}=0.1,1.0,10.0 , the values of J_{1} were designed to keep J_{1}/J_{0}=0.1 . The plots of the cumulative distribution are shown in Fig. 1 . Here, green, blue, and red lines are the distributions for J_{0}=0.1,1.0,10.0 , respectively. The theoretical line ( 4 ) is indicated by a black line. Thus, the experimental result agrees with our theory well, regardless of the value of J_{0} . These calculations were performed also for J_{1}/J_{0}=0.3,10.0 . Specifically for J_{1}/J_{0}=10.0 , we adopted Eq. ( 5 ) as the theoretical line because the condition of J_{1}>J_{0} holds. In all cases, the cumulative distributions

Read the complete text