VoodooNet: Achieving Analytic Ground States via High-Dimensional Random Projections

Wladimir Silva · 2026

Read VoodooNet: Achieving Analytic Ground States via High-Dimensional Random Projections by Wladimir Silva (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15613.txt.

Introduction

Silva presents Green-ELM, a non-iterative neural design that swaps gradient-based output training for a closed-form analytic solve over fixed random features. Projecting inputs into a high-dimensional random feature space untangles complex class boundaries without backpropagation. With Moore-Penrose, LU, and Cholesky solves, it hits 98.10 percent on MNIST and 86.63 percent on Fashion-MNIST, training 11.6 times faster than an SGD baseline on CPU.

Catalog record

CREATORWladimir Silva
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26939
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.15613

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VoodooNet: Achieving Analytic Ground States via High-Dimensional Random Projections Wladimir Silva Abstract We present VoodooNet , a non-iterative neural architecture that replaces the stochastic gradient descent (SGD) paradigm with a closed-form analytic solution via Galactic Expansion. By projecting input manifolds into a high-dimensional, high-entropy ”Galactic” space ( d ≫ 784 d\gg 784 ), we demonstrate that complex features can be untangled without the thermodynamic cost of backpropagation. Utilizing the Moore-Penrose pseudoinverse to solve for the output layer in a single step, VoodooNet achieves a classification accuracy of 98.10% on MNIST and 86.63% on Fashion-MNIST . Notably, our results on Fashion-MNIST surpass a 10-epoch SGD baseline (84.41%) while reducing the training time by orders of magnitude. We observe a near-logarithmic scaling law between dimensionality and accuracy, suggesting that performance is a function of ”Galactic” volume rather than iterative refinement. This ”Magic Hat” approach offers a new frontier for real-time Edge AI, where the traditional training phase is bypassed in favor of instantaneous manifold discovery. Finally, we derive the Voodoo Scaling Law , a theoretical framework that predicts accuracy bounds as a function of Galactic dimensionality and intrinsic manifold entropy. 1 Introduction The prevailing consensus in machine learning suggests that feature hierarchies must be painstakingly learned through backpropagation. However, the computational cost of this ”weight cooling” process is non-trivial. In this work, we propose VoodooNet , a model that treats neural network training as a structural discovery problem rather than an optimization marathon. The core of VoodooNet rests on two mathematical ”magic hats”: 1. The Galactic Layer: Utilizing the Johnson-Lindenstrauss (JL) Lemma, we perform a random projection of the 784-dimensional MNIST manifold into a d -dimensional hidden space (where d\gg 784 ). This expansion preserves the topological distances of the data while increasing linear separability. 2. The Magic Hat Solution: Instead of gradient-based search, we employ the Moore-Penrose Pseudoinverse ( \dagger ) to analytically solve for the output weights in a single matrix operation. VoodooNet demonstrates that for many tasks, the ”Ground State” of a neural network is not a destination to be reached through iteration, but a structural property that can be calculated via global analytic optimization. This approach aligns with the philosophy of Extreme Learning Machines (ELM) and Reservoir Computing, providing a ”Voodoo-like” speedup that defies conventional training timelines. 2 Related Works The development of VoodooNet is situated at the intersection of high-dimensional geometry, randomized neural networks, and analytic optimization. Our approach builds upon several foundational frameworks while introducing a novel ”Galactic” perspective on manifold untangling. 2.1 Extreme Learning Machines (ELM) and Reservoir Computing The ”Magic Hat” solution presented in this work shares a conceptual lineage with Extreme Learning Machines (ELM) [ 2 ] and Reservoir Computing (RC). These paradigms posit that hidden layers do not require iterative tuning if the input is projected into a sufficiently high-dimensional space. While ELMs focus on the efficiency of the Moore-Penrose pseudoinverse, VoodooNet extends this by explicitly leveraging the Johnson-Lindenstrauss Lemma to justify the expansion into the ”Galactic” regime as a topological preservation mechanism. 2.2 Random Projections and Compressed Sensing The utility of random linear mappings is well-documented in the context of the Johnson-Lindenstrauss (JL) Lemma [ 3 ] . Traditionally, random projections are used for dimensionality reduction to save computational costs. VoodooNet, however, utilizes ”Galactic Expansion”—a form of dimensionality augmentation . This aligns with the ”Cover’s Theorem” in pattern recognition, which states that a complex pattern-classification problem is more likely to be linearly separable in a high-dimensional space than in a low-dimensional space. 2.3 The Information Bottleneck and Weight Entropy Conventional training via Stochastic Gradient Descent (SGD) is often analyzed through the Information Bottleneck (IB) theory [ 5 ] . This theory suggests that networks learn by first expanding information and then compressing it to retain only relevant features. Our results in Section 3 contrast this by showing that VoodooNet bypasses the ”compression phase,” reaching a functional ground state while maintaining high Shannon Entropy. This positions VoodooNet as an uncompressed analytic alternative to the ”thermodynamic cooling” of iterative descent. 2.4 Analytic vs. Iterative Optimization While backpropagation has become the de facto standard for deep learning, interest in closed-form solutions persists for Edge AI. Recent work in Forward-Forward algorithms and Feedback Alignment explores alternatives to gradients, but often still relies on iterative steps. VoodooNet differs by providing a truly ”one-shot” analytic solution, trading structural sparsity for near-instantaneous convergence. 3 Methodology The VoodooNet architecture operates on the principle of Analytic Global Optimization rather than iterative local search. The process is divided into two distinct functional phases: the Galactic Projection and the Magic Hat Solution. 3.1 Phase I: The Galactic Projection The primary challenge in classifying the MNIST dataset is the non-linear entanglement of digit manifolds in the raw pixel space ( \mathbb{R}^{784} ). To resolve this, we employ a random projection into a high-dimensional “Galactic” space \mathbb{R}^{d} , where d=2000 . According to the Johnson-Lindenstrauss Lemma [ 3 ] , a random linear mapping essentially preserves the pairwise distances of the data points while significantly increasing their linear separability in the expanded space. We define the hidden state matrix H as: H=\sigma(XW_{1}+b_{1}) (1) where W_{1}\in\mathbb{R}^{784\times d} and b_{1}\in\mathbb{R}^{d} are initialized from a standard normal distribution \mathcal{N}(0,1) and remain frozen throughout the procedure. The non-linear activation \sigma (ReLU) serves to “fold” the space, creating the necessary boundaries for classification. 3.2 Phase II: The Magic Hat (Pseudoinverse) Solution Once the data is projected into the Galactic space, we treat the output layer as a linear system: HW_{2}=Y . Traditionally, W_{2} is found via backpropagation, which can be interpreted as a slow thermodynamic cooling toward a local minimum. In the VoodooNet paradigm, we jump directly to the optimal configuration by calculating the Moore-Penrose Pseudoinverse [ 1 ] . This provides the unique minimum-norm least-squares solution: W_{2}=H^{\dagger}Y=(H^{T}H)^{-1}H^{T}Y (2) While Equation 2 defines the theoretical ground state, the numerical path to H^{\dagger} introduces a trade-off between floating-point precision and latency. We distinguish between the Exact SVD Hat (singular value decomposition), which provides maximum stability by filtering near-zero eigenvalues, and Decomposition Hats (LU, Cholesky) (See Table 4 ), which offer O(d^{3}) speedups at the cost of slight accuracy degradation in rank-deficient regimes. As noted in the literature regarding Extreme Learning Machines (ELM) [ 2 ] , this analytic solution provides a functional “Ground State” instantly. 3.3 Information Density and Shannon Entropy While this method is computationally efficient, it produces a dense weight distribution. Following the principles of Information Theory [ 4 ] , we evaluate the structural complexity of W_{2} by measuring its Shannon Entropy: S=-\sum p(w_{i})\log p(w_{i}) (3) This allows us to compare the ”ordered” sparsity of iterative methods against the “high-entropy” efficiency of the Magic Hat solution, as discussed in the context of the Information Bottleneck [ 5 ] . 3.4 Random Projection and the JL Lemma The Johnson-Lindenstrauss Lemma ensures that a set of N points in high-dimensional space can be embedded into k\approx O(\log N) dimensions without distorting pair-wise distances. In VoodooNet, we use this in reverse: we expand the input to a higher dimension to ”untangle” the class boundaries. Let X be the input… Algorithm 1 VoodooNet: Instant Analytic Learning 0: Input X\in\mathbb{R}^{N\times 784} , Targets Y\in\mathbb{R}^{N\times 10} 0: Output Weights W_{2} for inference 1:

Read the complete text