Geometric Mixture Classifier (GMC) A Discriminative Per-Class Mixture of Hyperplanes for Fast, Transparent Classification
Prasanth K. K, Shubham Sharma · 2025
Read Geometric Mixture Classifier (GMC) A Discriminative Per-Class Mixture of Hyperplanes for Fast, Transparent Classification by Prasanth K. K, Shubham Sharma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-16769.txt.
Introduction
Prasanth K K and Shubham Sharma build a fast, clear classifier in a 2025 paper. Many real classes form several clumps rather than one blob. A single flat split handles clumps poorly. Heavy models fit clumps but are hard to read and tune. Their model gives each class a mix of flat splits instead.
The mix keeps the model open to inspection. A tuning step after training sharpens its probability scores. Tests show a good balance of accuracy, speed, and clarity. It beats plain linear models while staying lighter than kernel or deep rivals. Clear models help in fields that must explain each call.
Catalog record
| CREATOR | Prasanth K. K, Shubham Sharma |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-16769
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Geometric Mixture Classifier (GMC) A Discriminative Per-Class Mixture of Hyperplanes for Fast, Transparent Classification Prasanth K. K, Shubham Sharma Abstract Many real world categories are multimodal: a single class occupies several disjoint regions of feature space. Classical linear models (e.g., logistic regression, linear SVM) impose one global hyperplane and therefore struggle on such data, while high capacity alternatives (kernel SVMs, deep nets) can fit the structure but often sacrifice interpretability, require heavier tuning, and incur higher computational cost. We propose the Geometric Mixture Classifier (GMC), a discriminative model that represents each class as a mixture of hyperplanes. Within a class, GMC aggregates plane scores with a temperature controlled soft-OR (log sum exp) that smoothly approximates the maximum; across classes, it applies a standard softmax to produce probabilistic posteriors. GMC supports an optional Random Fourier Features (RFF) mapping that equips it with nonlinear power while keeping inference linear in the number of planes and lifted features. A practical training recipe : geometry aware initialization via k-means, automatic plane budgeting via silhouette score, alpha annealing, usage aware L 2 L_{2} regularization, label smoothing, and early stopping, makes GMC plug and play in practice. Experiments on synthetic multimodal benchmarks (moons, circles, anisotropic blobs, two spirals) and standard tabular/image features (iris, wine, WDBC breast cancer, digits) show that GMC consistently outperforms linear baselines and k-NN, and is competitive with RBF-SVM, Random Forests, and compact MLPs, and yields transparent geometric introspection via plane and class level responsibility visualizations. Because inference scales linearly in the number of planes and feature dimension, GMC is CPU friendly; on average, inference takes single-digit microseconds per example, typically faster than or comparable to RBF-SVM and compact MLPs. With post hoc temperature scaling, ECE drops from ≈ 0.06 \approx 0.06 to ≈ 0.02 \approx 0.02 on average. GMC therefore occupies a favorable point on the accuracy interpretability efficiency spectrum: more expressive than linear models, lighter and more transparent than kernel or deep models. 1 Introduction Supervised classification is a central problem in machine learning and data science. From medical diagnosis and fraud detection to document tagging and vision, practitioners routinely face datasets where classes are heterogeneous and multimodal , that is, a single semantic class occupies several disjoint regions of feature space. Classical linear models such as logistic regression and linear SVMs are appealing for their convex training, simplicity, and interpretability, but their single global hyperplane struggles on multimodal structures. Kernel machines can capture nonlinearity, yet they often entail higher computational cost, memory footprint, and less transparent decision rules. Deep neural networks excel in accuracy but can require significant tuning, large compute budgets, and typically provide limited geometric introspection into how decisions are made. Problem definition. We seek a classifier that (i) models multimodal class structure without resorting to heavy black box architectures; (ii) remains data and compute efficient; and (iii) provides geometric interpretability, clarifying how, where, and by which sub model a sample is classified. Existing approaches typically trade off at least one of these considerations: linear models lack expressivity; kernel methods and ensembles add complexity and cost; and deep models, while powerful, obscure local decision geometry. Our contributions. We address this gap with the Geometric Mixture Classifier (GMC) , a per class mixture of hyperplanes combined by a soft-OR operator. Our main contributions are: • Model. GMC represents each class c with M_{c} linear experts (\mathbf{w}_{c,m},b_{c,m}) . Within a class, scores are pooled via a temperature-controlled log sum exp (soft-OR) that smoothly approximates the maximum; across classes, a standard softmax produces probabilistic posteriors. • Nonlinear extension. GMC supports an optional Random Fourier Features (RFF) mapping that equips it with nonlinear power while preserving linear time inference in the lifted feature dimension. An auto mode chooses between linear and RFF based on validation likelihood, ensuring robustness across linear and nonlinear regimes. • Training recipe. A practical pipeline geometry aware initialization via k-means, automatic plane budgeting via silhouette score, \alpha annealing (soft \rightarrow sharp pooling), label smoothing, early stopping, and usage aware \ell_{2} regularization—makes GMC stable to train with minimal hyperparameter tuning. • Interpretability. GMC exposes per class plane responsibilities and a soft union of half spaces geometry in the working feature space. We provide diagnostics and visualizations (decision regions, plane overlays, responsibility maps, reliability diagrams) that make the model’s decisions transparent. • Empirical evaluation. Across synthetic multimodal datasets (moons, circles, anisotropic blobs, two spirals) and standard benchmarks (iris, wine, WDBC breast cancer, digits), GMC consistently outperforms linear baselines and k -NN, is competitive with RBF-SVM, Random Forests, and compact MLPs, and achieves low latency CPU inference. With post hoc temperature scaling, calibration improves substantially (ECE \approx 0.06\rightarrow 0.02 on average). In summary, GMC directly encodes the idea that a class is a soft union of locally linear regions, augments it with usage aware regularization for reliability, and demonstrates strong accuracy, interpretability, and efficiency with a light computational footprint. 2 Related Work Linear models. Logistic regression and linear SVMs are strong baselines thanks to convex training, efficiency, and interpretability. Their inductive bias one global hyperplane per class or margin works well when each class is roughly unimodal, but degrades when class regions are scattered or disjoint. GMC preserves the linear flavor but allocates multiple local hyperplanes per class, directly addressing multimodality. Kernel methods. Kernel SVMs (especially with RBF kernels) can model complex, nonlinear structure and often achieve excellent accuracy on small to medium sized datasets. However, their decision geometry is implicit in the kernel expansion, inference cost grows with the number of support vectors, and interpretability is limited. GMC attains similar flexibility via an explicit union of half spaces (optionally in an RFF lifted feature space) and maintains explicit, interpretable linear components with predictable inference cost. Ensemble methods. Tree ensembles such as Random Forests and Gradient Boosted Trees (e.g., XGBoost, LightGBM) are highly effective at capturing heterogeneous decision boundaries. They offer feature importance summaries and partial dependence or SHAP plots, but the overall decision surface is a large ensemble of trees that is less geometrically transparent. GMC, by contrast, provides explicit, piecewise linear acceptance regions per class with clear geometric meaning. Generative mixture models. Gaussian mixture models represent classes via density estimation and classify by comparing class conditional likelihoods. They capture multimodality but rely on distributional assumptions and optimize likelihood rather than direct classification loss. GMC is discriminative: it models separating geometry rather than densities and produces probabilistic posteriors via a class level softmax, with post hoc temperature scaling used when improved calibration is desired. Mixture of experts (MoE). Classical MoE frameworks train both experts and a separate gating network that assigns input dependent weights, combining expert outputs by weighted averaging. GMC shares the “many local experts” idea but differs in key ways: (i) there is no separate gating network responsibilities emerge from log sum exp pooling; (ii) experts are organized per class and combined by a soft-OR (approximate max), not by averaging; and (iii) Usage aware regularization discourages permanently idle experts. Maxout and latent part models. Maxout networks pool linear units with a max operator, and latent part models in vision use a max over components. GMC adopts a similar principle but applies it within each class, using a tunable soft-OR (log sum exp at temperature \alpha ) for smoother