Structured Proper Loss Geometries for Multiclass Classification: Theory and Controlled Empirical Evaluation

Soumyadip Sarkar · 2026

Read Structured Proper Loss Geometries for Multiclass Classification: Theory and Controlled Empirical Evaluation by Soumyadip Sarkar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29471.txt.

Introduction

Sarkar studies proper scoring rules, loss functions whose best possible answer is the true probability distribution over classes. Three multiclass objectives receive close attention. They are a class-aware quadratic score called CAPM, a strongly convex generator with log-cosh ridges called HPG, and an HPG variant with a fading probability-margin penalty called APMS.

Sarkar derives bounds on regret, curvature, range, and logit gradients for the first two objectives. Exact bounds on the penalty range and target shift follow for APMS. Controlled experiments with five random seeds cover the Digits and Wisconsin breast cancer datasets plus synthetic problems. Conditions include clean labels, two kinds of label noise, class imbalance, and adversarial perturbations, meaning small hostile input changes. The candidates match cross-entropy on clean data and gain in some noisy cells. Sarkar states plainly that the evidence does not support a claim of general superiority.

Catalog record

CREATORSoumyadip Sarkar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.29471

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Structured Proper Loss Geometries for Multiclass Classification: Theory and Controlled Empirical Evaluation Soumyadip Sarkar Abstract Strictly proper scoring rules identify the true conditional class distribution at population level, but their curvature can alter optimization and finite-sample behavior. We study three multiclass objectives: a class-aware quadratic Bregman score (CAPM), a strongly convex generator with constrained log-cosh ridges (HPG), and an HPG objective with an annealed probability-margin penalty (APMS). CAPM is treated as a structured instance of established quadratic scoring-rule theory. We derive conditional-regret, curvature, range, and logit-gradient bounds for CAPM and HPG, and prove exact penalty-range and conditional-target displacement bounds for APMS. Controlled five-seed experiments use Digits, Wisconsin breast cancer, and synthetic confusion and long-tail problems under clean labels, symmetric and pair-flip corruption, class imbalance, calibration evaluation, input corruption, and first-order adversarial perturbations. The candidates are close to cross-entropy on clean data and show descriptive gains in some noisy-label cells, but the five-seed comparisons are interpreted descriptively rather than as significance evidence. The selected noisy-label baselines perform better on Digits with 40% symmetric label noise, and explicit prior-adjustment methods perform better in the 30:1 synthetic long-tail experiment. Ablations do not show a consistent benefit from the candidate-specific graph, ridge, or margin components. The mathematical analysis establishes the stated properties, and the experiments delimit the empirical evidence; together they do not support a claim of general superiority. Keywords: proper scoring rules; Bregman divergence; multiclass classification; label noise; long-tailed learning; calibration. 1 Introduction The loss function used to train a probabilistic classifier determines the population quantity being estimated and affects the geometry through which model parameters are optimized. This paper uses softmax cross-entropy as the reference baseline; the corresponding logarithmic score is strictly proper under the usual probability-level interpretation [ 10 ] . Nevertheless, finite neural networks trained with cross-entropy can be miscalibrated, and sufficiently expressive networks can fit corrupted labels [ 12 , 2 ] . These observations have motivated objectives designed for particular failure modes, including focal loss for class imbalance in dense detection [ 17 ] , generalized and symmetric cross-entropy for noisy labels [ 34 , 30 ] , and prior- or margin-adjusted objectives for long-tailed recognition [ 8 , 5 , 27 , 20 ] . Strictly proper scoring rules provide a statistical constraint: at the population level, their conditional risk is uniquely minimized by the true class-probability vector [ 10 , 21 ] . Properness does not, however, determine finite-sample accuracy, calibration after model misspecification, optimization speed, robustness to corrupted labels, or performance under class-prior shift. The curvature of a proper score can therefore matter even when different scores share the same population minimizer. Existing work on composite losses, learned proper losses, and task-tailored proper scores shows that proper objectives can have adaptable geometry [ 25 , 26 , 15 , 24 ] . This paper studies three multiclass objectives. The first, class-aware proper Mahalanobis loss (CAPM), is a structured quadratic Bregman score. The second, hyperbolic proper generator loss (HPG), adds constrained log-cosh ridge terms to a quadratic generator. The third, annealed probability-margin shaping (APMS), augments HPG with a bounded margin penalty whose coefficient is scheduled toward zero. CAPM is an application-specific parameterization of classical quadratic proper-score theory rather than a new scoring-rule family. HPG and APMS are analyzed as concrete constructions, with the contribution centered on their exact properties and controlled comparison rather than on a broad claim about adaptable proper geometry. The study makes three contributions. First, it gives complete definitions and derives conditional regret, curvature, range, and gradient bounds for CAPM and HPG. Second, it proves the exact range of the APMS margin penalty on the simplex and establishes both square-root and linear bounds on the displacement of its conditional minimizer from the true probability vector. Third, it reports cross-entropy comparisons across the evaluated regimes and targeted comparisons with established noisy-label, long-tail, and calibration baselines. The empirical results are deliberately interpreted at the scale supported by the experiments: the proposed structures are close to or slightly above cross-entropy in several cells but do not consistently outperform specialized noisy-label or long-tail methods. 2 Related work 2.1 Proper scoring rules and composite losses The Brier score is a classical quadratic probability score [ 4 ] . General proper-scoring-rule theory characterizes losses whose expected value is optimized by truthful probabilistic reporting [ 10 ] . For differentiable finite-outcome scores, convex entropies and Bregman divergences provide a standard representation under appropriate regularity assumptions [ 21 ] . Composite-loss theory separates a probability-level proper loss from a link that maps model outputs to probabilities and characterizes convexity and classification calibration [ 3 , 25 , 26 ] . Recent studies have trained language models with non-logarithmic proper scores [ 28 ] , learned proper multiclass losses [ 15 ] , and tailored proper-score curvature to downstream estimation error [ 24 ] . These works motivate treating the present contribution as a specific construction and evaluation rather than as a broad novelty claim based solely on replacing log loss with a parameterized proper geometry. 2.2 Learning with noisy labels Robustness to label noise is model- and assumption-dependent. Symmetric-loss analyses establish risk invariance under idealized corruption models [ 9 ] . Loss-correction methods use an estimated transition matrix [ 22 ] . Generalized cross-entropy interpolates between cross-entropy and an MAE-like objective [ 34 ] ; symmetric cross-entropy combines ordinary and reverse cross-entropy [ 30 ] ; bi-tempered logistic loss modifies logarithmic and exponential functions in a Bregman construction [ 1 ] ; and normalized active–passive losses address the underfitting that can occur with bounded or symmetric objectives [ 18 ] . Synthetic symmetric and pair-flip noise are useful controlled settings but do not reproduce the heterogeneity of human annotation errors represented by datasets such as CIFAR-N [ 31 ] . 2.3 Long-tailed learning, calibration, and adversarial evaluation Long-tailed learning methods alter sample weights, margins, or class priors. Class-balanced loss uses the effective number of samples [ 8 ] ; LDAM introduces class-dependent margins [ 5 ] ; Balanced Softmax modifies the softmax normalization using class frequencies [ 27 ] ; and logit adjustment incorporates empirical priors during or after training [ 20 ] . Focal loss with a positive focusing parameter is classification calibrated but is not a proper probability loss without a correction map [ 6 , 14 ] . Temperature scaling is a simple post-hoc calibration method that preserves the predicted class for positive temperatures [ 12 ] . Adversarial accuracy is distinct from probability-space curvature. FGSM and projected-gradient attacks are empirical evaluation methods [ 11 , 19 ] ; adversarial training objectives such as TRADES optimize an explicit robustness–accuracy trade-off [ 33 ] ; and attack ensembles such as AutoAttack are designed to reduce evaluation pitfalls [ 7 ] . Consequently, the gradient bounds derived below are not adversarial certificates. 3 Structured loss formulations 3.1 Preliminaries Let Y\in\{1,\ldots,K\} , let \eta\in\Delta^{K-1} denote the conditional class distribution, and let p\in\Delta^{K-1} be a reported distribution. For a differentiable convex function F defined on a neighborhood of the simplex, its Bregman divergence is D_{F}(q,p)=F(q)-F(p)-\langle\nabla F(p),q-p\rangle. (1) We use the negatively oriented categorical score \ell_{F}(p,y)=D_{F}(e_{y},p), (2) where e_{y} is the y th standard basis vector. The following standard identity fixes the statistical

Read the complete text