Group Entropies and Mirror Duality: A Class of Flexible Mirror Descent Updates for Machine Learning

Andrzej Cichocki, Piergiulio Tempesta · 2026

Read Group Entropies and Mirror Duality: A Class of Flexible Mirror Descent Updates for Machine Learning by Andrzej Cichocki, Piergiulio Tempesta (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08651.txt.

Introduction

Andrzej Cichocki and Piergiulio Tempesta build a wide family of mirror descent updates from group entropies. Their group-based maps swap cleanly with their inverses, a trick they call mirror duality, and tuning the map parameters fits the update to the data shape. Tests on large simplex-bound programs back the updates' validity, toughness, and performance.

Catalog record

CREATORAndrzej Cichocki, Piergiulio Tempesta
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93478
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.08651

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Group Entropies and Mirror Duality: A Class of Flexible Mirror Descent Updates for Machine Learning Andrzej Cichocki, Piergiulio Tempesta Abstract . We introduce a comprehensive theoretical and algorithmic framework that bridges formal group theory and group entropies with modern machine learning, paving the way for an infinite, flexible family of Mirror Descent (MD) optimization algorithms. Our approach exploits the rich structure of group entropies, which are generalized entropic functionals governed by group composition laws, encompassing and significantly extending all trace-form entropies such as the Shannon, Tsallis, and Kaniadakis families. By leveraging group-theoretical mirror maps (or link functions) in MD, expressed via multi-parametric generalized logarithms and their inverses (group exponentials), we achieve highly flexible and adaptable MD updates that can be tailored to diverse data geometries and statistical distributions. To this end, we introduce the notion of mirror duality , which allows us to seamlessly switch or interchange group-theoretical link functions with their inverses, subject to specific learning rate constraints. By tuning or learning the hyperparameters of the group logarithms enables us to adapt the model to the statistical properties of the training distribution, while simultaneously ensuring desirable convergence characteristics via fine-tuning. This generality not only provides greater flexibility and improved convergence properties, but also opens new perspectives for applications in machine learning and deep learning by expanding the design of regularizers and natural gradient algorithms. We extensively evaluate the validity, robustness, and performance of the proposed updates on large-scale, simplex-constrained quadratic programming problems. 1. Introduction The purpose of this article is to merge formal group theory with machine learning techniques with the aim of constructing novel mirror descent algorithms that possess suitable group-theoretical properties, making them flexible tools for optimization problems. A variety of fundamental gradient-based update schemes have been thoroughly investigated within the domains of optimization, machine learning, and artificial intelligence. These include additive gradient descent (GD) [ 41 , 22 ] , its stochastic variant (SGD), multiplicative updates (MU) [ 18 , 14 , 17 ] , exponentiated gradient (EG) descent [ 37 , 3 ] , and mirror descent (MD) algorithms [ 41 , 2 , 9 , 22 , 45 ] . The additive gradient descent (GD) method and its multiplicative counterpart, the exponentiated gradient (EG) update, are among the most commonly used algorithms in machine learning. We recall that EG descent updates, which are a subclass of both multiplicative gradient schemes [ 18 ] and mirror descent (MD) schemes (e.g., [ 2 , 22 ] ), were introduced by Kivinen and Warmuth [ 37 , 36 ] . EG algorithms have since been extended and applied across diverse domains by numerous researchers [ 24 , 25 , 40 , 42 , 57 , 38 , 22 ] . Mirror descent is a versatile optimization framework, prevalent in optimization, AI, machine learning, and online learning, that generalizes classical gradient descent to non-Euclidean settings by means of a mirror map (or potential function). Rather than using the standard Euclidean norm, mirror descent employs a Bregman divergence, that serves as a proximal regularizer for the optimized loss function. A crucial aspect is that the geometry of the problem, in the mirror gradient descent approach, is incorporated by the strictly convex potential (or generating) function F(\bm{w}) . By choosing flexible potential functions, mirror descent can be adapted to the specific optimization problem, in order to ensure faster convergence and/or better robustness. Generally speaking, this makes mirror descent superior to standard gradient descent optimization methods. Observe that by adopting F(\bm{w})=(1/2)||\bm{w}||_{2}^{2} as a distance-generating function, then the Bregman distance reduces to the Euclidean distance, and mirror descent to standard gradient descent. By using parameterized potential functions, we can adapt to different geometries, such as the simplex or more general convex sets. Exponentiated Gradient descent and Mirror Descent have seen widespread adoption across numerous optimization and machine learning applications, and have also been successfully applied to non-convex problems [ 41 , 9 , 2 , 3 , 22 , 45 ] . Although additive gradient descent remains the predominant optimization method, it is often ill-suited for problems requiring all components of the weight vector to remain nonnegative. Furthermore, when employing additive gradient updates, practitioners frequently encounter vanishing or exploding gradients, which require meticulous learning-rate selection and tuning to ensure stable and effective training in real-world applications [ 41 ] . Interestingly enough, EG weight updates can mitigate many of the issues inherent to additive GD, without the need for finely tuned learning rates. In particular, when the target weight vectors are sparse, the un-normalized exponentiated gradient (EGU) and normalized exponentiated gradient variants have been shown to converge more rapidly than standard GD [ 37 , 25 , 2 ] . Conventional exponentiated gradient (EG) updates are most naturally suited to domains whose geometry is governed by the Kullback-Leibler divergence, as is common when optimizing over probability distributions. However, a significant drawback of these classic EG schemes is their rigidity: the absence of adjustable hyperparameters prevents them from adapting to datasets with diverse distributional properties, thereby constraining both their convergence dynamics and practical performance. The central contribution of this work is a general theoretical framework which establishes a rigorous connection between machine learning and group theory. In fact, in order to take into account the geometry of the problem under consideration, we design exponentiated gradient updates based on multi-parametric link functions grounded in group-theoretic principles. More precisely, the theory of formal groups [ 23 , 44 ] , a fundamental branch of algebraic topology, offers a rich source of link functions expressed in terms of generalized logarithms and exponentials. These generalized functions, in turn, find their primary physical motivation in the theory of group entropies . This approach was first introduced by one of us in [ 49 ] , further developed in [ 50 , 51 , 52 , 53 ] and subsequently explored by various authors (see [ 43 , 48 , 4 , 5 ] and [ 28 , 29 ] for reviews). The notion of group entropy represents an axiomatic formulation of the theory of generalized entropies, based on the first three Shannon-Khinchin axioms (continuity of entropy, maximum entropy principle, expansibility) and the new composability axiom formulated in [ 51 ] . These requirements are necessary to ensure that a given entropy can be suitable for information-theoretical and thermodynamic purposes. From a mathematical point of view, the composability axiom amounts to saying that the entropy S(A\times B) of a system composed of two systems A and B has to satisfy a group law. For instance, the standard Boltzmann-Gibbs entropy satisfies the additivity law; the Tsallis entropy fulfils the so-called multiplicative law S(A\times B)=S(A)+S(B)+(1-q)S(A)S(B) , and so on. In this work, we present two main results. First, we shall prove that the class of known mirror descent algorithms can be broadly generalized through the theory of group entropies. In fact, this theory naturally yields infinitely many multiparametric generalized logarithms , that we will interpret as group-theoretical link functions . We also propose a novel construction of group logarithms from known ones, and introduce the notion of chain link functions . They represent a novel and versatile tool which adapts to the features of the optimization problem under consideration. A key

Read the complete text