Group structure as a foundation for entropies
Henrik Jeldtoft Jensen, Piergiulio Tempesta · 2025
Read Group structure as a foundation for entropies by Henrik Jeldtoft Jensen, Piergiulio Tempesta (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06847.txt.
Introduction
Henrik Jeldtoft Jensen and Piergiulio Tempesta review group structure as a foundation for entropies. Their 2025 paper deals with a concept that means different things in thermodynamics and in data analysis.
Many entropies have been introduced, and judging their merits is difficult. The authors treat entropy abstractly, as a functional on a probability space. Two demands do the sorting: handling the simple case of non-interacting systems, and extensivity, the way entropy should grow with system size. Together these allow a systematic classification of which functional forms are allowed. The review offers a map for choosing among competing entropy definitions.
Catalog record
| CREATOR | Henrik Jeldtoft Jensen, Piergiulio Tempesta |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-06847
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Group structure as a foundation for entropies Henrik Jeldtoft Jensen, Piergiulio Tempesta Abstract . Entropy can signify different things: For instance, heat transfer in thermodynamics or a measure of information in data analysis. Many entropies have been introduced and it can be difficult to ascertain their different importance and merits. Here we consider entropy in an abstract sense, as a functional on a probability space and we review how being able to handle the trivial case of non-interacting systems together with the subtle requirement of extensivity allows a systematic classification of the functional form. 1. Introduction The term “entropy” is used extensively in the modern scientific literature. Originating in the 19th-century theory of thermal dynamics [ 1 ] , the concept is now, to a near bewildering extent, used much widely in one form or another across many sciences. For example, entropy is at the foundation of information theory [ 2 ] , it is of crucial use in computer science [ 3 , 4 ] ; also, neurosciences make use of entropy both as a tool to characterize and interpret data from brained scans [ 5 ] and, more fundamentally, in theories of the dynamics of the brain and mind [ 6 ] . Generally speaking, entropy is a fundamental notion in complexity science [ 7 ] . In this article we present a brief review of some recent mathematical developments in the theory of entropy. In mathematical terms, an entropy is a functional S[p] defined on a space of probability distributions p=(p_{1},p_{2},...,p_{W}) associated with a W -dimensional event space. Thus, we use the word “entropy” with the same meaning that it assumes, for instance, in the case of Rényi’s entropy, namely, with no direct reference to thermodynamics. From this perspective, the relevance of entropies is clear: They can be considered as analytic tools that can help the analysis of the inter-dependencies within the system behind a given event space. Similarly, their use in information-theoretic analysis of time series is likewise natural. The connection between entropies as mathematical functionals and the thermodynamic entropy of Clausius defined in terms of heat transfer is much less immediate. Here we will concentrate on the mathematical aspects of entropies as functionals and only make a few comments on the possible connection to thermodynamics. The first question to tackle is which functional form of S[p] yields useful entropies. i.e., how to limit the infinite number of choices for S[p] . It is well known that the Boltzmann-Gibbs-Shannon form (1.1) S_{\rm BGS}[p]=\sum_{i=1}^{W}p_{i}\log\frac{1}{p_{i}} (we assumed k_{B}=1 ) is the unique possibility if one assumes that the entropy must satisfy the four Shannon-Kinchin (SK) axioms [ 8 ] (SK1) (Continuity). The function S(p_{1},\ldots,p_{W}) is continuous with respect to all its arguments. (SK2) (Maximum principle). The function S(p_{1},...,p_{W}) takes its maximum value over the uniform distribution p_{i}=1/W,i=1,\ldots,W . (SK3) (Expansibility). Adding an event of zero probability to a probability distribution does not change its entropy: S(p_{1},\ldots,p_{W},0)=S(p_{1},...,p_{W}) . (SK4) (Additivity). Given two subsystems A, B of a statistical system, S(A\cup B)=S(A)+S(B|A) . Therefore, to derive entropies of a functional form different from the one in Eq. ( 1.1 ), it is necessary to go beyond the four SK axioms. Various strategies in this respect have been adopted in the literature. Let us start with recalling Constantino Tsallis’s [ 10 ] elegant observation that the formula (1.2) S_{q}[p]=k\frac{1-\sum_{i=1}^{W}p_{i}^{q}}{q-1},\;\;\;\;p=(p_{1},p_{2},...,p_{W})\in[0,1]^{W},\;\;\;k\in\mathbb{R}_{+} provides a possible generalization of Boltzmann’s entropy. This is the case in the sense that S_{q} is a functional on the space of probability distributions p:\{1,2.,...,W\}\mapsto[0,1]^{W} and that in the limit q\rightarrow 1 the entropy S_{q}[p] becomes equal to the Boltzmann-Gibbs-Shannon entropy in Eq. ( 1.1 ). Tsallis’s 1988 article [ 10 ] has inspired a tremendous effort to generalize Boltzmann’s entropy in different scenarios, including what we will review in this paper. Tsallis pointed out that the entropy S_{q} fulfills a specific procedure for combining independent systems which can be seen as a generalization of the additivity property (SK4) of the Boltzmann-Gibbs-Shannon entropy. In particular, Tsallis has suggested that the free parameter q should be determined by requiring that S_{q} for a given system is extensive (for a recent reference to Tsallis’s argument see [ 11 ] ), i.e. that in the uniform case where the probabilities are p_{i}=1/W for all i=1,2..,W the entropy S_{q}\propto N for n\rightarrow\infty , where N denotes the number of components in the system under analysis 1 1 1 For clarity, we note that when considering physical systems, the entropy may become volume dependent for example because the number of states available, W , depends on the volume. Volume dependence can also enter through the probabilities p_{i}^{*} determined by the maximum entropy principle. . Although the Tsallis entropy does not fulfill the axiom SK4 in its original form, and hence is non-additive , it does satisfy a composition relation different from addition. Another set of non-Boltzmann-Gibbs-Shannon entropies was derived by Hanel and Thurner [ 12 ] by simply discarding axiom SK4 and then determining the functional form from the asymptotic behaviour of the number of events, or states, W as a function of the number of components in the system. However, in this approach, there is no systematic rule for handling the computation of the entropy of a system consisting of independent parts. It is well known that in physics the investigation of required symmetries often has been helpful. Think of Einstein’s use of the symmetry between different reference frames to derive special and general relativity theory. Or the eightfold way and the derivation of QCD. Or the application of symmetry and group theory to atomic spectra. It therefore seems natural to rather than discarding the fourth SK axiom replace it in a way informed by the symmetry properties (and related group theoretic restrictions) that an entropy necessarily must satisfy. Consequently, the question is which symmetry cannot be ignored when dealing with entropies. The fourth SK axiom addresses how the entropy of a system AB consisting of two independent parts A and B can be expressed as a sum of the entropy of A and entropy of B . Tsallis entropy also allows the entropy of AB to be expressed in terms of the entropy of the two parts. Not as a sum but as a generalized combination of the entropies of the parts. The notion of group entropy, introduced in [ 13 , 9 ] , goes one step further and exploits the idea that the process of combining two independent systems can be seen as a group operation. The group entropies satisfy the three first SK axioms and a fourth one, which consists of a generalized composition procedure making use of formal group theory [ 14 , 15 ] . This approach leads to axiomatically defined entropies, whose composition rule is defined in terms of the generator G(t) of a suitable formal group law. Namely (1.3) S[p]=\frac{G\big(\ln\sum_{i=1}^{W}p_{i}^{\alpha}\big)}{1-\alpha}\;\;\mathrm{with}\;\alpha>0\;\mathrm{and}\;\alpha\neq 1. Although this restricts the allowed functional forms available for an entropy, it does not, as the four SK axioms, uniquely determine the entropy. We will discuss below how the analysis of combining independent systems by use of formal group theory together with requiring extensivity allows a systematic classification of entropies in “universality classes”: they are defined