α \alpha -Mutual Information for the Gaussian Noise Channel The research at the University of Minnesota was supported in part by the NSF under grant no. CCF-2045237 .

Mohammad Milanian 1 , Alex Dytso 2 , Martina Cardone 1 · 2026

Read α \alpha -Mutual Information for the Gaussian Noise Channel The research at the University of Minnesota was supported in part by the NSF under grant no. CCF-2045237 . by Mohammad Milanian 1 , Alex Dytso 2 , Martina Cardone 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10922.txt.

Introduction

Milanian and two colleagues study Sibson alpha-mutual information for the Gaussian noise channel. The classical Shannon case is well known, linked to mean-square error and Fisher information. General alpha properties remain less studied. The authors prove smoothness properties including finite bounds, change in signal-to-noise ratio and concavity and convexity giving single answers in tuning problems. A gain is an alpha-I-MMSE link extending the old rule, linking the slope of alpha-mutual information in signal-to-noise ratio to mean-square error under tilted inputs.

Catalog record

CREATORMohammad Milanian 1 , Alex Dytso 2 , Martina Cardone 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE157971
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10922

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

α \alpha -Mutual Information for the Gaussian Noise Channel The research at the University of Minnesota was supported in part by the NSF under grant no. CCF-2045237 . Mohammad Milanian 1 , Alex Dytso 2 , Martina Cardone 1 Abstract In this paper, we study Sibson’s α \alpha -mutual information, which we refer to as α \alpha -mutual information, in the context of the additive Gaussian noise channel. While the classical case α = 1 \alpha=1 is well understood and admits deep connections to estimation-theoretic quantities, such as the minimum mean-square error (MMSE) and Fisher information, many of the corresponding structural properties for general values of α \alpha remain less explored. Our goal is to develop a systematic understanding of α \alpha -mutual information in the Gaussian noise setting and to identify which properties extend beyond the Shannon case. To this end, we establish several regularity properties, including finiteness conditions, continuity with respect to the signal-to-noise ratio (SNR) and the input distribution, and strict concavity/convexity properties that guarantee uniqueness in associated optimization problems. A central contribution of this work is the development of an α \alpha -I-MMSE relationship, which generalizes the classical identity by relating the derivative of α \alpha -mutual information with respect to SNR to the MMSE evaluated under appropriately tilted distributions. This connection further leads to a generalized de Bruijn’s identity and new estimation-theoretic representations of Rényi entropy and differential Rényi entropy. We also characterize the low- and high-SNR behaviors of α \alpha -mutual information. In particular, we show that in the low-SNR regime the first-order behavior depends only on the variance of the input, mirroring the classical case. In the high-SNR regime, for discrete distributions, we show that α \alpha -mutual information converges to the Rényi entropy of order 1 / α 1/\alpha of the input. Furthermore, for general distributions, we establish connections between the high-SNR behavior of α \alpha -mutual information and α \alpha -information dimension. Overall, our results demonstrate that many of the fundamental relationships between information measures and estimation-theoretic quantities extend beyond the Shannon setting, albeit in a modified form involving α \alpha -tilted distributions. I Introduction In [ 50 ] , Rényi introduced a one-parameter generalization of Shannon entropy and relative-entropy, which are now known as Rényi entropy and Rényi divergence, respectively. These measures have found a wide variety of applications in source coding [ 13 ] , hypothesis testing [ 7 , 56 ] , and guessing [ 1 ] ; an interested reader is referred to [ 52 ] for an overview of this topic. In contrast, for mutual information, several different generalizations have been proposed: Arimoto’s mutual information [ 3 ] ; Sibson’s mutual information [ 58 ] ; Csiszár’s mutual information [ 16 ] (see also Augustin [ 6 ] ), and Lapidoth and Pfister’s mutual information [ 42 ] ; the reader is referred to [ 6 ] and [ 24 ] for a comprehensive review of the properties of these measures. In this work, we focus on Sibson’s mutual information , which we refer to throughout as \alpha -mutual information. The \alpha -mutual information has a long history in information theory, with classical applications in hypothesis testing and coding [ 16 , 46 ] , and has recently seen renewed interest in statistical learning [ 23 ] , and modern estimation theory [ 54 ] . Despite this renewed attention, many of its fundamental structural properties remain significantly less understood than those of classical mutual information. The goal of this work is to investigate properties of \alpha -mutual information in the context of the classical additive Gaussian noise channel. The Gaussian setting plays a distinguished role in information theory and estimation theory, serving as a canonical model in which sharp characterizations and tight bounds are often attainable. For the special case \alpha=1 , the Gaussian noise model admits explicit formulas and deep connections to estimation-theoretic quantities [ 29 , 70 ] , such as the minimum mean-square error (MMSE), Fisher information, and the I-MMSE relationship, making it a natural benchmark for comparison. By focusing on the Gaussian noise channel, we are able to isolate which properties of classical mutual information are intrinsic to the Gaussian structure and which are specific to the Shannon case \alpha=1 . In particular, since the behavior of mutual information under Gaussian noise is well understood, this setting allows us to systematically examine how, and to what extent, these properties generalize to the \alpha -mutual information. I-A Literature Review Several applications of \alpha -mutual information to classical problems have been reported in [ 67 ] . In particular, Shannon’s zero-error capacity of discrete memoryless channels with feedback [ 57 ] can be expressed in terms of \alpha -mutual information of order zero. Gallager’s random-coding analysis of the maximum-likelihood decoder [ 27 ] can be formulated using \alpha -mutual information. Moreover, certain converse results for variable-to-fixed data transmission [ 14 ] admit representations in terms of \alpha -mutual information, as does Arimoto’s converse on the probability of successful decoding achievable by any code over a discrete memoryless channel [ 2 , 46 ] . The interested reader is referred to [ 68 ] for an in-depth treatment of the connections between \alpha -mutual information and error exponents. More recently, [ 76 ] fully characterized the exponent of the celebrated soft-covering lemma and showed that it depends on \alpha -mutual information. The maximization of \alpha -mutual information has also been connected to privacy measures, e.g., maximal leakage [ 37 ] and \alpha -maximal leakage [ 43 ] . In addition, generalization error bounds between empirical risk and population risk in statistical learning have been established, both in expectation and with high probability, using \alpha -mutual information [ 23 ] . In the estimation-theoretic setting, \alpha -mutual information has been used to derive Bayesian lower bounds that require minimal regularity assumptions on the prior [ 54 , 75 , 25 ] . Connections between problems of universal prediction and \alpha -mutual information have also been established in [ 8 ] . Generic properties of \alpha -mutual information, such as convexity and the data-processing inequality, have been studied in [ 67 , 36 ] . More recently, [ 24 ] established several important properties of \alpha -mutual information, with a particular emphasis on variational characterizations. In [ 39 ] , the authors also derived variational representations with the goal of constructing algorithms to compute the maximum \alpha -mutual information. The problem of maximizing \alpha -mutual information, both with and without constraints, has been thoroughly studied in [ 12 ] . For the special case \alpha=1 , several results are known for the Gaussian noise channel. A complete characterization of the necessary and sufficient conditions for finiteness was provided in [ 74 ] . The derivative of mutual information with respect to the signal-to-noise ratio (SNR) was established [ 29 ] , yielding the celebrated I-MMSE relationship; the interested reader is referred to [ 33 , 32 , 18 ] for a comprehensive overview of related results. Finally, the high-SNR behavior of mutual information has been connected to the information dimension through a series of works, including [ 28 , 74 , 73 ] . I-B Outline

Read the complete text