On the Practical Estimation and Interpretation of Rényi Transfer Entropy

Zlata Tabachová, Petr Jizba, Hynek Lavička, Milan Paluš · 2026

Read On the Practical Estimation and Interpretation of Rényi Transfer Entropy by Zlata Tabachová, Petr Jizba, Hynek Lavička, Milan Paluš (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-01497.txt.

Introduction

Tabachová and 3 co-authors examine a tool for tracing which part of a complex system drives another. Transfer entropy tracks directed flow of information, and its Renyi form adds a dial that favors rare or common events. In practice, short or messy data sets make the tool hard to estimate and even harder to read.

The authors test a nearest-neighbor estimator on made-up data with known cause and effect, varying sample size, dimension, memory, and the dial setting. They set three reliability checks and give tuning advice to handle odd negative values. When checks pass, the estimator tracked the true direction of flow across linear and nonlinear cases.

Catalog record

CREATORZlata Tabachová, Petr Jizba, Hynek Lavička, Milan Paluš
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE111 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-01497

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On the Practical Estimation and Interpretation of Rényi Transfer Entropy Zlata Tabachová, Petr Jizba, Hynek Lavička, Milan Paluš Abstract Rényi transfer entropy (RTE) is a generalization of classical transfer entropy that replaces Shannon’s entropy with Rényi’s information measure. This, in turn, introduces a new tunable parameter α \alpha , which accounts for sensitivity to low- or high-probability events. Although RTE shows strong potential for analyzing causal relations in complex, non-Gaussian systems, its practical use is limited, primarily due to challenges related to its accurate estimation and interpretation. These difficulties are especially pronounced when working with finite, high-dimensional, or heterogeneous datasets. In this paper, we systematically study the performance of a k k -nearest neighbor estimator for both Rényi entropy (RE) and RTE using various synthetic data sets with clear cause-and-effect relationships inherent to their construction. We test the estimator across a broad range of parameters, including sample size, dimensionality, memory length, and Rényi order α \alpha . In particular, we apply the estimator to a set of simulated processes with increasing structural complexity, ranging from linear dynamics to nonlinear systems with multi-source couplings. To address interpretational challenges arising from potentially negative RE and RTE values, we introduce three reliability conditions and formulate practical guidelines for tuning the estimator parameters. We show that when the reliability conditions are met and the parameters are calibrated accordingly, the resulting effective RTE estimates accurately capture directional information flow across a broad range of scenarios. Results obtained show that the explanatory power of RTE depends sensitively on the choice of the Rényi parameter α \alpha . This highlights the usefulness of the RTE framework for identifying the drivers of extreme behavior in complex systems. I Introduction Understanding the causal mechanisms in complex systems represents a fundamental and ongoing challenge across a broad range of scientific fields, including neuroscience, energetics, climate science, finance, engineering, and nonlinear dynamical systems Kantz and Schreiber 2003 ; Lizier and Prokopenko 2010 . These systems are often characterized by high-dimensional, non-stationary, and nonlinear interactions, which complicate the identification of causal or effective relationships among their constituent components. When the processes under study evolve over time and are represented by sequences of measurements recorded at successive time points — commonly arranged as time series — it becomes possible to apply quantitative methods to infer causal relationships between variables across different datasets. One of the foundational approaches to causality in time series analysis is the Granger causality (GC), which was originally introduced by Granger Granger 1969 and based on earlier ideas about predictability proposed by Wiener Wiener 1956 . Granger’s formulation is grounded in linear autoregressive models, and it has been widely used due to its mathematical tractability and conceptual simplicity. In practice, however, many real-world systems exhibit complex, nonlinear dynamics that fall outside the scope of linear models. To address this limitation, a variety of nonlinear extensions of GC have been proposed in the literature, drawing on ideas from the theory of nonlinear dynamical systems Quiroga et al. 2000 ; Ye et al. 2015 ; Paluš et al. 2018 , data compression efficiency Kathpalia et al. 2022 , machine learning Huang et al. 2020 , and information theory Ay and Polani 2008 ; Jizba et al. 2012 ; Jizba et al. 2022 ; Hlaváčková-Schindler et al. 2007 , among others. In particular, information theory provides powerful tools and conceptually transparent frameworks for analyzing causal relationships through directional dependencies in information flow. In this context, one of the most widely adopted tools for detecting and quantifying the directional information flow between time-dependent processes is transfer entropy (TE) or equally conditional mutual information (CMI) — an information-theoretic measure introduced by Schreiber Schreiber 2000 , Kantz et al. Marschinski and Kantz 2002 , and Paluš et al. Paluš et al. 2001 . In essence, TE quantifies the amount of directed (time-asymmetric) information flow from a source process to a target process. It does so by measuring the reduction in uncertainty about the target’s future state when the past states of the source are considered, beyond what is already explained by the target’s own history. Unlike methods based on explicit modeling assumptions, TE is model-independent and does not require prior knowledge of the underlying dynamics. This makes TE particularly suitable for analyzing complex systems, where interactions may be highly nonlinear or otherwise difficult to parametrize. Consequently, TE has been successfully applied in diverse scientific disciplines Gencaga 2018 ; Bossomaier et al. 2016 , including the Earth sciences Paluš et al. 2024 ; Jajcay et al. 2018 , neurosciences Pereda et al. 2005 ; Vicente et al. 2011 and financial markets Marschinski and Kantz 2002 ; Permuter et al. 2011 ; Jizba et al. 2012 . Although most TE applications rely on the entropy framework introduced by Shannon, recent studies have begun to explore the utility of Rényi’s information theory Rényi 1976 ; Rényi 1961 ; Jizba and Arimitsu 2004a ; Paluš et al. 2024 . The latter provides a natural generalization of conventional TE and, in doing so, it introduces a new tunable parameter, \alpha , which modulates sensitivity to rare versus frequent events. The resulting extension, known as Rényi transfer entropy (RTE), offers several advantages, particularly for systems exhibiting heavy-tailed distributions, skewness, or the presence of outliers that play a critical role in the dynamics of the system Jizba et al. 2012 ; Jizba et al. 2022 ; Paluš et al. 2024 ; Zhang et al. 2023 ; Behrendt et al. 2019 . The RTE framework was originally proposed in Ref. Jizba et al. 2012 to investigate causal information transfer between bivariate financial time series. The underlying idea was that specific values of the Rényi parameter \alpha could selectively amplify only certain regions of the underlying probability distribution function (PDF) while strongly attenuating others. Since extreme events are typically associated with the tails of PDFs, RTE seems to be particularly well-suited for identifying the causal drivers of such rare events. This expectation has been confirmed in a number of works Jizba et al. 2012 ; Jizba et al. 2022 ; Paluš et al. 2024 ; Zhang et al. 2023 ; Korbel et al. 2019 ; Behrendt et al. 2019 . Yet, despite its theoretical appeal, RTE remains underutilized, largely due to a lack of systematic studies addressing its estimation and interpretability, especially in the context of empirical or high-dimensional data. In this paper, we conduct a comprehensive evaluation of the k -nearest neighbors ( k -NN) estimator for RE and RTE, originally proposed by Leonenko et al. Leonenko et al. 2008 . Our analysis proceeds in two stages. First, we systematically investigate how key parameters — such as sample size, number of neighbors k , data dimensionality and memory length — affect estimation accuracy across a broad range of Rényi \alpha -parameters, with particular attention to the regime \alpha<1 , which emphasizes tail events. To explore diverse PDF behaviors, we benchmark the estimator against the RE of the multidimensional t -distribution, which encompasses both the normal and Cauchy distributions as limiting cases with exponentially and polynomially decaying tails, respectively. In the second stage, we apply the estimator to a set of simulated coupled stochastic processes with explicitly defined causal structures. This setup enables a controlled

Read the complete text