Extended predictive coding framework as variational free-energy minimisation under exponential-family assumption

Asaki Kataoka, Kenji Doya · 2026

Read Extended predictive coding framework as variational free-energy minimisation under exponential-family assumption by Asaki Kataoka, Kenji Doya (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30882.txt.

Introduction

Kataoka and Doya widen predictive coding, a brain theory that casts perception as variational inference under the free-energy principle. Past versions tied to Gaussian assumptions miss key neural traits, like nonlinear input-output curves, varied responses across a net, and the fact that firing rates cannot go negative. By assuming the wider exponential family for prior and posterior, they recover those traits while keeping the free-energy and predictive-coding link up to the second cumulant. The model also learns by local plasticity rules that are biologically plausible.

Catalog record

CREATORAsaki Kataoka, Kenji Doya
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78634
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30882

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Extended predictive coding framework as variational free-energy minimisation under exponential-family assumption Asaki Kataoka, Kenji Doya Abstract The sensory cortices of the brain perform perceptual inference efficiently through their complex networks of neurons. One of the theoretical accounts of this process is the free-energy principle (FEP), which postulates that the brain performs variational Bayesian inference. Pioneering studies have shown that FEP can correspond to the predictive coding (PC) hypothesis under the Gaussian assumption and Laplace approximation. However, PC-based implementations of FEP within such a limited Gaussian regime have failed to capture several properties of biological neural networks, such as nonlinearity and heterogeneity of input–output properties within a network, and the biological implausibility of negative firing rates. This study shows that, when a broader class of probability distributions, namely the exponential family of distributions (EFD), is assumed for the variational posterior and prior, these missing characteristics are exhibited within the network, maintaining the FEP–PC correspondence up to the second cumulant of the posterior. We also show that the proposed model can be trained by biologically plausible local plasticity rules. Our results enrich the explanatory power of FEP regarding neural dynamics involved in perception as variational inference. 1 Introduction The free-energy principle (FEP), which has attracted immense attention from the neuroscience community, posits that the brain functions as a variational Bayesian estimator [ 11 , 10 , 12 , 6 ] . This theory is grounded in Helmholtz’s early proposal of unconscious inference; since perception is fundamentally an ill-posed problem of inferring the hidden state of the external world from noisy, information-discarding sensory observations, the brain must implicitly infer it based on past experiences [ 15 ] . This concept was later mathematically formalised as the Bayesian brain hypothesis, where prior experiences correspond to a prior distribution used to estimate the posterior [ 17 , 9 , 7 ] . Within Marr’s three-level notion, FEP serves as a principle at the computational level. It circumvents the intractable integration required for the marginal likelihood (or evidence) by stating that the brain instead minimises variational free energy—an upper bound on the negative log marginal likelihood, i.e. the sensory surprisal. Following the proposal of this theory by Friston, it has been extensively demonstrated that, under Gaussian assumption and/or Laplace approximation, a possible neural implementation of FEP takes a form structurally identical to predictive coding (PC) [ 10 , 6 ] . This was a foundational result, as it provided a plausible algorithmic and neural basis for this abstract principle. PC postulates two distinct types of neurons: representational neurons , which encode current belief about the state of the external world and generate top-down predictions regarding lower-level activities, and error-coding neurons , which compare these top-down predictions with bottom-up signals to send prediction errors back up the functional hierarchy [ 24 ] . However, it has been unknown whether this correspondence holds for non-Gaussian assumptions. Previous studies on FEP relating it to PC have assumed that posterior, prior, and likelihood are all Gaussian, and the current literature has only shown this relation under such a limited assumption. This has created a huge gap between the general and broad explanatory power of FEP expected from its fundamental notion and the narrowness of assumption it requires for formulating promising neural implementation. This study shows that the Gaussian assumption for the prior and posterior can be relaxed without drastically violating the FEP–PC correspondence. In particular, we show that the gradient descent dynamics can take the form of PC even when assuming that the posterior and prior belong to exponential family of distributions (EFD), which is a broader class of probability distributions beyond Gaussian regime. Under the EFD assumption for posterior and prior and Gaussian assumption for likelihood, the inferential dynamics of the representational neurons are driven by two additive signals: attraction drive towards the prior and prediction-error drive. Based on this theoretical result, we propose EFD–FEP model as an extended PC model. In each layer of this model, the approximate posterior is represented by a population of representational neurons. We define “internal states” of neurons, which correspond to the parameters of the posterior, and sampled random values corresponding to the firing activities of these neurons. The prior-attracting drives and prediction signals are represented by top-down message-passing, and the additive prediction errors are propagated in a bottom-up manner. Furthermore, our model reflects several known biological and electrophysiological properties. These features enhance the explanatory power of FEP as computational formulation of neural computation involved in perceptual inference in the brain. First, synaptic weight updates, also derived from gradient descents in the shared objective function as the inferential dynamics, further enhance the plausibility of its biological neural implementation. The fundamental learning rules of our model are twofold (Section 3.2.3 ): Hebbian-like plasticity and a more complex plasticity rule based on total input intensity. Although the latter might appear difficult to implement in biological neural systems at first glance, it conceptually aligns well with recent findings of plasticity rules based on dendritic plateau potentials and calcium-mediated firing (BAC firing) in the apical tufts of pyramidal neurons [ 18 , 19 , 5 ] . Second, our model, particularly under the condition that the posterior can be factorised into independent elementwise distributions, allows for heterogeneity or diversity and nonlinearity in neuron-level electrophysiological frequency–current (F–I) properties, or activation functions, within a single network, having been observed experimentally [ 25 , 22 , 2 , 8 , 20 ] . Previous PC models, especially those normatively derived from Gaussian modelling [ 24 , 10 , 6 , 21 ] , have assumed linear activations; these models require neurons to “fire” at firing rates proportional to the intensity of the input. Unlike such models, our framework accepts arbitrary F–I curves as long as they satisfy only two conditions: monotonically increasing and differentiable. This also resolves the negative-firing problem that has often been observed in previous PC models. While linear activations allow the model to sample negative firing activities, biological neurons do not emit negative firing rates. Although introducing rectification can suppress such negativity, this modification is non-normative and fundamentally violates VFE minimisation in that regime. Conversely, our model accepts nonlinear activations that converge to, but do not fall below, zero, thereby preventing negative firing while maintaining the normative objective of VFE reduction. Furthermore, our framework embraces the heterogeneity of nonlinearity. This allows for, for example, neuron-wise difference of shifts and gains of the sigmoidal activation, or coexistence of neurons with sigmoidal, exponential, and hyperbolic activations within a single population of neurons. Unlike the previous PC models and Gaussian-assuming FEP framework on which they have relied, our model keeps the PC framework while ensuring that it effectively reduces the VFE as postulated in FEP. In particular, the biology-aware properties found in our model further strengthen FEP and PC hypothesis as an explanatory theory for neural computations in the brain and its cortices. We also discuss several remaining issues regarding biological plausibility left even in our model. 2 Background 2.1 free-energy principle The brain infers the state of the world using limited information provided by sensory inputs. For example, inferring the spatial arrangement of two objects from two-dimensional retinal projections is theoretically unsolvable. Even when one object is in front

Read the complete text