When sufficiency is insufficient: the functional information bottleneck for identifying probabilistic neural representations
Ishan Kalburge*, Máté Lengyel · 2025
Read When sufficiency is insufficient: the functional information bottleneck for identifying probabilistic neural representations by Ishan Kalburge*, Máté Lengyel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-15671.txt.
Introduction
Ishan Kalburge and Mate Lengyel propose the functional information bottleneck for judging probabilistic neural codes in a 2025 paper. Brains seem to track uncertainty, and task trained networks were said to develop such codes on their own. Prior work, though, lacked a sharp test separating true codes from shortcut recodings of the input.
Their framework judges a code by both sufficiency and minimality, not sufficiency alone. Applied to networks trained on cue combination, coordinate change, and Kalman filtering, the minimality test reverses earlier claims. The networks do not form minimal codes of Bayesian posteriors, and instead rely on shortcut input recoding. When such codes truly arise remains open.
Catalog record
| CREATOR | Ishan Kalburge*, Máté Lengyel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-15671
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
When sufficiency is insufficient: the functional information bottleneck for identifying probabilistic neural representations Ishan Kalburge*, Máté Lengyel Abstract The neural basis of probabilistic computations remains elusive, even amidst growing evidence that humans and other animals track their uncertainty. Recent work has proposed that probabilistic representations arise naturally in task-optimized neural networks trained without explicitly probabilistic inductive biases. However, prior work has lacked clear criteria for distinguishing probabilistic representations—those that perform transformations characteristic of probabilistic computation—from heuristic neural codes that merely reformat inputs. We propose a novel information bottleneck framework, the functional information bottleneck (fIB) , that crucially evaluates a neural representation based not only on its statistical sufficiency but also on its minimality , allowing us to disambiguate heuristic from probabilistic coding. To demonstrate the power of this framework, we study a variety of task-optimized neural networks that had been suggested to develop probabilistic representations in earlier work: networks trained to perform static inference tasks (such as cue combination and coordinate transformation) or dynamic state estimation tasks (Kalman filtering). In contrast to earlier claims, our minimality requirement reveals that probabilistic representations fail to emerge in these networks: they do not develop minimal codes of Bayesian posteriors in their hidden layer activities, and instead rely on heuristic input recoding. Therefore, it remains an open question under which conditions truly probabilistic representations emerge in neural networks. More generally, our work provides a stringent framework for identifying probabilistic neural codes. Thus, it lays the foundation for systematically examining whether, how, and which posteriors are represented in neural circuits during complex decision-making tasks. Keywords: Information bottleneck, probabilistic representations, deep neural networks, Bayesian decision-making 1 Introduction Under a particular generative model of the world that prescribes how latent variables generate observations, the brain may employ one of two broad classes of recognition models to estimate those latent variables ( 18 ) : probabilistic models that employ Bayesian inference ( 25 ) , and non-probabilistic models that compute intermediate values that do not necessarily correspond to posteriors over latent variables, such as function approximating neural networks ( 21 ) . Probabilistic neural representations are advantageous over heuristic ones because they facilitate flexible modularity across computational circuits, information fusion under nuisance variation (such as during temporal filtering), optimal information gathering for maximally reducing one’s uncertainty, and learning the structure of the world ( 18 ) . From a machine learning perspective, they also correspond directly to the notion of adversarial robustness ( 8 ) : networks that act according to a posterior are inherently invariant to nuisance perturbations. Although behavioral evidence suggests that human and other animals are uncertainty-aware in perceptual judgements ( 11 ; 19 ; 7 ) , it remains unclear whether uncertainty is represented probabilistically— i.e., that neural circuits themselves compute with probabilities—or heuristically via dedicated channels for processing uncertainty ( 23 ; 28 ) or by circumventing explicit uncertainty representation altogether 21 . Recent work has suggested that neural networks develop robust internal representations of posteriors even without explicit probabilistic inductive biases ( 27 ) , suggesting that probabilistic representation is an emergent property of near-optimal behavior. However, previous decoding approaches ( 27 ; 37 ) did not separate genuinely probabilistic representations—codes that support the characteristic transformations of probabilistic computation—from trivial reformatting of inputs. To resolve this ambiguity, we seek to identify the defining features that distinguish a probabilistic representation from a non-probabilistic one. Recent debates in probabilistic coding have clarified that defining probabilistic representation ultimately depends on how we define representation itself ( 28 ; 23 ; 38 ) . Building on this view, we connect general criteria for representations to an information bottleneck perspective on probabilistic inference. From this perspective, neural activity constitutes a probabilistic code only when it is approximately sufficient and minimal—preserving exactly the information needed for behavior and generalization, and nothing more. Concretely, task-relevant posteriors should be decodable from hidden activities of networks that exploit uncertainty to behave optimally, but if raw inputs are also decodable, the network has not transformed them into a usable code for downstream computation. This reframes inference as compression, a long-standing idea in Bayesian ( 25 ) and Minimum Description Length ( 15 ) approaches to machine learning but one that has rarely been applied to distinguish neural representations of uncertainty. Here, we demonstrate how this compression-based view can sharpen our understanding of probabilistic coding. Classical information bottleneck approaches to deep learning and neural coding have been instrumental in understanding training dynamics ( 32 ) , characterizing flat minima ( 2 ) , and unifying competing theories about sensory encoding ( 9 ) . However, these analyses are critically limited by their reliance on estimating mutual information, an intractable and sometimes vacuous quantity to estimate in neural networks ( 29 ; 13 ; 12 ) . To circumvent this limitation, we propose the functional information bottleneck (fIB) a framework wherein we measure information content by training linear and nonlinear decoders (“probes”) on the hidden activations of fully trained networks ( 3 ; 22 ) . This not only offers a tractable alternative to mutual information but also provides a more principled test about the format of neural codes. Unlike mutual information, which only measures statistical dependence, probing reveals which information is organized such that it can be efficiently used by downstream neurons ( 41 ) . We study a variety of task-optimized neural networks that had previously been suggested to develop probabilistic representations ( 27 ) : networks trained to perform static inference tasks (such as cue combination and coordinate transformation) or dynamic state estimation tasks (Kalman filtering). While these tasks and their corresponding networks are relatively simple, they are precisely the settings where the relevant posterior is analytically known and experimental evidence suggests that humans and other animals perform probabilistic computation ( 11 ; 39 ; 31 ; 40 ) , allowing us to test whether such representations are truly probabilistic. Using more complex datasets (e.g., images) would obscure the underlying computational question because the ground-truth posterior is unknown; thus, one would not be able to validate our fIB framework without a well-defined generative model. Therefore, we intentionally keep our generative models simple to illustrate the basic principle underlying the framework. Using our novel approach, we demonstrate that, contrary to the findings of ( 27 ) , task-optimized neural networks do not generically form probabilistic representations. 2 Probabilistic representation as an information bottleneck For a function, \mathbf{r}=f(\mathbf{X}) , to be a specific probabilistic representation, where \mathbf{X} denotes the inputs to the function and p_{\mathrm{z}}=p(z|\mathbf{X}) is some target posterior, we want \mathbf{r} to sufficient for the posterior p_{\text{z}} , and we would like for it to be invariant to nuisances \bm{\nu} ( 2 ; 38 ) . For some generic measure of information content I(\cdot) , sufficiency is expressed as I(\mathbf{r};p_{\text{z}})=I(\mathbf{X};p_{\text{z}}) , or that the representation \mathbf{r} is maximally informative about p_{\text{z}} . Invariance enforces that \mathbf{r} filters out nuisance variables \bm{\nu} , i.e., I(p_{\text{z}};\bm{\nu})\approx 0 . Traditional tests for invariance typically isolate known nuisance dimensions ( 37 ) or evaluate out-of-distribution generalization under carefully designed generative models ( 27 ) . While compelling, these approaches cannot exhaust the entire space of possible nuisances nor can they diagnose in-distribution redundancy. From a Bayesian