Free Energy and Network Structure: Breaking Scale-Free Behaviour Through Information Processing Constraints
Peter R. Williams \affilnum 1,2 and Zhan Chen \affilnum 3 · 2025
Read Free Energy and Network Structure: Breaking Scale-Free Behaviour Through Information Processing Constraints by Peter R. Williams \affilnum 1,2 and Zhan Chen \affilnum 3 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12654.txt.
Introduction
Peter Williams and Zhan Chen use the free energy principle, a theory of how agents perceive and act, to explain why real networks bend away from the classic scale-free shape. Pure preferential attachment predicts a straight power-law pattern of connections, yet measured networks show a knee. In their minimal model, agents choose links while managing noisy detection and limited processing, and three regimes appear as those limits shift.
When noise dominates, agents hunt for better information and fewer end up isolated than attachment alone predicts. In a sweet spot, gains in detection, belief, and action compound, favouring one cluster size and super-linear growth. Finally, processing limits cap cluster growth altogether. Together the regimes reproduce the knee-shaped patterns seen in real data, casting them as fingerprints of bounded, well-tuned information processing.
Catalog record
| CREATOR | Peter R. Williams \affilnum 1,2 and Zhan Chen \affilnum 3 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-12654
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Free Energy and Network Structure: Breaking Scale-Free Behaviour Through Information Processing Constraints Peter R. Williams \affilnum 1,2 and Zhan Chen \affilnum 3 Abstract In this paper we show how The Free Energy Principle (FEP) can provide an explanation for why real-world networks deviate from scale-free behaviour, and how these characteristic deviations can emerge from constraints on information processing. We propose a minimal FEP model for node behaviour reveals three distinct regimes: when detection noise dominates, agents seek better information, reducing isolated agents compared to expectations from classical preferential attachment. In the optimal detection regime, super-linear growth emerges from compounded improvements in detection, belief, and action, which produce a preferred cluster scale. Finally, saturation effects occur as limits on the agent’s information processing capabilities prevent indefinite cluster growth. These regimes produce the knee-shaped degree distributions observed in real networks, explaining them as signatures of agents with optimal information processing under constraints. We show that agents evolving under FEP principles provides a mechanism for preferential attachment, connecting agent psychology with the macroscopic network features that underpin the structure of real-world networks. keywords Network formation; Information processing; Free Energy Principle; Mechanistic modelling; Preferential attachment; Scale-free networks; Biological constraints; Agent-based modelling; Collective behaviour; Complex systems † † runninghead: Williams and Chen † † affiliation: \affilnum 1Rinna KK, Tokyo, Japan \affilnum 2Independent Researcher \affilnum 3Microsoft Japan, Tokyo, Japan 1 Introduction Complex networks pervade natural and artificial systems, from cellular interactions to social relationships ( Barabási, 2016 ) . A common feature in these networks is the frequent emergence of power-law degree distributions, where the probability of a node having k connections follows P(k)\propto k^{-\gamma} ( Newman, 2003b ) . The canonical explanation for such distributions is preferential attachment: new nodes preferentially connect to highly connected existing nodes ( Barabási and Albert, 1999 ) . While this mechanism successfully describes network growth, it has faced criticism for lacking mechanistic underpinning ( Newman, 2018 ) . Some researchers argue that power-law distributions might arise from alternative processes ( Clauset et al., 2009 ) , suggesting the need for deeper theoretical foundations. Notably, many real-world networks exhibit systematic deviations from pure power-law behaviour, often showing a characteristic knee shape where moderate-degree nodes are overrepresented relative to both low and high degrees, when compared to the preferential attachment case. While various mechanisms have been proposed to explain these deviations, including aging effects ( Dorogovtsev and Mendes, 2002 ) and resource constraints ( Amaral et al., 2000 ) , an explanation based on information processing principles has not yet emerged. Understanding these deviations is crucial, as they may reflect constraints on how agents process and act on information in their environment. Parallel to network science, a different theoretical framework has emerged in cognitive science and neurobiology: the Free Energy Principle (FEP) ( Friston, 2010 ) . This principle posits that biological systems maintain their order by minimising a quantity called variational free energy, which minimises the surprise (negative log probability) of their sensory states ( Friston, 2013 ) . FEP provides a mathematical framework for understanding how agents perceive, learn, and act in their environment ( Ramstead et al., 2018 ) . Under this principle, biological systems actively sample their environment to confirm their internal models, a process known as active inference ( Friston et al., 2017 ) . Despite their separate development, these two frameworks—– network formation and free energy minimisation —–might share deeper connections. Both describe systems organising themselves through information-driven processes ( Lynn and Bassett, 2020a ; Kirchhoff et al., 2018 ) . While preferential attachment implicitly assumes agents can sense and respond to network structure ( Newman, 2003b ) , FEP explicitly describes how agents process and act on information ( Parr et al., 2020 ) . This suggests that preferential attachment might emerge as a results of more fundamental principles of information processing, an idea supported by recent work on information-theoretic approaches to network formation ( Ver Steeg, 2019 ) . Recent studies have shown that FEP can explain diverse biological phenomena, from cellular behaviour to neural organisation ( Friston, 2013 ; Constant et al., 2018 ) . The principle has been successful in bridging scales of organisation ( Ramstead et al., 2018 ) , although some researchers question its universal applicability ( Colombo and Wright, 2018 ) . Similarly, while preferential attachment has been observed in many systems ( Jeong et al., 2003b ) , from citation networks to protein interactions, its mechanistic origins remain debated ( Medo et al., 2011 ) . A key challenge in both fields is understanding how local, individual-level processes generate global, system-level patterns ( Anderson, 2018 ) . In network science, this manifests as the emergence of scale-free structures from individual attachment decisions ( Bianconi and Barabási, 2001 ) . In FEP, it appears as the generation of adaptive behaviour from local information processing ( Kirchhoff et al., 2018 ) . This parallel suggests an opportunity for theoretical synthesis, building on recent work connecting information theory and network formation ( Lynn and Bassett, 2020a ) . Our work bridges these domains by showing that FEP can provide a mechanistic foundation for network formation, revealing how deviations from pure preferential attachment can emerge from information processing constraints. This builds on previous attempts to ground network formation in cognitive processes ( Papadopoulos et al., 2012a ) while addressing criticisms of both preferential attachment’s lack of mechanism ( Newman, 2018 ) and FEP’s explanatory scope ( Colombo and Wright, 2018 ) . We show that agents following active inference can generate network structures with characteristic deviations from power-law distributions, reflecting regimes of information processing. The analysis focuses on a simple model where agents move in a one-dimensional space, sensing and responding to their neighbours according to FEP. This model was selected to be sufficiently complex to include all of the key behavioural processes, while staying simple enough to be analytically tractable. Through this model, we identify three distinct information processing regimes - noise-dominated, optimal detection, and saturation - that shape network formation. While similar models have been used to study collective behaviour ( Sumpter, 2010a ) , our work shows how these regimes can emerge from principles of uncertainty minimisation and create characteristic network structures. This synthesis has broad implications beyond network science. It suggests that network structures in biological and social systems reflect information processing constraints rather than arbitrary rules or external limitations. The identification of distinct information processing regimes provides new approaches to understanding and designing networks across domains, from social media platforms to networks of artificial agent systems. The framework offers practical insights for network design while deepening our understanding of how cognitive constraints shape collective structure. In the following sections we develop this theoretical framework. We begin by examining the constraints that biological agents face when processing information about their environment, showing how these create natural scales in network formation. We then derive how agents translate sensory information into movement decisions through free energy minimisation, leading to an attachment kernel that emerges from rational inference under uncertainty. This kernel reveals three distinct regimes of information processing—noise-dominated, optimal detection, and saturation—that shape network structure in characteristic ways. We show how the interplay between these regimes can explain commonly