Active Inference, homeostatic regulation and adaptive behavioural control.

Pezzulo G, Rigoli F, Friston K. · 2015

Read Active Inference, homeostatic regulation and adaptive behavioural control. by Pezzulo G, Rigoli F, Friston K. (2015) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/europmc-pmc4779150.txt.

Introduction

Giovanni Pezzulo, Francesco Rigoli, and Karl Friston set out a unified account of behavior control in Progress in Neurobiology in 2015. Animals keep their inner state steady, a task called homeostasis, while also acting on the world. Psychology describes that control with three systems: reflex-like Pavlovian responses, habits, and plan-driven goal seeking. The three are usually modeled apart.

The authors fold all three into one frame called active inference, in which the brain acts to make the world match its expectations. Reflexes, habits, and plans then fall out as settings of a single system, tuned by how much weight each signal carries. The paper also sketches an evolutionary path from simple controllers to layered ones, and maps the scheme onto brain regions.

Catalog record

CREATORPezzulo G, Rigoli F, Friston K.
YEAR2015
COLLECTIONPAPERS
FORMATTXT
FILE SIZE122 KB
LICENSECC BY 4.0
RIGHTSCC BY
SOURCEEurope PMC

Source: https://europepmc.org/articles/PMC4779150

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Active Inference, homeostatic regulation and adaptive behavioural control By Giovanni Pezzulo, Francesco Rigoli, Karl Friston Published in: Progress in Neurobiology DOI: 10.1016/j.pneurobio.2015.09.001 ABSTRACT Highlights - •An Active Inference account of homeostatic regulation and behavioural control. - •Pavlovian, habitual and goal-directed behaviours explained with one scheme. - •A possible phylogenetic trajectory from simpler to hierarchical controllers. - •Precision-dependent processes regulate habitual and goal-directed behaviour. Abbreviations: CS, conditioned stimuli; US, unconditioned stimulus; PFC, prefrontal cortex; SMA, supplemental motor area; IC, inferotemporal cortex; AIC, anterior insular cortex; PMC/MC, premotor/motor cortex; ipoT, ipothalamus; ANS, autonomous nervous system; ACC, anterior cingulate cortex; PPC, posterior parietal cortex; VTA/SN, the dopaminergic ventral tegmental area and substantia nigra Keywords: Active Inference, Homeostatic regulation, Adaptive control, Model-based control, Model-free control, Pavlovian control INTRODUCTION "The animal must respond to changes in the environment in such a manner that its responsive activity is directed towards the preservation of its existence. This conclusion holds also if we consider the living organism in terms of physical and chemical science. Every material system can exist as an entity only so long as its internal forces, attraction, cohesion, etc., balance the external forces acting upon it. […] Being a definite circumscribed material system, it can only continue to exist so long as it is in continuous equilibrium with the forces external to it. Ivan Pavlov" Current associative learning theories in psychology and neuroscience assume that animal behaviour depends on multiple forms of control (i.e., Pavlovian, goal-directed, and habitual processes). These control schemes are based on associations between stimuli, actions and outcomes and are either innate or learned through experience. Our aim is to offer an integrative perspective by contextualizing classical formulations of adaptive behaviour within the Active Inference framework, which extends predictive coding from the domain of perception to cover action (Friston et al., 2009). Active Inference assumes that organisms act to fulfil prior expectations that encode the (evolutionarily) values of their states (e.g., having access to food). The mathematical foundation of Active Inference rests on the notion of free energy minimization, where the long-term average of free energy approximates the entropy of sensory states. Minimizing free energy (and therefore entropy) enables an organism to resist the dispersive effects of “external forces acting upon it” to ensure “it is in continuous equilibrium with the forces external to it” (Pavlov, 2010). Crucially, free energy can also be interpreted in a statistical sense as an approximation to Bayesian model evidence. This means Active Inference can be described normatively as maximizing (a negative free energy bound on the logarithm of) Bayesian model evidence. In other words, minimizing free energy reduces the discrepancy (e.g., prediction error) between sensations and their predictions. This discrepancy can be reduced by changing predictions – through perception – or by selectively sampling sensory inputs that were predicted – through action (Friston, 2010). The basic premise of this article is that the ontology of behavioural paradigms in associative learning can be seen as a successive contextualisation of more elemental sensorimotor constructs, within generative models of increasing hierarchical depth. This formulation explains how the primitive sensorimotor architecture of homeostatic control – of our early evolutionary ancestors – evolved towards goal-directed and prospective forms of control. This phylogenetic progression rests on the hierarchical elaboration of more primitive architectures (Cisek and Kalaska, 2010, Pezzulo and Castelfranchi, 2009). Furthermore, this hierarchical elaboration dissolves any apparent mechanistic distinction between the optimization processes that underlie different control or learning schemes, suggesting that they are all manifestations of Active Inference – under various contexts or conditions (Friston et al., 2009). This novel hypothesis contrasts with the standard (associative learning) view that the computations underlying different forms of behavioural control are fundamentally different and appeal to different (optimization) principles. We first review associative learning theories of homeostatic and behavioural control. We then offer an Active Inference formulation of adaptive behaviour that fulfils homoeostatic imperatives in increasingly sophisticated ways – building upon peripheral (somatic and autonomic) reflexes to explain simple Pavlovian and instrumental motor responses and, finally, complex goal-directed behaviour. A crucial aspect of this hierarchical perspective is that higher-level hierarchical representations contextualize lower levels and predict longer sequences of cues and responses. This is accommodated by predictions about transitions over increasingly protracted time scales (Friston, 2008, Pezzulo, 2012). HOMEOSTATIC REGULATION AND ADAPTIVE BEHAVIOURAL CONTROL IN ASSOCIATIVE LEARNING THEORIES Cannon (1929) first proposed that the evolutionary function of physiology and behaviour is to restrict homeostatic states to a physiologically tenable range. Homeostatic regulation therefore allows animals to maintain a “continuous equilibrium” between the internal milieu and environmental states, which (Pavlov, 2010) considered the raison d’être for our brains. The regulation of homeostatic states – or of allostatic processes (Sterling and Eyer, 1988) – has long been described in terms of control-theoretic and cybernetic mechanisms of error cancellation and feedback control (Ashby, 1947). At the neurobiological level, one hypothesis is that homeostatic control requires interoceptive signals that report current homeostatic levels (e.g., the current level of glucose in the blood) (Craig, 2010, Damasio and Carvalho, 2013, Gu et al., 2013). A hungry animal can be described as an animal whose homeostatic condition departs significantly from a level that is ‘good’ for survival. With some simplifications, the ‘good’ level of glucose is used as a reference for the controller to steer action (e.g., ingest food to restore the level of glucose). This form of autonomic regulation involves triggering autonomic reflexes that control bodily processes such as heart rate, blood pressure and peristalsis. Under some conditions, autonomic reflexes can restore homeostatic levels (e.g., a hyperthermic animal can cool down by perspiring). Although elemental, autonomic reflexes are not sufficient to fully support homoeostasis: to satisfy hunger or thirst, the animal must act on the external world. Early theories of homeostatic regulation focused on simple (e.g., approach or avoidance) constituents of an innate behavioural repertoire. However, higher animals learn to achieve their goals in complex and flexible ways that go well beyond approach and avoidance. To do this they must acquire an adequate behavioural repertoire and learn to select from currently available actions or sequences of action (policies). This is the main focus of associative learning theories in psychology and biology. A taxonomy of behavioural controllers in the associative learning literature 2.1. Contemporary associative learning theories assume that action selection depends on the continuous cooperation and competition of several behavioural controllers, which can be divided into “Pavlovian” and “instrumental” (goal-directed and habitual) (Balleine and Dickinson, 1998, Daw et al., 2005, Dayan, 2009). Behavioural reflexes represent a basic form of controller that constitutes the innate repertoire of most animals. This controller is rather limited, as it calls on a limited set of unconditioned responses (e.g., approaching and ingesting food or withdrawing from a painful stimulus) in response to a circumscribed and predefined class of stimuli (called unconditioned stimuli). Still, this controller is sufficient for most animals to survive, even without any experience-dependent learning. Pavlovian (classical) conditioning is the process by which an unconditioned stimulus (say, food), that triggers an unconditioned reflex (say, salivation), is repeatedly paired with a neutral stimulus (say, a bell). The pairing is thought to produce stimulus-stimulus associations (between the food and the bell). After several pairings, the (formerly) neutral stimulus is able – on its own

Read the complete text