Controllable Information Production
Tristan Shah, Stas Tiomkin · 2026
Read Controllable Information Production by Tristan Shah, Stas Tiomkin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22449.txt.
Introduction
Tristan Shah and Stas Tiomkin reframe what drives an agent to act when no outside goal is set. Intrinsic motivation lets agents explore and learn on their own. Their 2026 preprint notes that most theory rests on information flow between picked variables, a choice left to the designer.
They propose a view based on information production, not transmission. The shift aims to free the drive from hand-picked variables. A cleaner base could yield agents that explore in richer, less biased ways. That matters for robots and game agents alike. The work is theory, with design in view.
Catalog record
| CREATOR | Tristan Shah, Stas Tiomkin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22449
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Controllable Information Production Tristan Shah, Stas Tiomkin Abstract Intrinsic Motivation (IM) is a paradigm for generating intelligent behavior without external utilities. The existing information-theoretic methods for IM are predominantly based on information transmission, which explicitly depends on the designer’s choice of which random variables engage in transmission. In this work, we introduce a novel IM principle, Controllable Information Production (CIP), that avoids both external utilities and designer-specified variables. We derive the CIP objective from Optimal Control, showing a connection between extrinsic and intrinsic behaviors. CIP appears as the gap between open-loop and closed-loop Kolmogorov–Sinai entropies, which simultaneously rewards the pursuit and regulation of chaos. We establish key theoretical properties of CIP and demonstrate its effectiveness on standard IM benchmarks. 1 Introduction Intrinsic Motivation (IM) is a biologically inspired paradigm for generating intelligent behavior without externally specified goals. A range of plausible information-theoretic objectives such as Empowerment ( Klyubin et al. 2005 ) , Curiosity ( Schmidhuber 2010 ) , Information Gain ( Bhavya et al. 2025 ) , Diversity (DIAYN) ( Eysenbach et al. 2019 ) , and Predictive Information have been proposed for IM and successfully applied to control problems including stabilization ( Tiomkin et al. 2024 ) , object manipulation, locomotion ( Karl et al. 2022 ) , and transportation. Despite these successes, several fundamental gaps remain. First, existing IM methods are predominantly based on mutual information, which implicitly assumes information transmission and requires designers to explicitly specify which random variables (RV) participate in information exchange ( Shannon 1948 ) . This introduces external knowledge, in the form of design choices, into otherwise goal-free formulations. Second, although empirically effective for solving Optimal Control (OC) tasks, the connection between existing IM objectives and OC theory remains unclear, limiting principled interpretation and leading to ad hoc heuristics that produce non-robust solutions ( Tiomkin et al. 2024 ) . Third, beyond plausibility arguments, rigorous derivations of IM objectives from intrinsic properties of the underlying dynamical system are lacking. We address these gaps by formally deriving an IM objective from OC, grounded in the native information production properties of dynamical systems, which are independent of design choices. The resulting objective emerges as a problem of information production rather than transmission, and is characterized by the gap between open-loop and closed-loop Kolmogorov–Sinai entropies (KSE), h_{\mbox{ks}} , ( Kolmogorov 1958 ) : Controllable Information Production \mathrm{CIP}=h_{\mbox{ks}}(f^{\mathbf{ol}})-h_{\mbox{ks}}(f^{\mathbf{cl}}), (1) where f^{\mathbf{ol}} and f^{\mathbf{cl}} are dynamics parametrized by open-loop and closed-loop actions, respectively. Intuitively, Controllable Information Production (CIP) rewards an agent for seeking out and suppressing controllable sources of chaos. The objective compares the rate at which the environment produces entropy when actions are injected blindly (open-loop) versus when they are selected adaptively in response to state feedback (closed-loop). A large gap indicates regimes where the environment rapidly produces entropy yet remains sensitive to control: open-loop behavior generates high entropy, while closed-loop actions suppress it. Hence, the agent is motivated to position itself near the edge of chaos , where small interventions have maximal impact on system evolution ( Tower 2024 ; Zhang et al. 2025 ; Bertschinger et al. 2004 ) . Crucially, purely chaotic or uncontrollable dynamics are not rewarded: when entropy cannot be influenced by the agent’s actions, open-loop and closed-loop entropy production coincide, yielding low CIP. As such, the objective values chaos that can be consumed through control. To understand how CIP differs fundamentally from existing IM methods, consider two questions: Q1. How is information transmitted through dynamics? Q2. How is information produced by dynamics? Existing IM methods based on Shannon mutual information respond to Q1. They assume information already exists somewhere (e.g., in actions, messages, or skills) and that dynamics propagate it to future observations or states. This requires choosing which variables are deemed important sources of information and which are relevant targets for transmission. As such, the resulting IM agent is endowed with a designer-specified information interface, in which the choice of source and target variables implicitly constrains how the agent perceives, acts, and exchanges information with the environment. CIP responds to Q2, which involves no designer choice but rather captures an intrinsic property of dynamics with only two possible modes of interaction between agent and environment: open-loop or closed-loop. Q2 does not privilege any variables as sources or targets of information. Instead, it has a clear interpretation grounded solely in dynamical systems properties and, as we show, emerges naturally from OC theory. Our main contributions are as follows: 1. We formulate a new approach to IM as a problem of information production rather than transmission, which emerges from OC theory. This closes the knowledge gap in understanding the connection between IM and OC, and opens doors for novel optimization schemes grounded in OC. 2. We establish key theoretical properties of CIP including its positiveness and its connection to the Riccati Equation: the fundamental equation in OC. The latter provides a new method for estimating KSE through Riccati Recursions. In this paper we focus on a new conceptual paradigm in IM and its theoretical properties, deferring the development of efficient optimization methods for large scale experiment to future work. We demonstrate the effectiveness of CIP on important benchmark problems in IM, including chaotic dynamics, while experiments with arbitrary large scale environments are beyond scope of the current work, and as such deferred to follow up work. Code to reproduce experiments and figures is provided at: https://anonymous.4open.science/r/controllable-information-production-E0C8/readme.md . 2 Existing Intrinsic Motivation Methods A wide variety of IM objectives have been proposed to generate intelligent behavior without external goals. These methods can be categorized based on which random variables (RVs) participate in information exchange. For example, Empowerment ( Klyubin et al. 2005 ; Salge et al. 2014 ) computes the channel capacity from an open-loop control sequence to the final state, effectively measuring an agent’s controllability. Curiosity-based IM methods such as Active Inference ( Parr et al. 2022 ; Friston 2010 ) and Surprise Maximization ( Schmidhuber 2010 ; Burda et al. 2019 ) measure the information gained by the agent’s model parameters upon observing states. Diversity-based methods such as DIAYN ( Eysenbach et al. 2019 ) measure information between latent skill representations and states. By selecting which variables transmit and receive information, an IM objective can be obtained that encodes an implicit bias about what the agent “should” consider to be important or privileged. In ( Tiomkin et al. 2024 ) , it is shown that an entire family of IM objectives can be derived from a single Proto Objective through different designer choices regarding which input RVs are “important” (equivalently, the information source) and which relevance RVs allow reconstruction of the input RVs after transmission through dynamics. This variety of IM objectives emphasizes the designer choice and subsequent bias in the resulting methods. IM methods have been shown to be effective in optimal control problems which otherwise require engineered reward in order to solve. For example, the challenging task of swinging up and balancing the chaotic double pendulum was shown to be solved with Empowerment ( Tiomkin et al. 2024 ) . Other work shows that an Active Inference approach leads to unsupervised object manipulation ( Ferraro et al. 2025 ) . Diversity-based methods such as DIAYN have been used