Probabilistic Modeling of Latent Agentic Substructures in Deep Neural Networks

Su Hyeong Lee, Risi Kondor, Richard Ngo · 2025

Read Probabilistic Modeling of Latent Agentic Substructures in Deep Neural Networks by Su Hyeong Lee, Risi Kondor, Richard Ngo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06701.txt.

Introduction

Hidden sub-agents inside large neural networks are given a mathematical theory by Su Hyeong Lee and two co-authors in a 2025 paper on artificial intelligence foundations. The authors model an agent as a distribution over outcomes, scored by a log-score utility that rewards accurate beliefs. Combining agents means pooling their distributions.

Weighted logarithmic pooling, they prove, can strictly improve every member's welfare, something linear pooling cannot do, and binary outcome spaces rule it out too; three or more outcomes are needed. The framework supports recursive structure through cloning invariance, continuity, and openness. As an application to language models, they formalize a persona effect: prompting a benevolent character can summon an antagonistic counterpart. A strategy that first elicits then suppresses the counterpart cuts misalignment more than reinforcing the good persona alone. The theory aims to explain how sub-agents merge into coherent larger agents.

Catalog record

CREATORSu Hyeong Lee, Risi Kondor, Richard Ngo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE168 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-06701

Browse the full library · About the Tower of Babel