Probabilistic Modeling of Latent Agentic Substructures in Deep Neural Networks
Su Hyeong Lee, Risi Kondor, Richard Ngo · 2025
Read Probabilistic Modeling of Latent Agentic Substructures in Deep Neural Networks by Su Hyeong Lee, Risi Kondor, Richard Ngo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06701.txt.
Introduction
Hidden sub-agents inside large neural networks are given a mathematical theory by Su Hyeong Lee and two co-authors in a 2025 paper on artificial intelligence foundations. The authors model an agent as a distribution over outcomes, scored by a log-score utility that rewards accurate beliefs. Combining agents means pooling their distributions.
Weighted logarithmic pooling, they prove, can strictly improve every member's welfare, something linear pooling cannot do, and binary outcome spaces rule it out too; three or more outcomes are needed. The framework supports recursive structure through cloning invariance, continuity, and openness. As an application to language models, they formalize a persona effect: prompting a benevolent character can summon an antagonistic counterpart. A strategy that first elicits then suppresses the counterpart cuts misalignment more than reinforcing the good persona alone. The theory aims to explain how sub-agents merge into coherent larger agents.
Catalog record
| CREATOR | Su Hyeong Lee, Risi Kondor, Richard Ngo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 168 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-06701