The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Models
Cedric Flamant, Udaya Ghai, Kanna Shimizu · 2026
Read The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Models by Cedric Flamant, Udaya Ghai, Kanna Shimizu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11167.txt.
Introduction
Cedric Flamant, Udaya Ghai, and Kanna Shimizu present the Bicameral Model, which links two frozen language models with a trainable neural interface laid over their intermediate hidden states. Existing multi-model systems talk by generating text, pushing every exchange through the output vocab. This model asks whether two models can coordinate through a continuous, concurrent channel instead.
At each step both models move in lockstep. A primary model handles the task while an auxiliary model runs tools, solves constraints, or executes code. Each draws on the other's activations via a translation network plus a learned suppression gate, using about 1 percent of combined parameters. The gate learns its own communication protocol from task loss alone. Coupling two 0.5B models with a calculator lifts arithmetic accuracy from 36 to 96 percent, and coupling with a Z3 solver gives 1.7 times the baseline on ZebraLogic. Tool-augmented model builders get coordination without text serialization.
Catalog record
| CREATOR | Cedric Flamant, Udaya Ghai, Kanna Shimizu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 97420 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.11167
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Models Cedric Flamant, Udaya Ghai, Kanna Shimizu Abstract Existing multi-model and tool-augmented systems communicate by generating text, serializing every exchange through the output vocabulary. Can two pretrained language models instead coordinate through a continuous, concurrent channel? The Bicameral Model couples two frozen language models through a trainable neural interface on their intermediate hidden states. At every generation step, both models run in lockstep: a primary model drives the task while an auxiliary model operates tools, solves constraints, or executes code, with both conditioning on each other’s activations through a translation network and a learned suppression gate ( ∼ {\sim} 1% of combined parameters). The gate learns a selective communication protocol from task loss alone, without a prescribed format. We demonstrate the mechanism across three tool backends. On arithmetic, coupling two 0.5B models with a calculator raises accuracy from 36% to 96%. On logic grid puzzles, coupling two 0.6B models with a Z3 solver achieves 1.7 × 1.7\times the unaugmented baseline on ZebraLogic. On mathematical reasoning, coupling with a Python sandbox enables the auxiliary to generate problem-specific code from hidden-state signals alone, without ever seeing the problem text. 1 Introduction Today, when two pretrained language models must cooperate, they communicate by generating text, serializing every exchange through discrete tokens. This paper asks whether two frozen LLMs can instead coordinate through their intermediate hidden states, and what the learned coupling looks like when trained only from task loss. The Bicameral Model couples two language models through a trainable neural interface that provides a bridge for activations to flow between their intermediate hidden states. A primary model M_{p} orchestrates the task and produces the final response while an auxiliary model M_{a} maintains a concurrent stream with access to external tools. Both models advance in lockstep at every generation step, so the two streams condition on each other simultaneously and bidirectionally rather than through serialized text round-trips. The interface \phi sits between configurable transformer layers, translating activations between the models’ representation spaces and modulating the coupling strength through a learned suppression gate. Both models’ weights remain frozen; only \phi is trained via dual-target supervised fine-tuning. The primary model continues generating while the auxiliary operates tools in parallel, with results delivered through the neural channel rather than a text round-trip. We test the mechanism across three tool backends chosen to create a large, unambiguous capability gap between primary and auxiliary: a calculator, the Z3 constraint solver, and a Python sandbox. Tools are the device we use to produce this gap, not the mechanism itself; the same hidden-state coupling should function for any pair of complementary models. Coupling a 0.5B model with a calculator-equipped auxiliary raises arithmetic accuracy from 36.2% to 96.5%; coupling a 0.6B model with a Z3 solver achieves a 1.7\times improvement over the unaugmented model on ZebraLogic ( 24 ) ; and in the Python-tool configuration, the auxiliary generates correct problem-specific code from hidden-state signals alone, without ever seeing the problem text. Trained only on next-token loss, with no prescribed format for what the hidden-state channel should carry, the suppression gate converges to a selective, directionally structured communication protocol: forward coupling ( M_{p}\to M_{a} ) concentrates on task-relevant tokens such as numbers and operation keywords, while reverse coupling ( M_{a}\to M_{p} ) spikes when tool output becomes available or is recalled (§ 5 ). Moreover, this channel develops during training in a strict causal order, forward coupling first, then tool recall, then accuracy, consistent with the architecture’s dependency structure. Adapter-equivalent ablations confirm the gains come from the auxiliary model’s reasoning rather than from the trainable interface capacity. 2 Architecture Tools (Z3, Calc, Python) Neural Interface \phi Auxiliary M_{a} Primary M_{p} aux output aux input output token input token layers read write write read Frozen weights Trainable ( {\sim} 1% params) ; Activation flow Figure 1: Bicameral architecture. A frozen primary model M_{p} and a frozen auxiliary model M_{a} run in parallel, coupled through a lightweight trainable neural interface \phi . At each generation step, \phi reads hidden states from M_{p} at layer \ell_{r}^{p} and injects a perturbation into M_{a} at layer \ell_{w}^{a} (forward coupling), then reads from M_{a} at layer \ell_{r}^{a} and injects into M_{p} at layer \ell_{w}^{p} (reverse coupling). The auxiliary model’s output tokens are routed to external tools (Z3 solver, calculator); tool results are forced back as input tokens. Notation. We use m\in\{p,a\} to index the primary and auxiliary models. Each model is a pretrained transformer with L_{m} blocks; we write its residual-stream state before block \ell\in\{0,1,\ldots,L_{m}\} executes as \mathbf{h}_{m}^{(\ell)}(t)\in\mathbb{R}^{d_{m}} , where t is the token position. When \ell=0 this is the input embedding, and when \ell=L_{m} it is the state that feeds the final layer norm and unembedding; throughout the paper, “layer \ell ” refers to this residual-stream index. The tokens produced by each model at position t are denoted x_{t}^{p} and x_{t}^{a} (the primary and auxiliary tokens; sources described in § 2.3 ). The interface reads and writes at four layer indices (read and write for each direction): \ell_{r}^{p\to a},\ell_{w}^{p\to a},\ell_{r}^{a\to p},\ell_{w}^{a\to p} . Writing the forward direction for concreteness, the interface reads \mathbf{h}_{p}^{(\ell_{r}^{p\to a})}(t) from M_{p} and replaces \mathbf{h}_{a}^{(\ell_{w}^{p\to a})}(t) in M_{a} with a gated combination of the original state and a translated signal (§ 2.2 ); the reverse direction is symmetric. Our identity-coupling experiments constrain \ell_{r}^{p\to a}=\ell_{w}^{p\to a} and \ell_{r}^{a\to p}=\ell_{w}^{a\to p} (same-depth coupling, since identity requires d_{p}=d_{a} ); PullStandard experiments sweep over general four-tuples. 2.1 System overview The Bicameral Model consists of three components (Figure 1 ): a primary model M_{p} that generates user-facing text, an auxiliary model M_{a} that maintains a concurrent reasoning stream with access to external tools, and a neural interface that couples the two through their intermediate hidden states. Both M_{p} and M_{a} are pretrained transformer language models whose weights are frozen; only the parameters of the interface are trained. The interface consists of two directional coupling operators, \phi^{p\to a}:\mathbb{R}^{d_{p}}\times\mathbb{R}^{d_{a}}\to\mathbb{R}^{d_{a}} and \phi^{a\to p}:\mathbb{R}^{d_{a}}\times\mathbb{R}^{d_{p}}\to\mathbb{R}^{d_{p}} , each mapping a sender and receiver hidden state to an updated receiver state. Their concrete form (translation network plus learned suppression gate) is given in § 2.2 . A per-step forward pass then unfolds as (Figure 2 ): 1. Both models run from their input embeddings up to their forward-coupling layers, producing \mathbf{h}_{p}^{(\ell_{r}^{p\to a})}(t) and \mathbf{h}_{a}^{(\ell_{w}^{p\to a})}(t) . 2. Forward coupling updates M_{a} ’s hidden state at \ell_{w}^{p\to a} : \mathbf{h}_{a}^{(\ell_{w}^{p\to a})}(t)\;\leftarrow\;\phi^{p\to a}\!\left(\mathbf{h}_{p}^{(\ell_{r}^{p\to a})}(t),\;\mathbf{h}_{a}^{(\ell_{w}^{p\to a})}(t)\right). (1) 3. Both models continue through the middle layers to the reverse-coupling layers, producing \mathbf{h}_{a}^{(\ell_{r}^{a\to p})}(t) and \mathbf{h}_{p}^{(\ell_{w}^{a\to p})}(t) . 4. Reverse coupling updates M_{p} ’s hidden state at \ell_{w}^{a\to p} by symmetry: \mathbf{h}_{p}^{(\ell_{w}^{a\to p})}(t)\;\leftarrow\;\phi^{a\to p}\!\left(\mathbf{h}_{a}^{(\ell_{r}^{a\to p})}(t),\;\mathbf{h}_{p}^{(\ell_{w}^{a\to p})}(t)\right). (2) 5. Both models complete their remaining layers and produce a token (sampled or forced; see Section 2.3 ). Forward coupling lets the auxiliary observe the primary’s evolving representation; reverse coupling lets the auxiliary’s reasoning flow back and influence primary generation. We write \phi=\{\phi^{p\to a},\phi^{a\to p}\} for the interface as a whole; \phi is the only trainable component, and M_{p},M_{a} are frozen. 2.2 Neural interface We now unpack the coupling operators \phi^{p\to a} and \phi^{a\to p} introduced in § 2.1 . Each direction has two trainable components: a translation network f^{\cdot} that maps a sender hidden