Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models
Rituraj Sharma, Tu Vu · 2026
Read Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models by Rituraj Sharma, Tu Vu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24898.txt.
Introduction
Rituraj Sharma and Tu Vu expose a blind spot in how looped language models are trained. Looped models reuse one block many times, turning hidden states into a running state. Each loop's state is decoded into a prediction and fed onward. The authors ask which state variables the per loop loss actually controls. Their answer: only the variables the readout can see.
Common readouts like RMSNorm are blind to radial scale, so the loss cannot feel it. The recurrent loop, though, keeps carrying and updating exactly that scale. In looped transformers of 44 and 129 million parameters, hidden state norms balloon into the thousands despite per loop training. Scale visible readouts and explicit norm penalties hold norms in the tens instead. Removing scale from the recurrence is the complementary fix. Scale controlled variants then reach lower perplexity at matched depth. The design rule: dense supervision trains exits, not loop scale.
Catalog record
| CREATOR | Rituraj Sharma, Tu Vu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.24898
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dense Supervision Is Not Enough: The Readout Blind Spot in Looped Language Models Rituraj Sharma, Tu Vu Abstract Looped language models turn hidden states into runtime state: each state is decoded for prediction and fed back into future computation. This creates a basic supervision question: which state variables does cross-entropy actually control? We show that dense per-loop cross-entropy controls the variables exposed by the readout, not every variable active in the recurrent transition. Hidden-state scale gives a concrete failure mode. Scale-invariant readouts such as RMSNorm and LayerNorm hide radial scale from the immediate cross-entropy loss, while pre-norm residual recurrence continues to carry and update that same scale. Thus per-loop loss can make early exits usable without controlling recurrent scale. In 44M and 129M looped transformers without inter-loop normalization, per-loop cross-entropy through RMSNorm readouts still drives final hidden-state norms into the thousands or tens of thousands. Scale-visible readouts and explicit norm penalties keep norms in the tens, and scale-removing recurrence is the complementary architectural fix. The resulting design rule is simple: dense supervision trains exits; recurrent scale control requires either making scale visible to a loss or removing it from the loop. Consistent with this rule, scale-controlled variants achieve lower perplexity at matched inference-depth operating points in our variable-depth benchmarks. 1 Introduction Looped language models reuse the same block across recurrent depth. 1 1 1 Code: https://github.com/Rituraj003/readout-blind-spot . This makes test-time depth a natural compute knob: stop after a few loops when the prediction is easy, or continue iterating when additional computation helps. Universal Transformers ( Dehghani et al. 2019 ) , Huginn ( Geiping et al. 2025 ) , Ouro ( Zhu et al. 2025 ) , LoopFormer ( Jeddi et al. 2026 ) , and Elastic Looped Transformer ( Goyal et al. 2026 ) all explore versions of this idea. Looping changes the role of a hidden state. In a fixed-depth transformer, a late hidden state is mostly an interface to the output head. In a looped model, the same hidden state is also runtime state: it is decoded for prediction and then reused as input to future computation. This makes a simple question unavoidable: which parts of the recurrent state does the supervised loss actually control? Dense supervision is the natural answer. Apply cross-entropy (CE) at every loop, and each intermediate state receives direct prediction supervision. This indeed trains the prediction interfaces: early exits become usable. But dense CE does not automatically control every variable carried by the recurrent state. It controls what the readout makes visible. If an active recurrent variable is hidden by every supervised readout, the model can be trained at every loop while that variable remains underconstrained. We study this readout blind spot for hidden-state scale. In pre-norm residual loops, scale is active : the residual skip carries it forward and the learned update can change it. With an RMSNorm or LayerNorm readout, scale is locally invisible : multiplying the hidden state by a positive scalar leaves the readout unchanged up to the normalizer epsilon, so the immediate CE loss has essentially no radial derivative. The result is a visibility–activity mismatch. In our looped transformers, per-loop CE through normalized readouts still allows hidden-state norms to drift by orders of magnitude when scale remains active in the recurrent path. The model is supervised at every loop, yet every supervised interface hides the variable that is drifting. This yields the central design rule of the paper: Per-loop cross-entropy trains exit usability; recurrent scale control requires either making hidden-state scale visible to a loss or removing it from the recurrent path. Figure 1 : Exit training and scale control are orthogonal in the core 2 \times 2. The x-axis measures early-exit failure, the cross-entropy gap between the first and fourth recurrent loops, \mathrm{CE}(K{=}1)-\mathrm{CE}(K{=}4) , where K is the inference loop count; the axis uses a symlog scale. The y-axis measures final-loop norm drift, with H_{K} denoting the hidden state after K loops. Lower is better on both axes. Per-loop CE moves models left by making intermediate exits usable, while scale-visible readouts move models down by controlling recurrent scale. The desired region requires both interventions; later norm-penalty and final-only-normalization controls show that the effect is not merely a raw-readout artifact. We support this rule with a compact mechanism and controlled ablations. The mechanism has two components. Scale-invariant readouts such as RMSNorm or LayerNorm produce zero immediate radial cross-entropy signal, while pre-norm residual recurrence continues to preserve and update scale. Future-loop losses can still carry indirect radial information, but that signal weakens at large hidden-state norms. The experiments then test the interventions suggested by this mechanism: exposing scale through raw readouts or explicit norm penalties, or removing scale through normalization inside the recurrent path. Our contributions are: 1. We introduce the readout blind spot as a visibility–activity mismatch: dense CE controls variables exposed by the readout, not every variable active in recurrence. In 44M- and 129M-parameter looped language models where hidden-state scale remains active in the recurrent path, per-loop cross-entropy through normalized readouts still produces large norm drift. 2. We instantiate the blind spot for hidden-state scale. Normalized readouts hide scale from the immediate cross-entropy loss, while pre-norm residual recurrence continues to propagate it. Raw readouts, norm penalties, final-only normalization, gradient diagnostics, and recurrent-scale clamps all support this interpretation. 3. We show that scale control improves the variable-depth perplexity–compute frontier. Scale-controlled per-loop models maintain usable exits while achieving lower perplexity than the K -invariant RMSNorm baseline at matched inference-depth operating points in our benchmarks. 2 Mechanism: Visible Losses Control Visible Coordinates The failure mode is a mismatch between what the recurrent state carries and what the loss can see. At loop k , a hidden state H_{k} is both a prediction interface and the input to future computation: H_{k+1}=F_{\theta}(H_{k}),\qquad z_{k}=W_{\mathrm{out}}r(H_{k}),\qquad\mathcal{L}_{k}=\mathrm{CE}(z_{k},y). (1) Here F_{\theta} is the recurrent transition, r is the readout, W_{\mathrm{out}} maps readout states to logits z_{k} , y is the next-token target, and \mathcal{L}_{k} is the loop- k cross-entropy loss. A degree of freedom is immediately visible if the local readout loss has a derivative in that direction, and active if the transition updates it and carries it forward. Per-loop CE makes each loop predictive, but it does not make every active degree of freedom visible. We study the scale direction. Let H=su , where s=\|H\|_{\mathrm{rms}} , \|u\|_{\mathrm{rms}}=1 , and \|v\|_{\mathrm{rms}}^{2}=d^{-1}\|v\|_{2}^{2} for hidden-vector dimension d . A raw readout preserves scale, r_{\mathrm{raw}}(su)=su . A normalized readout removes it, r_{\mathrm{norm}}(su)=u up to the usual normalization epsilon. Figure 2 sketches the two paths the same state takes. Figure 2 : Visibility–activity mismatch. The same hidden state H_{k}=s_{k}u_{k} serves as both a prediction interface and the input to the next recurrent step. Along the readout path, output normalization removes scale, so the immediate CE loss has approximately zero radial gradient. Along the recurrent path, the pre-norm residual update still carries the skip state H_{k} , so scale remains active and can drift through the recurrence. Here \mathrm{Norm} denotes the pre-sublayer normalizer and B denotes the residual branch applied to the normalized state. Lemma 1 (Visibility: scale-invariant readouts remove the immediate radial CE signal) . If r(\alpha H)=r(H) for all \alpha>0 and \mathcal{L}(H)=\mathrm{CE}(W_{\mathrm{out}}r(H),y) is differentiable at H\neq 0 , then \langle\nabla_{H}\mathcal{L}(H),H\rangle=0.