Rethinking State Tracking in Recurrent Models Through Error Control Dynamics
Jiwan Chung, Heechan Choi, Seon Joo Kim · 2026
Read Rethinking State Tracking in Recurrent Models Through Error Control Dynamics by Jiwan Chung, Heechan Choi, Seon Joo Kim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07755.txt.
Description
Jiwan Chung and two co-authors argue that the theory of state tracking in recurrent networks has asked the wrong question. Research focused on expressivity, whether an architecture could in principle realize a set of symbolic transition rules. Equally decisive, they claim, is error control: how hidden-state errors drift along the directions that separate one symbolic state from another.
They prove that affine recurrent networks, a class covering state-space models and linear attention, cannot correct errors along state-separating directions once they preserve state representations. Practical trackers therefore learn only finite-horizon solutions, readable while accumulated spread stays small against the initial separation. On group state-tracking tasks the predicted breakdown point, where a distinguishability ratio crosses the decoder's threshold, matched where accuracy actually collapsed.
Catalog record
| CREATOR | Jiwan Chung, Heechan Choi, Seon Joo Kim |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07755