Rethinking State Tracking in Recurrent Models Through Error Control Dynamics

Jiwan Chung, Heechan Choi, Seon Joo Kim · 2026

Read Rethinking State Tracking in Recurrent Models Through Error Control Dynamics by Jiwan Chung, Heechan Choi, Seon Joo Kim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07755.txt.

Description

Jiwan Chung and two co-authors argue that the theory of state tracking in recurrent networks has asked the wrong question. Research focused on expressivity, whether an architecture could in principle realize a set of symbolic transition rules. Equally decisive, they claim, is error control: how hidden-state errors drift along the directions that separate one symbolic state from another.

They prove that affine recurrent networks, a class covering state-space models and linear attention, cannot correct errors along state-separating directions once they preserve state representations. Practical trackers therefore learn only finite-horizon solutions, readable while accumulated spread stays small against the initial separation. On group state-tracking tasks the predicted breakdown point, where a distinguishability ratio crosses the decoder's threshold, matched where accuracy actually collapsed.

Catalog record

CREATORJiwan Chung, Heechan Choi, Seon Joo Kim
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07755

Browse the full library · About the Tower of Babel