Silent Collapse:Trajectory-Level Precursors in Recursive Learning
Zhipeng Zhang · 2026
Read Silent Collapse:Trajectory-Level Precursors in Recursive Learning by Zhipeng Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14588.txt.
Introduction
Zhang studies recursive learning, where models train on data produced by earlier versions of themselves, and names a failure he calls silent collapse. Internal distributions such as predictive entropy and tail coverage are claimed to contract even while loss and accuracy look stable, with three trajectory precursors appearing generations ahead of visible decline. His Monitor-Trust-Regulator loop watches those signals and adjusts learning intensity to warn of and prevent collapse.
Catalog record
| CREATOR | Zhipeng Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30961 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.14588
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Silent Collapse:Trajectory-Level Precursors in Recursive Learning Zhipeng Zhang Abstract Recursive learning — where models iteratively learn from feedback whose reliability cannot be fully verified locally — is increasingly common in modern AI systems. However, standard endpoint validation metrics such as loss, perplexity, and accuracy may fail to reflect the true internal dynamical state of the learning system. Here we identify a phenomenon we call silent collapse : a regime in recursive learning where latent geometric contraction and progressive loss of recoverability emerge before visible endpoint degradation. We show that silent collapse follows an ordered dynamical sequence rather than an abrupt failure event. Across independent recursive trajectories, hidden contraction — measured through anchor entropy and representation dynamics — consistently precedes visible collapse. This hidden regime is accompanied by progressively weakened recoverability: later-stage checkpoints require substantially stronger intervention to restore stable learning dynamics under identical weak recovery conditions. More importantly, we find that endpoint observables may become non-identifiable with respect to latent internal state. Checkpoints with nearly identical validation perplexity can occupy substantially separated latent geometry states and exhibit strongly different recovery dynamics under the same intervention protocol. Trajectory-level geometry analysis further reveals that recursive open-loop learning follows a continuous latent-state evolution while observable validation metrics become non-monotonic, producing self-intersections in observable space without corresponding intersections in latent geometry space. Based on these trajectory-level precursors, we further explore an adaptive regulation strategy (Monitor–Trust–Regulator, MTR) that modulates recursive pressure using internal trajectory signals. Entropy-regulated trajectories remain substantially closer to an approximately injective mapping between latent geometry and observable behavior. Together, our results show that observable equivalence does not imply latent-state equivalence in recursive learning systems and that trajectory-level observables provide critical information unavailable from standard endpoint evaluation alone. Corresponding author: Zhipeng Zhang ( [email protected] ) Keywords: silent collapse, trajectory-level precursors, recursive learning, model collapse, metacognitive regulation 1 Introduction Modern artificial intelligence systems increasingly learn iteratively from feedback whose reliability cannot be fully verified from individual observations alone. A language model fine-tunes on synthetic text generated by an earlier version of itself [ 8 , 2 ] . A reinforcement learning agent improves using rewards that may be systematically biased [ 4 , 1 ] . A self-supervised classifier refines its predictions using pseudo-labels that become increasingly confident — and increasingly wrong. More broadly, similar feedback structures arise in AI-mediated scientific and decision-making pipelines, including astronomy, medical prediction systems, climate inference, and autonomous discovery workflows, where inferred quantities may recursively influence future interpretation, selection, or optimization. In all these cases, each individual training or inference signal may appear locally plausible, yet the overall learning trajectory may drift toward a degenerate state. We define silent collapse as a dynamical regime in which iterative learning systems undergo progressive internal degeneration — contraction of predictive entropy and representational dynamics, and, in some settings, erosion of support statistics — while endpoint validation metrics remain stable or improve. Crucially, this hidden degeneration is accompanied by a progressive loss of recoverability: the deeper the system enters silent collapse, the harder it becomes to restore healthy dynamics with a fixed budget of real data. We identify trajectory-level precursors that reliably emerge before visible degradation, including robust contraction of anchor entropy and representation drift, and (in some settings) erosion of support statistics. These precursors manifest before any detectable change in validation metrics, providing a practical early warning window. A related phenomenon, model collapse , was recently demonstrated by 9 in recursively trained generative models [ 5 ] . Their work showed that models inevitably lose tail information and converge to delta functions when trained on their own outputs. However, their analysis left two critically important questions unanswered: is collapse predictable before standard metrics degrade, and can it be prevented without access to pristine real data? Here we show that recursive degradation is both predictable before visible failure and partially preventable without continued access to pristine real data. Beyond providing a method for early warning and intervention, our results reveal a more fundamental limitation: standard validation metrics alone are insufficient to determine whether iterative learning dynamics remain stable. Thus, silent collapse reframes recursive degradation as an observability problem: the health of an iterative learning system cannot be inferred from endpoint validation metrics alone. Based on the trajectory-level precursors we identify, we propose the MTR (Monitor–Trust–Regulator) framework, illustrated in Figure 3 . MTR operates a secondary trajectory-level control loop alongside the base learner: it monitors trajectory statistics, aggregates them over a slower timescale into a trust estimate \tau , and adaptively modulates the effective learning intensity. MTR requires no continued access to pristine real data beyond a fixed anchor set — only a small, fixed set of anchor examples drawn from the initial checkpoint. We validate MTR on recursive language modeling (TinyStories) and recursive pseudo-labeling on CIFAR-10 under a stress-test regime. In both settings, MTR detects collapse precursors before visible degradation emerges and can stabilize learning. Our contribution is not the observation that recursive models can collapse, but the identification of trajectory-level precursors that reveal collapse before visible failure and enable intervention. This work reframes the study of iterative learning degradation in terms of trajectory-level observability of recursive collapse. MTR serves as one demonstration that the precursor regime is actionable. 2 What is silent collapse? Definition. We define silent collapse as a dynamical regime in which iterative learning systems undergo progressive internal degeneration while endpoint validation metrics remain stable or improve. Internal degeneration refers to contraction of predictive entropy, loss of representational diversity, and in some settings erosion of support statistics, coupled with a progressive loss of recoverability. Endpoint validation metrics include loss, perplexity, and accuracy. Silent collapse therefore reflects a decoupling between visible predictive performance and internal dynamical health. Trajectory-Level Precursors. We define trajectory-level precursors as reproducible changes in internal trajectory statistics that systematically emerge before visible degradation in endpoint performance metrics. In this work, we focus on two robust precursors: (1) contraction of anchor entropy, and (2) contraction of representation drift. (In some settings, erosion of support statistics provides additional evidence.) These precursors are measurable, exhibit a consistent onset order relative to visible collapse, and are coupled with a progressive loss of recoverability. Early and late silent collapse. Silent collapse follows a consistent dynamical sequence. Early silent collapse corresponds to the hidden contraction precursor regime: predictive entropy and representation drift contract (and in some settings, support statistics contract) while endpoint validation metrics remain stable or improve. In this regime, the system is already degrading but conventional evaluation remains deceptively stable. Late silent collapse corresponds to visible degradation: perplexity explosion, accuracy loss, calibration collapse, or support erosion become detectable. This progression defines a critical early-warning window during which intervention can substantially mitigate later collapse. Sources of silent collapse. Silent collapse arises from three coupled sources: Feedback reliability error. The feedback used for learning or inference cannot be locally verified. Examples include synthetic text, pseudo-labels, biased reward signals [ 7 ] , and model-derived scientific quantities used in downstream inference pipelines. Each individual signal may appear locally plausible, yet the distribution from which it is drawn may be systematically distorted. Trajectory amplification error. Small reliable-looking biases are repeatedly reinforced across iterations. A mild concentration of probability mass in one generation becomes amplified in the