Mechanism-Driven Monitors for Preemptive Detection of LLM Training Instability
Ruixuan Huang, Yipei Wang, Wenyi Fang, Hantao Huang, Yifan Huang, Ansheng You, Zhenxing Zhang, Shuai Wang, Fan Wu, Yang Zheng, [4pt] HKUST · 2026
Read Mechanism-Driven Monitors for Preemptive Detection of LLM Training Instability by Ruixuan Huang, Yipei Wang, Wenyi Fang, Hantao Huang, Yifan Huang, Ansheng You, Zhenxing Zhang, Shuai Wang, Fan Wu, Yang Zheng, [4pt] HKUST (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28116.txt.
Introduction
Training a frontier language model burns fleets of accelerators for weeks. A fault can destabilise training silently, with loss and gradient norms looking normal for thousands of steps. Ruixuan Huang and nine colleagues build mechanism-driven monitors in 2026 to catch such failures early, each derived from what a critical module is supposed to compute. Monitors cover two modules, low-precision flash attention and mixture-of-experts routers.
For attention, the spectral entropy of a QK decomposition is tracked. Its first-order term turns abnormal before the loss collapses. Fault-injection experiments cover low-precision attention, large learning rate, and combined faults. Spectral collapse preceded loss divergence by about seventeen thousand steps. Singular-spectrum collapse gave about nine thousand steps of warning. Router indicators, built from per-token entropy and weight similarity, respond to router faults instead. Different failures leave distinct signatures. Detection thousands of steps early can save compute.
Catalog record
| CREATOR | Ruixuan Huang, Yipei Wang, Wenyi Fang, Hantao Huang, Yifan Huang, Ansheng You, Zhenxing Zhang, Shuai Wang, Fan Wu, Yang Zheng, [4pt] HKUST |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.28116
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Mechanism-Driven Monitors for Preemptive Detection of LLM Training Instability Ruixuan Huang, Yipei Wang, Wenyi Fang, Hantao Huang, Yifan Huang, Ansheng You, Zhenxing Zhang, Shuai Wang, Fan Wu, Yang Zheng, [4pt] HKUST Abstract Frontier large language model training consumes massive accelerator fleets and long wall-clock computation, making stability failures costly when they occur. After a numerical or a hyperparameter fault has already destabilized the training dynamics, it may continue for thousands of steps while loss and gradient norms still appear normal. We study mechanism-driven detection of training instability by deriving internal monitors from the functional role of each critical module and from the earliest computational sites where failures are expected to produce measurable signatures. For low-precision flash attention, we monitor the spectral entropy of a QK bilinear decomposition, whose first-order term becomes abnormal before the loss fully collapses. For MoE routers, we derive indicators from their role in expert selection. Our fault-injection experiments on low-precision attention, large learning-rate, and combined faults show that these signals provide distinct signatures for different failures, triggering thousands of steps before loss divergence. 1 Introduction Frontier large language model (LLM) training typically occupies thousands of accelerators for weeks to months ( 7 ; 32 ) . Parameter counts now reach hundreds of billions to over a trillion ( 38 ; 23 ; 10 ) ; pre-training corpora span tens of trillions of tokens ( 16 ; 26 ; 29 ) . DeepSeek-V3 gives an explicit cost accounting, where 14.8T pre-training tokens required 2.788M H800 GPU-hours and a reported $5.576M in direct rental cost, excluding prior research and ablation experiments ( 9 ) . At this scale, training stability has become an engineering concern of the training system. GLM-130B describes unexpected 100B-scale training challenges, especially loss spikes and divergence ( 41 ) ; DeepSeek-V3 highlights FP8 mixed-precision training and explicitly reports no irrecoverable loss spikes or rollbacks ( 9 ) ; and Kimi K2 introduces MuonClip with QK-clip to address training instability and reports 15.5T-token pre-training with zero loss spike ( 23 ) . The risk of training instability often comes from two sources. The first is numerical precision error. For example, flash attention (FA) exhibits substantially larger BF16 numeric deviation than baseline attention in isolated forward passes ( 17 ) , and low-precision FA can corrupt weight updates through biased rounding errors and gradually derail training dynamics ( 28 ) . The second is hyperparameter interaction, such as the coupling among global batch size (GBS), learning rate schedule and MoE auxiliary loss. However, before the global symptoms appear, a training run may already have entered an unstable state in its weights or optimizers, while training silently continues for thousands of steps before the symptoms become visible. Exhaustive ablation only increases these sunk costs. A useful monitor should therefore identify which subsystem has been destabilized before loss divergence appears. Current training stability monitoring mainly relies on global training curves and symptom-level indicators. Loss, gradient norms, and weight norms are the most delayed indicators. Once a loss spike or divergence appears, the fault may already have been written into weights or optimizer state. Attention entropy, maximum attention logit, and spectral indicators further characterize attention-side instability symptoms ( 42 ; 33 ; 17 ; 23 ) . Edge-of-stability analysis explains high-level loss dynamics, but does not identify which module failed first ( 8 ) . Max-logit signals are difficult to expose in production FA because they require kernel modification and recomputation ( 23 ) . Hessian or curvature diagnostics can provide finer geometric information, but are too expensive to run as routine online checks at frontier scale ( 39 ; 22 ) . (a) LM loss (b) |W_{QKV}|_{F} (c) Gradient norm (d) \Delta W_{QKV} spectrum Figure 1: Monitoring signals over the first 25,000 steps of a training run. (a)–(c) are standard symptom-level indicators: LM loss, QKV weight norm, and gradient norm. (d) shows an internal update monitor used in this paper. In (b) and (d), the solid curve is the layer-wise average and the shaded band spans the 10th–90th percentile across all layers. Our core idea is mechanism-driven monitoring. For each critical module, we ask what is this module supposed to compute, and where would a malfunction first leave an attributable trace. We apply this principle to two modules. For low-precision FA, we monitor weight updates, where low-precision backward errors first enter the model state (Section 3 ). We decompose the two-snapshot increment of the QK operator and monitor the spectral entropy of \Delta W . Figure 1 shows an example under low-precision FA training, where the \Delta W spectrum collapses thousands of steps earlier than loss, gradient norms and weight norms. For MoE routers, the intended computation is discriminative and non-collapsed expert selection (Section 4 ). Therefore, we monitor router weight similarity and centered conditioning as weight indicators that characterize whether the effective expert-selection axes become redundant. For the behavior, we monitor per-token routing entropy. It reads the full softmax distribution and can therefore capture the collapse of routing behavior before downstream discrete quantities such as top- k counts, capacity overflow, or load-balance statistics change. We further analyze how learning rate and GBS interact through stable-winner reinforcement. A larger learning rate amplifies coherent margin growth, while a smaller batch size increases margin noise; both can reduce router entropy and accelerate expert-use collapse. Our fault-monitoring experiments demonstrate the separate roles of the two monitor families, and combined faults inherit both signatures without obscuring their attribution. 2 Related Work Training-stability monitors. Existing work has proposed monitors to detect or mitigate training instability. On the attention side, max-logit clipping, introduced for ViT-22B ( 11 ) and studied through small-scale Transformer proxies ( 35 ) , catches softmax explosion directly. Kimi K2 adds QK-clipping and per-head MuonClip ( 23 ) . Other approaches target attention-entropy collapse via \sigma Reparam ( 42 ) , loss spikes via spectral-norm control ( 33 ) , or Flash-Attention output distributions ( 17 ) . On the MoE router side, LongCat-Flash monitors the average cosine similarity among expert router weights and the gradient-norm ratio between the load-balancing objective and the language-modeling objective on average expert probabilities ( 26 ) . Attention Circuit Analyses. The attention QK-circuit has been analyzed primarily as a static object. 2 characterize attention localization through the eigenspectrum variance of W_{q}^{\top}W_{k} , and 27 examine singular-vector correspondence on the QK kernel for vision transformers. Researches show that attention maps and QK kernels can exhibit strong low-rank structure. 3 study the rank-deficiency bottleneck of W_{q}W_{k}^{\top} at small d_{k} , while 12 prove doubly-exponential rank collapse in pure self-attention with depth. Recent works also show that weight updates contain informative low-rank structure. LoRA ( 18 ) , GaLore ( 43 ) , and the Muon optimizer family ( 25 ) exploit low-rank or spectral structure in updates for parameter-efficient adaptation or optimization, and 40 survey spectral evolution of weights as a window onto training dynamics. Mechanistically, 28 identify low-precision FA failure as biased rounding accumulating into similar low-rank update directions. These works motivate using \Delta W itself as an analysis object. 3 Attention Updates Monitoring Flash Attention ( 17 ) fuses the softmax-scaled QK^{\top} computation in on-chip memory and not materializes the full N\times N logit