Model-Centric Diagnostics: A Framework for Internal State Readouts
Fangzheng Wu, Brian Summa · 2026
Read Model-Centric Diagnostics: A Framework for Internal State Readouts by Fangzheng Wu, Brian Summa (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16874.txt.
Introduction
Fangzheng Wu and Brian Summa build a check kit for the guts of neural nets. A net's train state hides in its weights and steps. Gauges can read that state from in-house signs. Signs span slope size, faith, doubt, and margin. Their 2026 preprint treats train state as one hidden root. Each sign is then a view of that root.
The views are set next to fixed anchor marks. That set lets signs from unlike nets share one scale. An early form of the kit picked good save points. The full kit may flag bad runs while they train. It could save power and time in big model labs.
Catalog record
| CREATOR | Fangzheng Wu, Brian Summa |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-16874
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Model-Centric Diagnostics: A Framework for Internal State Readouts Fangzheng Wu, Brian Summa Abstract We present a model-centric diagnostic framework that treats training state as a latent variable and unifies a family of internal readouts—head-gradient norms, confidence, entropy, margin, and related signals—as anchor-relative projections of that state. A preliminary version of this work introduced a head-gradient probe for checkpoint selection. In this version, we focus on the unifying perspective and structural diagnostics; full algorithmic details, theoretical analysis, and experimental validation will appear in a forthcoming paper. We outline the conceptual scaffold: any prediction head induces a local loss landscape whose geometry (gradient magnitude, curvature, sharpness) reflects how well the upstream features are aligned with the task. Different readout choices—gradient norms, softmax entropy, predictive margin—correspond to different projections of this geometry, each with complementary strengths. The framework suggests that checkpoint selection, early stopping, and lightweight architecture pre-screening can all be viewed as querying the same underlying state through different lenses. Illustrative experiments on ImageNet classification and COCO detection/segmentation hint at the practical potential; rigorous benchmarks and ablations are deferred to the full paper. 1 1 footnotetext: Disclaimer: This manuscript presents a conceptual framework and differs substantially from a forthcoming submission that provides full algorithmic details, theoretical analysis, and comprehensive experimental validation. The preliminary results shown here are illustrative only. 1 Introduction Modern training loops heavily lean on held-out validation sets to decide when to stop and which checkpoint or model to keep. This practice costs extra compute for periodic evaluation, can be impossible when labels are scarce or private, and may be unreliable under distribution shift. We revisit a minimalistic question: can one predict model quality using internal state readouts—signals derived from the model itself—without any validation data at selection time? A unifying perspective. We argue that seemingly disparate diagnostics—gradient norms, softmax confidence, entropy, margin, and curvature-based measures—can all be understood as anchor-relative projections of a latent training state. When features become more linearly separable, the classifier head operates in a flatter local landscape. This geometry manifests through multiple observable proxies: smaller head gradients, higher confidence, lower entropy, and wider margins. Each readout offers a different “lens” into the same underlying state. Framework contributions. This paper sketches a model-centric diagnostic framework with the following elements: • Training state as latent variable. We treat the quality of learned representations as a hidden variable that causally influences multiple observable readouts. • Anchor-relative projections. Each diagnostic (gradient norm, confidence, etc.) projects the latent state onto a scalar or low-dimensional summary, analogous to different projections of a high-dimensional object. • Task-agnostic structure. The same conceptual scaffold applies to classification, detection, segmentation, and generative modeling, though the optimal readout may differ by task. Scope of this version. A preliminary version of this work focused on one specific instantiation: the head-gradient norm as a checkpoint selection signal. Here, we step back to emphasize the unifying framework rather than any single method. We include illustrative experiments to motivate the perspective, but defer rigorous benchmarks, ablations, and theoretical analysis to a forthcoming full paper. Why this matters. If internal readouts reliably track model quality, practitioners gain several benefits: validation-free early stopping, lightweight pre-screening of architectures, and monitoring in settings where labels are unavailable or expensive. The framework perspective suggests that these applications share a common foundation and that progress on one readout informs the others. 2 Related Work A large line of work aims to rank architectures or checkpoints without (or before) full training using “zero-cost” or training-free indicators computed from a few forward/backward passes at (near) initialization. Representative examples include linear-region/Zen-based measures ( 22 ) , trainability indicators (e.g., TE-NAS 5 , NWOT 29 , and gradient-flow proxies such as SynFlow ( 36 ) . These methods are chiefly designed for architecture search : they compare many untrained candidates in a fixed search space and correlate the proxy with the eventual trained accuracy. Our setting is complementary: we operate on running checkpoints or off-the-shelf pre-trained models , and the goal is early stopping / model or checkpoint selection rather than universal ranking inside a NAS space. Several NAS papers ( 1 ; 38 ) report a baseline termed grad-norm that measures the \ell_{p} -norm of the full-parameter gradient \|\nabla_{\theta}\mathcal{L}\| near initialization to score architectures without training (see the comparisons in ZenNAS ( 22 ) ). Our probe is different in both where and what it measures: (i) we evaluate at trained or training checkpoints rather than at random initialization; (ii) we backpropagate through the classifier head only and detach features , producing a signal that directly reflects the linear separability of current features. Empirically, this one-batch, head-only signal is competitive while being simpler and cheaper (no search-space–specific derivations). Connections between loss landscape geometry and generalization have been studied via flatness/sharpness and Hessian spectra (e.g., 19 ; 18 ; 33 ; 12 ). Large gradients often co-occur with sharper regions of the loss; our findings align with this picture: models/checkpoints with smaller head gradients tend to exhibit better Top-1 and lower loss, suggesting improved separability of the learned features under the current head. Prior attempts to bypass a held-out validation set include curve extrapolation of the training trajectory, self-supervised or auxiliary criteria, and proxy losses (e.g., 8 ; 20 ; 34 ; 28 ; 2 ). Compared to those, our approach is instant (one forward+backward on a single supervised mini-batch), requires no extra data , and is architecture-agnostic. We also provide ablations against alternative head-only signals (L1/L2/L \infty , Fisher trace 3 , and scale-normalized variants), and show that a plain \ell_{2} head-gradient norm offers a strong accuracy proxy with minimal complexity. While our focus is validation-free checkpoint/model selection, the same probe can serve as a lightweight pre-screening module in NAS: quickly filter a pool of trained-once or partially trained candidates before expensive fine-tuning. This use is orthogonal to training-free NAS proxies ( 22 ; 5 ; 29 ; 36 ) and avoids assuming a search space. 3 Framework: Internal State Readouts This section describes the conceptual framework that unifies various internal diagnostics as projections of training state. We present one concrete instantiation—the head-gradient norm—as an illustrative example; a complete treatment with theoretical analysis and systematic comparisons will appear in the full paper. 3.1 Training State as Latent Variable We posit that the “quality” of learned representations at any checkpoint can be modeled as a latent variable \mathcal{S} that causally influences multiple observables. As training progresses and features become more linearly separable, this latent state improves, and various readouts reflect that improvement: • Gradient-based: Head-gradient norms, full-model gradient norms, Fisher information traces • Output-based: Softmax confidence, predictive entropy, margin between top-2 classes • Geometry-based: Loss curvature, sharpness measures, Hessian eigenvalues Each readout is a different projection of \mathcal{S} —analogous to viewing a 3D object from different angles. 3.2 Why Gradient Norms? The gradient norm at a checkpoint reflects the local loss curvature 19 ; 18 . For a model with parameters \theta at checkpoint t : \|\nabla_{\theta}\mathcal{L}(\theta_{t})\|\approx\lambda_{\max}(H_{t})\cdot\|\theta_{t}-\theta^{*}\| (1) where H_{t} is the Hessian at \theta_{t} , \lambda_{\max} is its maximum eigenvalue (sharpness), and \theta^{*} is a local minimum. The connection to generalization is well-established: flat minima (low sharpness) generalize better than sharp minima. The head-gradient norm, in particular, measures how