Measuring Uncertainty in Transformer Circuits with Effective Information Consistency
Anatoly A. Krasnovsky · 2025
Read Measuring Uncertainty in Transformer Circuits with Effective Information Consistency by Anatoly A. Krasnovsky (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-07149.txt.
Description
Anatoly A. Krasnovsky, in a solo 2025 paper, tackles a trust problem inside large language models. Research on how these models work has found small subnetworks, called Transformer circuits, that seem to carry out set tasks. The open question is how to tell, in a single pass, whether an active circuit is behaving in a coherent way and so is likely to be trustworthy.
Krasnovsky defines a score called EICS, for Effective-Information Consistency Score. The score joins a measure of local mismatch, built from Jacobians and activations, with a proxy for causal emergence at the whole-circuit level. Both parts come from the same forward run, and the score has no units. Guidance on reading the score and on computing cost is included. Tests on real model tasks are left for later work, so the claim stays a proposal for now.
Catalog record
| CREATOR | Anatoly A. Krasnovsky |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-07149