Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models
Nay Myat Min, Long H. Pham, Jun Sun · 2026
Read Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models by Nay Myat Min, Long H. Pham, Jun Sun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-24542.txt.
Introduction
Min, Pham and Sun offer a runtime monitor for large language models that needs no retraining, no clean reference model and no knowledge of the trigger. Their Layerwise Convergence Fingerprinting tracks how hidden states change from layer to layer and flags unusual trajectories. Across backdoors, jailbreaks and prompt injections it cut backdoor success sharply, caught most jailbreak attempts and flagged injected text payloads, at low extra computing cost.
Catalog record
| CREATOR | Nay Myat Min, Long H. Pham, Jun Sun |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 108545 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.24542
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models Nay Myat Min, Long H. Pham, Jun Sun Abstract Large language models deployed at runtime can misbehave in ways that clean-data validation cannot anticipate: training-time backdoors lie dormant until triggered, jailbreaks subvert safety alignment, and prompt injections override the deployer’s instructions. Existing runtime defenses address these threats one at a time and often assume a clean reference model, trigger knowledge, or editable weights, assumptions that rarely hold for opaque third-party artifacts. We introduce Layerwise Convergence Fingerprinting (LCF), a tuning-free runtime monitor that treats the inter-layer hidden-state trajectory as a health signal: LCF computes a diagonal Mahalanobis distance on every inter-layer difference, aggregates via Ledoit–Wolf shrinkage, and thresholds via leave-one-out calibration on 200 clean examples, with no reference model, trigger knowledge, or retraining. Evaluated on four architectures (Llama-3-8B, Qwen2.5-7B, Gemma-2-9B, Qwen2.5-14B) across backdoors, jailbreaks, and prompt injection (56 backdoor combinations, 3 jailbreak techniques, and BIPIA email + code-QA), LCF reduces mean backdoor attack success rate (ASR) below 1% on Qwen2.5-7B and Gemma-2 and to 1.3% on Qwen2.5-14B, detects 92–100% of DAN jailbreaks (62–100% for GCG and softer role-play), and flags 100% of text-payload injections across all eight (model, domain) cells, at 12–16% backdoor FPR and < 0.1 % {<}0.1\% inference overhead. A single aggregation score covers all three threat families without threat-specific tuning, positioning LCF as a general-purpose runtime safety layer for cloud-served and on-device LLMs. 1 Introduction Large language models and LLM-powered agents, once deployed, can produce attacker-directed behavior in response to inputs that look ordinary. Three families of runtime threats illustrate this. Backdoors lie dormant in third-party fine-tuned artifacts until a trigger appears ( 9 ; 15 ) . Jailbreak prompts subvert safety alignment through adversarial framing ( 38 ) . Prompt injections embed task-overriding instructions inside the content that the model is asked to process ( 33 ) . In all cases, it is challenging to tell from the prompt alone whether it will cause misbehavior, and ordinary clean-data validation during evaluation fails to catch it. Existing runtime defenses are typically threat-specific or rely on assumptions that limit their reach. Some require auxiliary models: a reference LLM with a matching tokenizer ( 16 ) , or an external language model for input-perplexity filtering ( 21 ) . Others train an auxiliary classifier on labeled triggered examples, separately per threat family ( 36 ; 35 ) ; among these, LLMScan applies a unified causal-scanning recipe across threats but still requires per-threat labeled training and incurs seconds-scale overhead from per-token and per-layer interventions. Others modify deployed weights ( 19 ) . A final set narrows the setting: output-confidence anomalies for backdoored LLMs ( 30 ) , or activation statistics for BERT-family classification models ( 32 ) . Dedicated jailbreak and prompt-injection defenses likewise build on threat-specific priors. Practitioners deploying community LLMs need a runtime signal that is calibration-only on clean data, decides at prefill before any token is emitted, and handles backdoors, jailbreaks, and prompt injections under a single calibrated threshold. We observe that the transformer’s layerwise computation admits a natural health measure . During clean inference, adjacent-layer hidden states change smoothly as the representation matures. When a misbehavior-inducing input arrives, the model must redirect its internal state toward the misbehavior trajectory, producing anomalous jumps in the inter-layer difference vectors. The all-layer profile of these jumps is a fingerprint of the model’s internal thinking process, and a disrupted fingerprint signals derailment regardless of whether the cause is a backdoor, a jailbreak, or a prompt injection. We instantiate this idea as Layerwise Convergence Fingerprinting (LCF), a tuning-free runtime monitor built on three design choices: per-layer diagonal Mahalanobis scoring of the inter-layer hidden-state difference, Ledoit–Wolf aggregation across all layers to capture correlated deviations without layer selection, and leave-one-out calibrated thresholding on 200 clean examples. Applied without modification to a deployed model, LCF captures qualitatively different threats at different depths. We find preliminary evidence of a three-band depth stratification: jailbreak prompts produce their largest anomalies at early layers (unusual token patterns disrupt initial processing), prompt injections at mid layers (task-overriding instructions compete with the intended context), and training-time backdoors at architecture-specific depths in the later half of the network (mid on Llama-3, late on Gemma-2 and Qwen; Section 5.3 ). Because no single layer provides universal coverage, a single all-layer aggregation captures all three bands without threat-specific tuning. Contributions. (1) We frame runtime LLM misbehavior detection as monitoring the health of the transformer’s layerwise computation, and introduce LCF as a tuning-free, threat-agnostic realization. (2) A single all-layer Ledoit–Wolf Mahalanobis aggregation reduces backdoor mean ASR below 1% on Qwen2.5-7B and Gemma-2 (0.9% and 0.7%) and to 1.3% on Qwen2.5-14B (48 layers), detects DAN-style jailbreaks at 92–100% (GCG and softer role-play at 62–100%), and flags 100% of text-payload prompt injections, at {<}0.1\% measured inference overhead. (3) We present evidence for a three-band depth stratification (jailbreaks early, prompt injections mid, backdoors late) captured by a single score. (4) LCF requires no reference model, no editable weights, and no trigger knowledge, and leaves accepted outputs unmodified. 2 Problem setup Setting. We consider a deployed autoregressive LLM, or an LLM-powered agent, whose weights may already be compromised and whose inputs may be adversarial. Three concrete threat families cover the bulk of runtime LLM misbehavior reported in practice. Training-time backdoors are installed when a compromised fine-tuned artifact (a LoRA adapter, merged checkpoint, or fully fine-tuned model) is released on top of a clean base; triggered inputs produce attacker-chosen behavior while clean inputs remain apparently benign. Jailbreak prompts and prompt injections control only the input at inference time, using adversarial framing or task-overriding instructions (embedded in user messages or in retrieved content) to override the deployer’s intended behavior. In all three cases, the deployer cannot tell from the input alone whether the model will produce aligned behavior, and clean-data validation during evaluation fails to catch the problem. Assumptions. We envision LCF as a runtime monitor embedded in the inference layer of an AI service provider, or bundled with the model on an edge device, co-located with the deployed LLM so that it has white-box access to intermediate hidden states. LCF calibrates on a small clean dataset (e.g., 200 examples) drawn from the expected deployment distribution and requires no reference model, no knowledge of trigger tokens or adversarial templates, and no prior knowledge of whether the model is compromised. When the runtime score exceeds the calibrated threshold, LCF signals abstention and the serving stack returns a safe fallback; we see output steering as a natural future extension of the same signal. Table 1 contrasts these requirements with five baseline defenses. Scope. We study white-box runtime detection of misbehavior induced by backdoors, jailbreak prompts, and prompt injections, under an abstention-based response. We do not address black-box deployments, where LCF’s hidden-state requirement cannot be satisfied, nor model-repair techniques that modify the deployed weights. Appendix E evaluates a defense-aware adaptive adversary on LCF. 2.1 Runtime defenses for LLM misbehavior The closest prior runtime monitors all impose assumptions that limit reach in opaque third-party deployments: per-attack labeled training data and seconds-scale forward-pass overhead (LLMScan, 35 ); editable weights and frequent output incoherence (CROW, 19 ); a trusted clean reference model run in lockstep at