SelfGrader: Stable Jailbreak Detection for Large Language Models using Token-Level Logits

Zikai Zhang, Rui Hu, Olivera Kotevska, Jiahao Xu · 2026

Read SelfGrader: Stable Jailbreak Detection for Large Language Models using Token-Level Logits by Zikai Zhang, Rui Hu, Olivera Kotevska, Jiahao Xu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-01473.txt.

Introduction

Zhang and colleagues propose SelfGrader, a lightweight guardrail formulating jailbreak detection as grading using token-level logits. It evaluates safety within tokens, such as 0 to 9, and interprets their logit distribution as a safety signal. To align these signals with the safety rubric, it constructs PAC-guided anchor examples. It introduces a scoring rule considering harm and safety, yielding a score for harmfulness while reducing false alarms. Experiments across jailbreak benchmarks, adaptive attacks, benign benchmarks, language models, and baselines demonstrate robustness with low false alarm rates and latency.

Catalog record

CREATORZikai Zhang, Rui Hu, Olivera Kotevska, Jiahao Xu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE89046
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.01473

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SelfGrader: Stable Jailbreak Detection for Large Language Models using Token-Level Logits Zikai Zhang, Rui Hu, Olivera Kotevska, Jiahao Xu Abstract Large Language Models (LLMs) are powerful tools for answering user queries, yet they remain highly vulnerable to jailbreak attacks. Existing guardrail methods typically rely on internal features or textual responses to detect malicious queries, which either introduce substantial latency or suffer from the randomness in text generation. To overcome these limitations, we propose SelfGrader , a lightweight guardrail method that formulates jailbreak detection as a numerical grading problem using token-level logits. Specifically, SelfGrader evaluates the safety of a user query within a compact set of numerical tokens (NTs) (e.g., 0–9) and interprets their logit distribution as an internal safety signal. To align these signals with human intuition of maliciousness, SelfGrader introduces a dual-perspective scoring rule that considers both the maliciousness and benignness of the query, yielding a stable and interpretable score that reflects harmfulness and reduces the false positive rate simultaneously. Extensive experiments across diverse jailbreak benchmarks, multiple LLMs, and state-of-the-art guardrail baselines demonstrate that SelfGrader achieves up to a 22.66 % 22.66\% reduction in ASR on LLaMA-3-8B, while maintaining significantly lower memory overhead (up to 173 × 173\times ) and latency (up to 26 × 26\times ). 1 Introduction Large Language Models (LLMs) 37 ; 45 ; 31 ; 36 have achieved notable success across diverse tasks, such as question-answering (Q&A) 3 , mathematical reasoning 7 , and code generation 6 . Despite these advances, LLMs remain highly vulnerable to jailbreak attacks 35 ; 38 , where adversaries craft malicious queries to circumvent the safety alignment of LLMs and induce the model to produce harmful or policy-violating responses. Ensuring robust defenses against such attacks is thus critical for the safe and reliable deployment of LLMs. Guardrail methods 42 ; 26 ; 17 have emerged as a promising direction of defense, functioning independently of the target LLM’s generation and serving as an external protective layer. Existing guardrail methods generally fall into three categories: internal feature-based , classification-based , and generation-based guardrails. Internal feature-based methods 42 ; 14 leverage intermediate signals (e.g., activations, gradients) during the LLM’s generation process but often require threshold calibration from external datasets and incur significant computational cost, particularly when gradient signals are used. Classification-based methods 26 employ auxiliary classifiers to detect malicious queries. These models are usually trained for specific attack vectors, making them less robust against adaptively designed or unseen attacks. Generation-based methods 39 ; 13 ; 25 ; 15 rely on the safety alignment of LLMs, using red-list (unsafe) and green-list (safe) keywords to infer query intent. However, they depend on sampled outputs (e.g., the final generated response) and on manually constructed red/green lists (e.g., refusal messages, safety statements), which can never be fully comprehensive. Hence, these approaches operate in safety-semantic space suffer from both sampling bias (decisive tokens may not be generated) and keyword-matching bias (incomplete lists inevitably miss harmful tokens or misclassify benign ones), which reason over semantic content that is inherently coarse, lossy, and highly dependent on linguistic form. To address these limitations, we propose SelfGrader , a lightweight guardrail method that turns jailbreak detection into a simple but effective grading problem. Unlike prior approaches that depend on heavy model training or keyword matching, SelfGrader directly leverages the LLM’s logit outputs as safety signals. Specifically, the key idea is to use a small set of numerical tokens (NTs) (e.g., 0–9) as a scoring scale: given a user query, the guardrail model “grades” its maliciousness, and the logits over these NTs reveal the model’s internal safety judgment. Finally, we introduce a dual-perspective logit (DPL) scoring rule that considers both malicious and benign views, producing a stable score to decide whether the user query should be blocked or allowed. By doing so, SelfGrader interprets the model’s internal safety judgment in numerical space , using the logits associated with digit tokens. This converts the LLM’s latent belief state about safety (either maliciousness or benignness) into a calibrated numerical signal, rather than text or discrete labels. Our main contributions are summarized as follows: • We introduce SelfGrader, a lightweight guardrail method that solves jailbreak detection as a numerical grading problem using token-level logits. SelfGrader eliminates the need for expensive safety-classifier training or slow internal-representation inspection, making it advantageous for practical deployment and compatible with low-resource or latency-sensitive settings. • Instead of relying on the ambiguous textual generations or the entire vocabulary-level logits, SelfGrader operates on a compact set of NT-based logits, forming a closed, invariant, and task-aligned yet flexible numerical space that addresses the instability of safety-semantic space. • We propose a DPL scoring rule that evaluates the safety of a user query from both maliciousness and benignness perspectives. This dual-view design captures the true degree of harmfulness while mitigating prompt-specific bias, resulting in a more stable guardrail signal that effectively reduces the false positive rate. • We conduct extensive empirical evaluation, demonstrating that SelfGrader is robust against diverse and adaptive jailbreak attacks while maintaining low latency and memory overhead. 2 Related Works Guardrail Methods for Jailbreak Attacks. Recent research on jailbreak defenses has explored several directions. Internal feature-based methods exploit hidden representations of LLMs when processing malicious queries. For example, Perplexity Filter 18 computes the perplexity of model responses, GradSafe 42 measures gradient cosine similarities between the user query and reference prompts (safe vs. unsafe), and GradientCuff 14 perturbs token embeddings and evaluates the norm of the refusal-loss gradient. Classification-based methods such as PromptGuard 26 employ external classifiers to categorize queries into Benign , Prompt Injection , or Jailbreak . Generation-based methods instead rely on guardrail prompts and keyword matching. For example, SelfDefend 39 tasked LLM with highlighting policy-violating segments in the user query or summarizing user query intent, and then checking for keyword “No”. Similarly, QGuard 21 uses the logits of the semantic tokens “yes” and “no” to build a binary classifier. WildGuard 13 jointly evaluates query harmfulness, response refusal, and harmful content. GuardReasoner 25 applies multi-step reasoning with keywords such as “unharmful,” and Llama Guard 17 classifies outputs into “safe” or “unsafe.” These methods rely heavily on keyword matching in generated responses, leaving them vulnerable to paraphrasing-based evasion. In contrast, our proposed SelfGrader directly leverages token-level logit distributions without external classifiers, calibration datasets, or full response generation, offering an efficient and robust mechanism for jailbreak detection. Confidence Estimation for Model Responses. Token-level logits have been widely explored as confidence signals for assessing response quality. For instance, Self-Evaluation 33 prompts models to evaluate their own outputs, using the logits of “yes” or “no” tokens as confidence indicators. Self-Certainty 20 distinguishes between correct and incorrect answers by leveraging response logits. However, these approaches are designed for evaluating response quality rather than defending against jailbreaks. They rely on specific keyword tokens over the entire vocabulary, yet not all vocabulary dimensions are meaningful for safety judgment. The keyword space is open-ended, highly context-dependent, and impossible to exhaustively defined for robust maliciousness or benignness assessment, which makes their direct adaptation for guardrail design non-trivial. Due to the page limitation, more discussions about jailbreak attacks, other LLM attack methods like prompt injection attacks, and safety alignment training can be found in Appendix H . 3 Problem Formulation and

Read the complete text