NExT-Guard: Training-Free Streaming Safeguard without Token-Level Labels
Junfeng Fang, Nachuan Chen, Houcheng Jiang, Dan Zhang, Fei Shen, Xiang Wang, Xiangnan He, Tat-Seng Chua · 2026
Read NExT-Guard: Training-Free Streaming Safeguard without Token-Level Labels by Junfeng Fang, Nachuan Chen, Houcheng Jiang, Dan Zhang, Fei Shen, Xiang Wang, Xiangnan He, Tat-Seng Chua (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-02219.txt.
Introduction
Junfeng Fang, Nachuan Chen, Houcheng Jiang, and colleagues build a live safety filter for language models in this 2026 paper. Text arrives token by token, so a filter must act mid-stream, not after the fact. Filters trained per token need costly labels and tend to overfit. The team argues that ready-made filters already hold token-level risk clues inside them.
Their filter, NExT-Guard, reads those clues with sparse autoencoders, tools that split inner signals into clear features. It needs no new training and uses autoencoders from public base models. In tests it beat both late filters and trained live filters. Gains held across models, autoencoder types, and risk types. The approach offers a cheap, flexible path to real-time safety.
Catalog record
| CREATOR | Junfeng Fang, Nachuan Chen, Houcheng Jiang, Dan Zhang, Fei Shen, Xiang Wang, Xiangnan He, Tat-Seng Chua |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47101 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.02219
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
NExT-Guard: Training-Free Streaming Safeguard without Token-Level Labels Junfeng Fang, Nachuan Chen, Houcheng Jiang, Dan Zhang, Fei Shen, Xiang Wang, Xiangnan He, Tat-Seng Chua Abstract Large language models are increasingly deployed in streaming scenarios, rendering conventional post-hoc safeguards ineffective as they fail to interdict unsafe content in real-time. While streaming safeguards based on token-level supervised training could address this, they necessitate expensive annotations and suffer from severe overfitting. In this work, we challenge the paradigm that streaming safety must rely on token-level supervised training. Instead, it is an inherent capability of well-trained post-hoc safeguards, as they already encode token-level risk signals in hidden representations. Hence, we introduce NExT-Guard , a training-free framework that achieves streaming safeguards by monitoring interpretable latent features from Sparse Autoencoders (SAEs). It uses pretrained SAEs from publicly available base LLMs, enabling flexible, low-cost deployment without token-level supervision. Experimental results show that NExT-Guard outperforms both post-hoc and streaming safeguards based on supervised training, with superior robustness across models, SAE variants, and risk scenarios. These results make NExT-Guard a universal and scalable paradigm for real-time safety, accelerating the practical deployment of streaming safeguards. Code is available at https://github.com/NashChennc/NExTGuard . † † affiliationnotice: Equal contribution 1 Introduction Large Language Models (LLMs) have been extensively integrated into real-time applications, ranging from interactive dialogue systems to live collaborative assistants ( 19 ; 9 ; 3 ) . In these streaming scenarios, model outputs are generated and exposed to users token-by-token in near real-time ( 18 ; 26 ; 29 ) . However, current safety mechanisms predominantly adhere to a post-hoc evaluation paradigm, where the safeguard evaluates the content only after the entire sequence has been generated ( 31 ; 2 ; 7 ) . This creates a critical temporal misalignment between streaming generation and post-hoc safeguard: harmful information can be exposed to the user as soon as a single unsafe token appears. Even if the final output is eventually flagged and intercepted, the safety breach has already occurred ( 28 ; 16 ) . Consequently, conventional post-hoc safeguards are inherently reactive, failing to provide the preemptive guarantees required for real-world applications. Establishing a Streaming Safeguard for real-time monitoring and token-level intervention has therefore become an imperative necessity for the safety deployment of LLMs ( 16 ; 15 ; 32 ) . Despite the clear necessity, transitioning to streaming safeguards is far from trivial. As shown in Figure 1 (a), prevailing approaches typically rely on token-level supervised training , which requires large-scale datasets with per-token safety annotations ( 28 ; 16 ; 14 ; 15 ; 32 ) . This token-level annotation is prohibitively expensive and inherently subjective, as token harmfulness often depends on long-range, evolving context ( 16 ) . Especially in specialized domains such as law and medicine, nuanced safety boundaries make large-scale token labeling particularly impractical. Worse still, as shown in Figure 1 (b), such token-level supervised training is prone to severe overfitting: even industrial-grade streaming safeguards such as Qwen3Guard-8B ( 32 ) may make biased judgments based on isolated keywords rather than a holistic understanding of the context. Besides these limitations, any change in safety policy, risk definition, or application scenarios necessitates the whole re‑annotation and retraining process, significantly limiting the adaptability and scalability of current methods. Hence, the reliance on token-level supervised training has emerged as a major obstacle to deploying streaming safeguards in real-world environments. Does streaming safety truly require additional training? In this paper, we propose a divergent perspective: streaming safety is not an external skill to be acquired via supervision, but rather an intrinsic capability already latent within the hidden representations of existing post-hoc safeguards. Specifically, we posit that for a post-hoc safeguard to arrive at a reliable final judgment, it must incrementally encode and integrate safety-critical semantics within its latent representations as the sequence unfolds. From an information-theoretic standpoint, the global risk captured by the final classification head is the culmination of localized risk signals embedded in each token’s representations. Therefore, the challenge of streaming safety is not to inject safety knowledge through training, but to faithfully decode and project these existing internal signals into an interpretable space for real-time monitoring. This motivation leads us to the use of Sparse Autoencoders (SAEs) ( 8 ; 4 ) , which can disentangle these representations into sparse, semantically-grounded latent features. Figure 1 : Comparison between current streaming safety and NExT-Guard . (a) Current paradigm relies heavily on token-level labels. (b) Qwen3-Guard-8B-Streaming ( 32 ) is prone to severe overfitting to individual token semantics. (c) NExT-Guard achieves streaming safety without training. (d) Unsafe tokens are precisely identified by NExT-Guard . Best viewed in color. Building on this insight, we introduce NExT-Guard , a training-free framework that upgrades any post-hoc safeguard into a streaming safeguard by directly exposing and monitoring its latent safety signals. As illustrated in Figure 1 (c), NExT-Guard identifies risk-relevant SAE features via an offline contrastive analysis of SAE activations between safe and unsafe samples, circumventing the need for expensive token-level labels. Importantly, NExT-Guard does not require training SAEs from scratch: we directly leverage publicly available SAEs trained on the same base LLM as the post-hoc safeguard, relying on the strong generalization of SAEs. During streaming inference, the identified dimensions are tracked online as token-level risk indicators, enabling real-time intervention in a training-free manner. Interestingly, empirical results show that NExT-Guard often outperform its parent post-hoc safeguard, despite operating on only a fraction of the sequence. This result suggests that existing safeguards possess a latent risk-awareness that significantly exceeds their manifested detection performance, and NExT-Guard effectively unlocks this potential in real time. Beyond efficiency, this paradigm redefines streaming safety as a flexible, service-oriented defense paradigm. By decoupling detection from learned weights, NExT-Guard eliminates the safety lag inherent in traditional retraining cycles, enabling instantaneous adaptation to emerging threats with mechanistic transparency. Extensive experiments have validated the efficacy of NExT-Guard across diverse safety benchmarks ( e.g. , Aegis ( 6 ) and SimpST ( 24 ) ). Specifically, NExT-Guard achieves superior performance compared to advancing training-based streaming and post-hoc safeguards. Its remarkable robustness across different base models, SAE variants, and risk scenarios underscores its potential as a universal and scalable paradigm for real-time safety. In summary, NExT-Guard not only accelerates the practical deployment of streaming safeguards, but also bridges the long-standing gap between post-hoc detection and real-time intervention. Figure 2 : Overview of NExT-Guard , which identifies safety-relevant SAE features offline and integrates them to calculate the safety score for training-free streaming intervention. Best viewed in color. 2 Preliminary Streaming Safeguard. A streaming safeguard performs real-time risk detection during autoregressive generation ( 16 ) . Given an input prompt {X} , an LLM produces an output sequence {Y}=\{y_{1},\ldots,y_{T}\} token by token. At step i , the model generates y_{t} conditioned on the prompt and the previously generated prefix {Y}_{<i}=\{y_{1},\ldots,y_{t-1}\} . In parallel, a streaming safeguard observes only the partial context ({X},{Y}_{<t},y_{t}) and assigns the current token a risk score that measures its contribution to an unsafe response: c_{t}=P\left(r=1\mid{X},{Y}_{<t},y_{t}\right)=f\bigl(z({X},{Y}_{<t},y_{t})\bigr), (1) where r=1 denotes the unsafe class, z(\cdot) extracts a token-level representation, and f(\cdot) maps this representation to a real-time risk prediction, which is typically implemented as a supervised classifier trained on token-level safety annotations ( 15 ; 32 )