Robust and Efficient Guardrails with Latent Reasoning Siddharth Sai, Xiaofei Wen, Muhao Chen Abstract Maintaining the safety of large language models (LLMs) is crucial as they are increasingly deployed in real-world applications. Existing safety guardrails typically rely on single-pass classification or, more recently, distilled reasoning. Reasoning-based guardrails significantly outperform classification-only baselines, but they incur substantial query latency and token overhead that make them impractical for high-throughput deployment. To address this challenge, we propose CoLaGuard , a guardrail model that transfers multi-step safety reasoning into a continuous latent space through a stage-wise training curriculum, enabling direct hidden-state propagation at inference. Evaluated on ten prompt- and response-moderation settings spanning eight safety benchmarks, CoLaGuard improves macro-F1 by 8.24 points over Llama Guard 3 and matches our explicit reasoning baseline, GuardReasoner, in macro-F1 while delivering a 12.9 × \times speedup and 22.4 × \times reduction in token usage. Our results suggest that latent reasoning offers a practical alternative to explicit rationale generation for deployable guardrails, jointly improving safety robustness and inference efficiency rather than treating them as competing objectives. 1 Introduction Figure 1: Overview of CoLaGuard . Unlike explicit reasoning guardrails (left) that generate chain-of-thought tokens before assigning labels, CoLaGuard (right) reasons through recurrent latent states, preserving moderation performance while avoiding token generation overhead and enabling 12.9 \times faster inference and 22.4 \times fewer tokens. CoLaGuard ’s stage-wise internalization curriculum (center) begins with explicit CoT supervision and progressively replaces reasoning tokens with latent states, shifting reasoning into hidden activations. As Large Language Models (LLMs) become integral to daily and industrial applications, ensuring their alignment with human values is critical. Although alignment training methods such as RLHF ( 29 ; 32 ) can improve model behavior, they require modifying the target model and are costly to update after deployment. External safety guardrails ( 13 ; 10 ) therefore provide a practical alternative by offloading input and output moderation to smaller, third-party models. Early guardrails typically formulate moderation as single-pass classification, which is efficient but often becomes brittle under ambiguous, adversarial, or context-dependent safety decisions. Recent explicit reasoning guardrails ( 39 ; 21 ) improve robustness by learning from distilled chain-of-thought (CoT) supervision ( 12 ; 17 ) and generating intermediate rationales before predicting a safety label. MrGuard ( 40 ) further extends reasoning-based guardrails to multilingual safety moderation by combining synthetic multilingual supervision with curriculum-guided Group Relative Policy Optimization (GRPO) ( 35 ) . However, this robustness comes at a steep computational cost. Because these models verbalize their intermediate rationales, moderation becomes a long autoregressive generation process. The additional CoT tokens substantially inflate inference time and completion-token cost, making explicit reasoning guardrails difficult to deploy in high-traffic, real-time settings ( 21 ; 36 ) . Existing efficiency-oriented variants, such as shorter supervised traces or reasoning on/off switches ( 27 ; 36 ) , reduce the amount or frequency of rationale generation but still rely on explicit decoding and may sacrifice robustness. This motivates a natural question: can guardrails retain the benefits of reasoning supervision without generating reasoning tokens at inference time? We study this question through CoLaGuard , a latent-reasoning safety guardrail that internalizes explicit safety rationales into continuous recurrent states as shown in Figure 1 . Inspired by Coconut ( 11 ) and ICoT-SI ( 4 ) , CoLaGuard performs a fixed number of latent recurrent steps in place of explicit rationale generation. It first learns from CoT supervision and then progressively replaces rationale tokens with latent states, allowing the model to directly predict the safety label without autoregressive rationale generation. A practical challenge is that pretrained LLMs are optimized to consume token embeddings rather than recirculated contextual hidden states, which can create a distribution mismatch during latent recurrence. To reduce this mismatch, we adopt Context-Prediction Fusion ( 20 ) , which combines contextual hidden-state information with predictive semantic guidance from the vocabulary embedding space. This stabilizes latent recurrence while preserving the latency and token-efficiency benefits of avoiding explicit CoT generation. In summary, this work makes three main contributions. (1) We introduce CoLaGuard , a latent-reasoning safety guardrail that internalizes explicit safety rationales through a stage-wise curriculum, enabling moderation without autoregressive rationale generation at inference time. (2) We show that CoLaGuard preserves the robustness of explicit reasoning guardrails while substantially reducing inference cost, suggesting that reasoning-based moderation can be made practical without verbalized rationales. (3) We analyze the latent recurrence process and find that CoLaGuard improves over vanilla Coconut, consistent with progressive safety-relevant representation shifts across latent steps that are largely absent in vanilla Coconut recurrence. 2 Related Work LLM Guardrails External guardrails provide a lightweight mechanism for safety moderation without modifying the base LLM. Early architectures such as Llama Guard ( 13 ) and WildGuard ( 10 ) treated moderation as classification, followed by models like ShieldGemma ( 43 ) , Aegis ( 6 ) , and Qwen3Guard ( 45 ) , which improved performance through broader taxonomies. The broader guardrail literature has expanded robustness through adversarially resilient moderation ( 41 ) and structured safety knowledge ( 16 ) . Recent work further improves performance with reasoning: GuardReasoner ( 21 ) and ThinkGuard ( 39 ) use chain-of-thought rationales ( 37 ) from expert models to improve generalization, while MrGuard ( 40 ) extends reasoning-based guardrails to multilingual moderation through synthetic multilingual supervision and curriculum-guided GRPO. Others explore efficiency trade-offs through shorter rationale traces and on/off switches ( 27 ; 36 ; 33 ) . However, because these models verbalize reasoning in natural language, they incur steep autoregressive decoding costs that limit their practicality for high-traffic, real-world deployment. Latent Reasoning A growing literature suggests that effective reasoning can occur within a model’s hidden states rather than through explicit tokens ( 2 ; 46 ; 1 ) . This space includes augmenting models with "thinking" tokens ( 8 ; 42 ; 30 ) , internalizing CoT through staged curricula ( 5 ; 4 ) , and feeding hidden states back as continuous input embeddings ( 11 ; 3 ; 46 ) . However, these methods have largely been studied on mathematical and logical reasoning tasks, and directly recycling raw hidden states can become unstable at larger scales due to distribution mismatch with the token embedding manifold. Latent Thoughts Tuning ( 20 ) addresses this with a context-prediction fusion mechanism that aligns contextual hidden states with predictive signals from the vocabulary embedding space. CoLaGuard adapts these techniques, showing that latent reasoning can drastically reduce latency costs and preserve the robustness of explicit baselines in safety moderation. 3 CoLaGuard We now present CoLaGuard , a latent-reasoning guardrail framework for efficient prompt and response moderation. CoLaGuard uses explicit safety rationales generated by expert models as training-time supervision, then progressively internalizes this step-by-step reasoning into recurrent latent states so that inference incurs only a fixed latent computation budget before decoding the safety labels. We formulate the guardrail task in § 3.1 , describe reasoning-augmented supervision and explicit warm-up in § 3.2 –§ 3.3 , and present latent recurrence, stage-wise internalization and efficient inference in § 3.4 –§ 3.6 . 3.1 Guardrail Task Given a user prompt x and a model response s , a guardrail model G_{\theta} predicts the safety of both the input request and the generated response (\hat{y}^{p},\hat{y}^{r})=G_{\theta}(x,s) , where \hat{y}^{p}\in\mathcal{Y} denotes the predicted prompt harmfulness label, \hat{y}^{r}\in\mathcal{Y} denotes the predicted response harmfulness label, and \mathcal{Y} denotes the set of safety categories in the guardrail’s policy. 3.2 Reasoning-Augmented Supervision The central challenge is maintaining the robustness of reasoning-based guardrails without requiring that the guardrail verbalize its reasoning process at inference time. CoLaGuard addresses this by using explicit rationales for the initial training scaffolding. This follows prior work on chain-of-thought reasoning, step-by-step distillation, and reasoning-based safety guardrails, where intermediate rationales provide richer supervision than final labels alone ( 37 ; 12 ; 17 ; 21 ; 39 ) . We assume access to a reasoning-augmented guardrail corpus \mathcal{D}=\{(x_{i},s_{i},r_{i},y_{i})\}_{i=1}^{N} , where x_{i} is a user prompt, s_{i} is the corresponding model response, y_{i}=(y_{i}^{p},y_{i}^{r}) contains the final prompt and response safety labels, and r_{i}=(r_{i}^{1},r_{i}^{2},\ldots,r_{i}^{m_{i}}) is a step-separated safety rationale. Unlike standard label-only guardrail training, this supervision exposes the model to the reasoning underlying the final moderation decision. However, CoLaGuard does not aim to generate these rationales at inference time. Instead, the rationales serve as targets during the initial stages of training so that the model can later compress the deliberation process into latent steps. 3.3 Explicit Reasoning Warm-Up The first stage (Stage 0) trains the model as an explicit reasoning guardrail. Given an instruction I , prompt x , response s , rationale r , and final label tuple y , the model is optimized to generate structured safety-relevant rationale followed by the final safety labels: \mathcal{L}_{\mathrm{warm}}=-\mathbb{E}_{(x,s,r,y)\sim\mathcal{D}}\log p_{\theta}(r,y\mid I,x,s). This warm-up follows the explicit reasoning guardrail paradigm, where models learn to verbalize intermediate safety reasoning before predicting final moderation labels ( 21 ; 39 ) . We denote the resulting model as G_{\theta}^{0} , from which subsequent stages progressively replace explicit rationale steps with latent recurrent steps. 3.4 Dual-Mode Latent Recurrence To internalize reasoning, CoLaGuard switches between two modes. In language mode, the model consumes standard token embeddings and predicts the next token autoregressively. In latent mode, the model does not consume a standard token embedding; instead, the previous hidden state is fed back as the next input representation. Let e(\cdot) denote the token embedding function and let h_{t}\in\mathbb{R}^{d} be the last-layer hidden state at position t . For a sequence with a latent span beginning at position a and ending at position b , vanilla latent recurrence replaces the input embedding at each latent position with the previous hidden state: E_{t}=\begin{cases}e(w_{t}),&tb,\\ h_{t-1},&a\leq t\leq b,\end{cases} where w_{t} is the discrete token at position t outside the latent span. This follows the chain-of-continuous-thought formulation introduced by 11 , allowing the model to perform recurrent computation in continuous latent space rather than generating intermediate rationale tokens. While this latent recurrence lays the foundation of CoLaGuard , directly feeding contextual hidden states back into a pretrained transformer creates a distribution mismatch since the base model is trained to consume token embeddings, while h_{t-1} is a hidden representation. To reduce the hidden-state/token-embedding mismatch observed in latent recurrence, we adopt context-prediction fusion from Latent Thoughts Tuning ( 20 ) . At each latent position, the model first computes a predictive embedding from the next-token distribution induced by the previous hidden state: e_{\mathrm{pred}}(h_{t-1})=\sum_{v\in\mathcal{V}_{p}}\tilde{p}_{\theta}(v\mid h_{t-1})e(v), where \mathcal{V}_{p} is the nucleus-filtered vocabulary set, and \tilde{p}_{\theta}(v\mid h_{t-1}) is the renormalized probability distribution over this set. Structural latent-control tokens are excluded from this distribution. The recurrent input is then constructed by fusing the contextual hidden state with the predictive embedding: \tilde{e}_{t}=\alpha h_{t-1}+(1-\alpha)e_{\mathrm{pred}}(h_{t-1}), where \alpha\in[0,1] controls the balance between contextual continuity and semantic anchoring. Finally, a lightweight projection module maps the fused representation back into the model input space: \mathbf{e}^{\mathrm{in}}_{t}=\begin{cases}h_{t-1},&\alpha=1,\\ g_{\phi}(\tilde{e}_{t}),&\alpha<1\text{ and adapter is used},\\ \tilde{e}_{t},&\alpha<1\text{ and no adapter is used}.\end{cases} where g_{\phi} is a trainable adapter. When \alpha=1 , this reduces to vanilla hidden-state recurrence; when \alpha<1 , the recurrent state is anchored by predictive information from the vocabulary embedding. 3.5 Stage-Wise Internalization The core of CoLaGuard is a stage-wise curriculum that progressively replaces natural-language rationale steps with recurrent latent steps. The staged replacement schedule follows prior internalization curricula showing that gradually replacing explicit reasoning tokens is more stable than removing rationales all at once ( 5 ; 4 ; 11 ) . For an example with m rationale steps, we write the step-separated rationale as r=(r^{1},r^{2},\ldots,r^{m}), let K denote the maximum number of reasoning steps represented by the latent budget, and define \ell_{k}=\min(k,K) . At stage k , the first k rationale steps are removed and replaced with \ell_{k}c latent positions: (r^{1},\ldots,r^{k})\rightarrow(z_{1},\ldots,z_{\ell_{k}c}), where c is the number of latent positions allocated per replaced reasoning step within the latent budget. We denote the resulting training sequence as q^{(k)} , which contains the instruction, prompt, response, latent span, any remaining rationale steps r^{k+1},\ldots,r^{m} , and the final labels y . If k\geq m , the rationale is fully replaced and the final label tuple follows the latent span directly; however, because the latent budget is fixed, examples with more than K rationale steps may still contain explicit rationale tokens after the maximum latent stage is reached. We therefore include a final compression stage that keeps the latent span fixed at Kc positions while removing all remaining rationale steps: r^{1:m}\rightarrow(z_{1},\ldots,z_{Kc}). This extra stage enables the absorption of residual explicit reasoning signal into the fixed latent recurrence rather than remaining decoded as text. Training optimizes only the remaining language tokens and final labels. The prompt, response, latent-control tokens, and latent positions are masked from the language-modeling loss. Let \mathcal{M}^{(k)} be the set of supervised token positions in q^{(k)} . The internalization objective is \mathcal{L}_{\mathrm{int}}^{(k)}=-\mathbb{E}_{(x,s,r,y)\sim\mathcal{D}}\sum_{t\in\mathcal{M}^{(k)}}\log p_{\theta}(q^{(k)}_{t}\mid q^{(k)}_{