LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails

Nanxi Li, Zhengyue Zhao, Chaowei Xiao · 2026

Read LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails by Nanxi Li, Zhengyue Zhao, Chaowei Xiao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-17329.txt.

Introduction

Li, Zhao and Xiao introduce Latent Policy Guardrail, a safety layer that adapts to policies stated at inference time without retraining. It compresses the deliberation behind intent reading and policy grounding into continuous states, then emits only a compact verdict tied to violated clauses. The 4-billion-parameter version reaches about 85 percent safety accuracy with just ten latent tokens, beating the strongest dynamic baseline while running roughly eleven times faster than a thinking-model rival.

Catalog record

CREATORNanxi Li, Zhengyue Zhao, Chaowei Xiao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59755
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.17329

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails Nanxi Li, Zhengyue Zhao, Chaowei Xiao Abstract Guardrails are a critical safety layer for modern AI systems, but their operating regime is changing. As LLMs are deployed as customized assistants, safety policies are increasingly specified at inference time by users, organizations, or regulatory contexts. This makes safety enforcement fundamentally dynamic: the guardrail should adapt to changing safety policies without retraining. Yet this requirement creates a fundamental tension: faithfully judging complex policy contexts demands reasoning capability, while practical deployment requires low-latency responses. We introduce Latent Policy Guardrail (LPG), a guardrail framework that learns semantic latent deliberation over dynamic policies. LPG compresses the internal deliberation needed for intent interpretation and policy grounding into continuous states supervised by decision-relevant semantics. At inference time, it generates only a compact verdict anchored to the violated policy clauses, preserving auditability while avoiding the latency of explicit reasoning. Across policy guardrail benchmarks, LPG-4B reaches 84.5% average safety accuracy and 77.9% F1 by compressing deliberation into just 10 latent tokens, outperforming the strongest dynamic baseline while running roughly 11 × \times faster than Qwen3-4B-Thinking under the single-sample evaluation setup. Code and data are available at https://github.com/SaFo-Lab/Latent_Policy_Guard . 1 Introduction Large language models (LLMs) now power customer-facing chatbots ( 18 ) , autonomous agents ( 30 ) , and healthcare assistants ( 26 ) , and ensuring that their outputs respect deployment-specific norms has become a central safety concern ( 25 ; 2 ) . The dominant defense is the guardrail model , a lightweight classifier that screens prompts and responses against a safety taxonomy ( 11 ; 6 ; 37 ) . Such models are typically trained on fixed taxonomies (e.g., the OpenAI Moderation API) and have been deployed at scale precisely because they are fast and decisive. Fixed taxonomies, however, are increasingly out of step with how real systems are deployed. An HR chatbot must prohibit hiring discrimination, a financial advisor must enforce regulatory disclaimers, an educational platform must filter age-inappropriate content. The rise of agentic AI multiplies the demand further, as rules are attached to specific tools, tasks, and tenants ( 23 ; 21 ) . These constraints cannot all be enumerated at pre-training time, and they evolve faster than any retraining cycle. Recent policy-aware guardrails ( 9 ; 19 ) address this by accepting natural-language policies at inference time and judging each interaction against the supplied rules, with no retraining required when the policy changes. Policy-awareness, however, shifts the burden onto reasoning : the guardrail must read a potentially long policy document, identify which clauses apply, and produce a verdict grounded in those clauses, yet existing approaches sit on opposite ends of an unfavorable trade-off. Non-reasoning guardrails such as DynaGuard ( 9 ) are fast but brittle (Section 3 ): their verdicts swing by up to 7.4 accuracy points under simple permutations of the policy list, and a counterfactual probe that drops only the rule an unsafe sample violates fails to flip their verdict, betraying reliance on positional artifacts and content priors rather than the supplied clauses. Reasoning guardrails such as GuardReasoner ( 22 ) and ThinkGuard ( 34 ) buy accuracy with explicit chains of thought that, in our measurements, run more than an order of magnitude slower, prohibitive for real-time moderation pipelines. An effective guardrail must therefore anchor each verdict in the specific violated clause and do so independently of surface position, both signatures of deep policy understanding, yet the cost of explicit deliberation rules out solutions that rely on long verbal rationales. A natural remedy is to move the deliberation off the surface tokens. Recent work has shown that LLM reasoning can be performed in continuous latent space rather than via discrete tokens ( 7 ; 8 ; 1 ) , yielding speedups on math and planning. Adopting this idea for guardrails, however, is far from a drop-in transfer, and is where the central technical question of this paper lies. In math, latent thoughts are useful because the answer is a deterministic function of an algorithmic working memory, so token-level reconstruction of a teacher’s rationale is a natural training target ( 7 ; 28 ) . Safety moderation has no comparable algorithmic substrate: the answer is a function of which clause in a heterogeneous, user-supplied policy is implicated, and of the latent intent behind a possibly disguised request, both paraphrastically variable across deployments. Forcing the latents to reproduce specific tokens of a teacher’s rationale would over-constrain them, while leaving them unsupervised reduces them to opaque computation buffers untethered from policy content. We argue that what a guardrail actually needs is semantic latent deliberation : each latent stage should retain the decision-relevant gist of its reasoning, the user’s underlying intent and the policy clauses it touches, so that a short latent rollout can stand in for a long verbal one without losing policy grounding. This contract, together with the need to anchor verdicts to specific clauses, calls for a guardrail-specific latent design that prior continuous-thought work does not address. We instantiate semantic latent deliberation in the proposed Latent Policy Guardrail (LPG), a framework that reasons deeply but fast over user-supplied policies. The two components below are not independent contributions but coupled instruments of one underlying principle, that every reasoning step, whether explicit text or compressed latent, must commit to specific policy semantics rather than diffuse safety priors: Structured reasoning (Section 4.2 ). Rather than unconstrained free-form rationales, we impose a three-stage evaluation template: (1) intent analysis , examining the conversation history to surface jailbreaks hidden behind role-play, hypotheticals, or indirect phrasing; (2) policy analysis , selectively identifying the subset of relevant clauses rather than enumerating all K items, encouraging semantic matching; and (3) verdict formulation , producing the final decision anchored to the violated indices P^{*} . The format simultaneously grounds reasoning in policy content and yields auditable, machine-parseable verdicts. Latent reasoning tailored to policy grounding (Section 4.3 ). Building on continuous-thought reasoning ( 7 ; 8 ; 1 ) , we compress the intent and policy stages into continuous hidden states. The harder half of the problem is supervising what those states carry; three coupled design choices, none of which arise in math-style continuous-thought reasoning, realize the semantic contract above. (i) Stage-aligned latent slots. Instead of a single undifferentiated continuous-thought buffer, we allocate separate latent budgets to the intent and policy stages, so that the two reasoning sub-tasks live in separable subspaces. (ii) Semantic-content supervision. Rather than reconstructing teacher tokens, we train each slot to preserve the gist of its stage by reconstructing a compact teacher summary , routed through the same base LM that consumes the latents, which directly shapes the latents to be policy-meaningful representations, not opaque computation buffers. (iii) Teacher hidden-state distillation at decision-relevant positions , applied at both stage boundaries and the verdict-onset position, so that the latent pathway converges on the same internal computation that produces the teacher’s explicit decision. The full multi-objective loss is detailed in Section 4.6 . Empirically, semantic latent deliberation delivers a substantially better accuracy–latency Pareto frontier (Sections 5 and 6 ). On in-distribution benchmarks, LPG-4B reaches 84.5\% average safety accuracy and 77.9\% F1, outperforming the strongest dynamic baseline and reasoning baselines while

Read the complete text