Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion
Sahil Kadadekar · 2026
Read Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion by Sahil Kadadekar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25097.txt.
Introduction
Speculative decoding speeds up language model inference: a small draft model proposes tokens and the target model checks them. Sahil Kadadekar asks whether draft side behaviour can leak into safety scored outputs when decoding runs at temperature zero. He builds a screen, the Typical Acceptance Invariance Screen, that runs target only and speculative decoding side by side on the same safety battery. The screen demands byte identity evidence, statistical equivalence within a tight margin, and effect sizes below a calibrated cutoff.
Across a confirmatory core and a large matched expansion, the tested stacks show no detectable safety divergence. The largest effect size on refusal behaviour is an order of magnitude below the trivial effect floor. An adversarially tuned draft produces byte identical output to the standard draft across thousands of samples. A change of numerical precision rewrites a large share of output bytes, yet no safety rate moves outside equivalence.
Catalog record
| CREATOR | Sahil Kadadekar |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.25097
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion Sahil Kadadekar Abstract Speculative decoding accelerates inference by letting a draft model propose tokens for a target model to verify, raising a concrete safety question: at temperature zero, can draft-side behavior leak into safety-scored outputs? We answer with Typical-Acceptance Invariance Screen (TAIS), a behavioral-equivalence screen that pairs target-only and speculative outputs on the same safety battery and requires byte-identity evidence, TOST equivalence at ± 3 \pm 3 pp, and per-task Cohen’s h h below a calibrated null cutoff of | h | < 0.1 |h|<0.1{} . Applied to a 16,783-sample confirmatory core plus 44,066 matched expansion samples (fp16/bf16 execution, canonical and DPO-adversarial drafts, GPTQ-4bit drafts, two seeds, and four safety benchmarks), the tested temperature-zero vLLM stacks show no detectable safety divergence under TAIS. The largest absolute Cohen’s h h on matched target-only versus speculative refusal is 0.024, roughly an order of magnitude below the conventional trivial-effect floor; 25 of 27 per-task TOST contrasts pass at the ± 3 \pm 3 pp margin (the two non-pass contrasts are capability-domain Wald-CI edge cases at p target = p spec p_{\text{target}}{=}p_{\text{spec}} at the ceiling, not genuine non-equivalence); the DPO-adversarial draft produces byte-identical output to the canonical draft across 4,006 samples (the expected algorithmic consequence of strict rejection sampling at T = 0 T{=}0 , validated operationally rather than presented as discovery); and bf16 changes 36%–53% of output bytes without moving any per-task safety rate outside equivalence. A separate 4,006-sample 70B production-scale probe, which lacks a matched 70B target-only arm and is therefore not counted as a TAIS pass, produces AdvBench refusal 0.839 over 700 AdvBench completions with 95% Wilson CI [0.809, 0.864]. We make no claim about sampling temperatures, untested frameworks, untested model families, or tree-speculation variants such as EAGLE and Medusa. 1 Introduction Safety can move at several points in the model pipeline. Training-time alignment can trade refusal behavior against other capabilities [ 18 ] , and even benign or adversarial fine-tuning can quickly erode aligned behavior [ 31 ] . Deployment-time compression is another axis: quantization and related methods can change refusal, truthfulness, and bias behavior even when quality metrics remain stable. This paper studies a third axis: inference-time acceleration. Speculative decoding [ 24 , 6 , 33 ] is now common in vLLM [ 22 ] , TensorRT-LLM, SGLang, and HuggingFace TGI, and follow-on work has produced tree variants [ 4 , 25 , 28 , 7 ] , retrieval-based drafting [ 15 ] , and parallel/lookahead schemes [ 12 ] ; Xia et al. 2024 survey the design space. A smaller draft model proposes tokens and a larger target verifies them. Because the draft has its own alignment profile, it is natural to ask whether a weaker or adversarial draft can push unsafe behavior through the acceptance pathway. The closest published security-side result on speculative decoding is Wei et al. 2024 , who exhibit a timing side-channel that lets an observer fingerprint inputs and exfiltrate datastore contents from speculative-decoding traffic; that work demonstrates speculative decoding as a security-relevant surface, but it does not test whether draft-side alignment changes verified safety-scored outputs. Whether the acceptance pathway itself can leak draft-side behaviour into safety-scored verified outputs is the question this paper takes up empirically. We test the greedy operating point. At temperature zero, does speculative decoding change safety-scored outputs relative to target-only decoding on the same inputs? Our answer is a bounded null: in the tested matched vLLM configurations, across Llama and Qwen model families, four safety benchmarks, canonical and adversarial drafts, GPTQ-4bit drafts, two seeds, and fp16/bf16 execution, greedy speculative decoding shows no detectable safety divergence from target-only decoding under the screen defined below. A separate 70B run tests whether the same production-scale speculative stack remains in the expected refusal range, but because it does not include a matched 70B target-only arm, it is reported as a scale sanity probe rather than as a TAIS equivalence cell. Named method: TAIS. We define the Typical-Acceptance Invariance Screen (TAIS) as the behavioral-equivalence screen used to establish the null (see Figure 1 for a protocol overview). Given a speculative-decoding stack and a matched safety battery, TAIS pairs target-only and speculative outputs on identical inputs, measures byte-identity, evaluates TOST equivalence at \pm 3 pp, and computes per-task Cohen’s h . A stack is null-consistent on the tested battery if \max_{t}|h_{t}| is below 0.1 and every well-conditioned per-task TOST contrast passes (capability-domain Wald-CI edge cases at the ceiling, where p_{\text{target}}{=}p_{\text{spec}} , are flagged separately rather than counted as failures). The cutoff is calibrated from this expansion: it sits above the loudest observed null-consistent contrast and below the conventional trivial-effect floor [ 10 ] . TAIS composes paired testing, TOST [ 23 ] , and Cohen’s h into a calibrated equivalence protocol with explicit pass criteria. Evidence scope. The confirmatory core contains 16,783 paired samples on three Llama and Qwen pairs. The expansion adds 48,072 samples across five probes: E1 scales to a Llama-3.1-70B target with an 8B draft as a one-arm production-scale probe; E2 inserts a DPO-adversarial draft trained on flipped Anthropic/hh-rlhf preferences; E3 swaps in a GPTQ-4bit draft; E4 repeats the core pairs over two seeds; and E5 switches the accumulator dtype from fp16 to bf16. Across the matched TAIS probes, every well-conditioned per-task contrast falls inside the \pm 3 pp equivalence bound (25 of 27 in the core ledger; the two non-passing contrasts are capability-domain Wald-CI edge cases at the ceiling), and the largest Cohen’s h observed across matched AdvBench refusal contrasts is 0.024. Contributions. 1. A scoped behavioral null: in the tested temperature-zero setting, greedy speculative decoding shows no detectable safety divergence from target-only decoding under byte-identity, TOST, and Cohen’s h criteria across a 16,783-sample core and 44,066 matched expansion samples, with an additional 4,006-sample 70B scale probe reported separately. 2. TAIS, a reusable equivalence screen with explicit pass criteria and a stated calibration procedure. 3. A direct stress test of the acceptance-pathway leakage concern at the greedy operating point: a DPO-adversarial draft and a GPTQ-4bit draft produce no leakage detectable above the per-cell MDE under the target’s verification step. We frame the byte-identity outcome on the DPO-adversarial draft as operational validation that the Leviathan et al. 2023 distributional guarantee, which collapses to byte-equality at T{=}0 modulo numerical non-associativity, survives in practice on a strongly-perturbed draft, rather than as a novel theoretical discovery. 4. A scope boundary that separates inference-time acceleration from training-time alignment and deployment-time compression: the null applies to temperature zero, vLLM v0.19, two model families, strict rejection sampling plus typical acceptance, and the tested safety battery (full enumeration in § 5 ). 2 Related Work Safety tax across the pipeline. Modern safety alignment is built on RLHF and preference-learning machinery [ 8 , 29 , 2 , 32 ] , yet the resulting alignment is fragile in identifiable ways. Huang et al. 2025 identify a training-time safety tax : adding safety alignment to a reasoning-trained large model reduces reasoning benchmark performance. Qi et al. 2023 make the analogous finding from the fine-tuning side: even benign task-specific fine-tuning on a small set of examples can erode the safety behaviour of an aligned base model, and a separate