Antidistillation Sampling
Yash Savani, Asher Trockman, Zhili Feng, Yixuan Even Xu, Alexander Robey, Marc Finzi, J. Zico Kolter · 2025
Read Antidistillation Sampling by Yash Savani, Asher Trockman, Zhili Feng, Yixuan Even Xu, Alexander Robey, Marc Finzi, J. Zico Kolter (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-13146.txt.
Introduction
Yash Savani and co-authors propose a way for top AI firms to share less of what makes their models strong. Their 2025 paper notes that long chains of thought from a top model can be used to train a cheaper copy. That copying is called distillation, and model owners may wish to blunt it. Their method tweaks how the model picks its next word as it writes its chain of thought.
The tweaks poison the trace for training use while the model still answers well in daily use. Owners keep the value of their model's work, but rivals gain less from its traces. The authors point readers to a project site for full detail on how the sampling works.
Catalog record
| CREATOR | Yash Savani, Asher Trockman, Zhili Feng, Yixuan Even Xu, Alexander Robey, Marc Finzi, J. Zico Kolter |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-13146
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Antidistillation Sampling Yash Savani, Asher Trockman, Zhili Feng, Yixuan Even Xu, Alexander Robey, Marc Finzi, J. Zico Kolter Abstract Frontier models that generate extended reasoning traces inadvertently produce token sequences that can facilitate model distillation. Recognizing this vulnerability, model owners may seek sampling strategies that limit the effectiveness of distillation without compromising model performance. Antidistillation sampling provides exactly this capability. By strategically modifying a model’s next-token probability distribution, antidistillation sampling poisons reasoning traces, rendering them significantly less effective for distillation while preserving the model’s utility. Our code is available at https://github.com/locuslab/antidistillation-sampling . 1 Introduction Large language models (LLMs) trained to produce reasoning traces have achieved strong performance on math, coding, and reasoning benchmarks [ 1 , 2 , 3 ] . These traces, however, serve a dual purpose. They not only enhance model performance, but also enable distillation, a process by which a secondary model replicates the original model’s capabilities by training on its generations [ 4 , 5 , 6 ] . Notably, distillation can result in substantial capability gains at a fraction of the computational cost needed to train similarly performant models from scratch. While effective and efficient, the viability of distillation poses several downsides for companies deploying frontier reasoning models. First, returning reasoning traces represents a forfeiture of intellectual property, which can allow competitors to cheaply replicate frontier capabilities. Second, the threat of distillation incentivizes limiting user access by obscuring token probabilities or truncating reasoning traces. Finally, model safety is often not preserved by distillation, which enables the generation of harmful content [ 7 , 8 ] . To address these issues, we introduce antidistillation sampling (see Figure 1 ). The main idea underpinning antidistillation sampling is to adjust a model’s sampling distribution so that generated traces maintain high likelihood under the unadjusted distribution, and distillation attempts are simultaneously poisoned. To operationalize this idea, we first formulate the general problem of poisoning reasoning models trained via distillation. We then derive one solution to this problem (see Algorithm 1 ), which facilitates a precise trade-off between two competing objectives—the utility of the original model and the effectiveness of distillation poisoning—while incurring minimal computational overhead. To illustrate our empirical results, consider a reasoning model that achieves 72% accuracy on MMLU. Naively distilling this model using greedy sampling can produce a student that reaches up to 52% accuracy. If the teacher model increases its sampling temperature, its accuracy slightly decreases (e.g., by 4%), yet the distilled student’s accuracy remains largely unchanged at 52%. In contrast, using antidistillation sampling with the same 4% reduction in teacher accuracy significantly reduces the student’s capabilities, lowering its accuracy to as low as 40%. Our findings on GSM8K [ 9 ] , MATH [ 10 ] , and MMLU [ 11 ] indicate that model owners can effectively limit distillation quality via antidistillation sampling. Figure 1: Reasoning traces generated via antidistillation sampling poison distillation attempts while simultaneously preserving the teacher’s performance. The teacher’s logits are perturbed in a direction \Delta , leading to samples that significantly degrade distilled model performance relative to naive temperature sampling. For more details, see Figure 2 and § 3 . 2 Related work Model distillation. The prominence and effectiveness of model distillation—and, more generally, model compression [ 12 , 13 , 14 ] —is rooted in a rich literature dating back to Schmidhuber [5] , and, more recently, to Hinton et al. [4] . Since these seminal works, a growing body of literature has sought to benchmark the performance of distilled models and to algorithmically maximize the effectiveness of distillation [ 15 , 16 , 17 , 18 , 19 ] . Indeed, a variety of frontier AI labs have incorporated distillation as a core technique, both to efficiently enable frontier capabilities [ 20 , 21 ] and to improve model safety via context distillation [ 22 , 23 , 24 ] . However, this practice constitutes a strategic vulnerability for frontier model maintainers, given the demonstrated value of these reasoning traces (see, e.g., [ 1 , 25 , 26 , 27 ] ). Model security. The setting we address in this paper—where a student model is trained on data generated by a teacher model—intersects with several aspects of model security. For instance, model extraction attacks acquire weights via query-level access, whereas training data extraction attacks are designed to harvest training data [ 28 , 29 ] . While antidistillation sampling may offer some protection against these attacks, such analysis remains beyond our scope. More relevant is the literature on data poisoning, where maliciously crafted data is injected into a model’s training set to induce specific downstream effects (see, e.g., [ 30 ] ). In this vein, Rando and Tramèr [31] show the effectiveness of adding backdoors to preference data, sabotaging LLMs finetuned with RLHF. Our contribution bridges data poisoning and privacy techniques to protect the valuable knowledge encoded in frontier models. Distillation prevention. A related line of work has sought to develop algorithms that prevent distillation. In the context of computer vision, Ma et al. [32] corrupt the teacher’s logits via self-training, whereas follow-up work shows that returning only the top- k logits tends to harm distillation [ 33 ] . Also related are watermarking algorithms, which seek to adjust model logits to detect whether a model has been distilled [ 34 , 35 , 36 , 37 ] . And while this family of methods preserves teacher accuracy, their static nature—each input generally yields a deterministic logit vector—presents security vulnerabilities: a distiller can learn an inverse transformation by saving input-output pairs, and thereby fine-tune to recover uncorrupted logits. To mitigate this shortcoming, Chen et al. [38] propose a session-dynamic defense that monitors the sensitivity of a user’s queries and perturbs the logits once a threshold is crossed. In contrast, antidistillation sampling is fully dynamic ; it perturbs each token’s distribution on-the-fly using gradients from a hidden proxy model, turning generation into a moving target in a similar fashion to cryptographic stream ciphers. Language model decoding. Finally, we position antidistillation sampling within the broader framework of controlled decoding for LLMs [ 39 ] , where supplementary objectives steer the decoding process. Existing approaches in this domain include using contrastive objectives to enhance generation quality [ 40 ] , reformulating constrained decoding as an optimization problem [ 41 ] , and incorporating energy-based constraints [ 42 ] . While related, antidistillation sampling solves a different problem: by implementing a new, distillation-aware penalization term in the decoding objective, our approach poisons generated reasoning traces to undermine the performance of models fine-tuned on these outputs. 3 Antidistillation sampling Algorithm 1 Antidistillation sampling Input: Prompt x_{1:n} , max tokens N , penalty multiplier \lambda , approximation parameter \epsilon , temperature \tau 1. (Initialization) Compute the gradient of the downstream loss g\leftarrow\nabla\ell(\theta_{P}) 2. For each token index t=n,n+1,\ldots,N-1 : i. Compute the antidistillation penalty term \widehat{\Delta}(\>\cdot\>|x_{1:t})\leftarrow\frac{\log p(\>\cdot\>|x_{1:t};\theta_{P}+\epsilon g)-\log p(\>\cdot\>|x_{1:t};\theta_{P}-\epsilon g)}{2\epsilon} ii. Sample the next token x_{t+1} from the teacher’s adjusted distribution x_{t+1}\sim\frac{1}{Z}\exp\left(\frac{1}{\tau}\log p(\>\cdot\>|x_{1:t};\theta_{T})+\lambda\widehat{\Delta}(\>\cdot\>|x_{1:t})\right) Output: Sampled sequence x_{1:N} To motivate antidistillation sampling, we first sketch a high-level overview of our problem setting in § 3.1 . Based on this setting, we provide a desiderata outlining the desired qualities for poisoning distillation attempts in § 3.2