Learning Perturbations to Extrapolate Your LLM
Zetai Cen, Chenfei Gu, Jin Zhu, Ting Li, Yunxiao Chen, Chengchun Shi · 2026
Read Learning Perturbations to Extrapolate Your LLM by Zetai Cen, Chenfei Gu, Jin Zhu, Ting Li, Yunxiao Chen, Chengchun Shi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13284.txt.
Introduction
Cen and colleagues improve how language models stretch to new settings by learning to nudge token prefixes. Fixed set nudges limit range, so they nudge prefixes through a learnable map of a smooth latent vector. They derive fair estimating equations for the settings and tune them by stochastic gradient descent. Tests on made up and real data show clear gains outside the training domain over strong baselines.
Catalog record
| CREATOR | Zetai Cen, Chenfei Gu, Jin Zhu, Ting Li, Yunxiao Chen, Chengchun Shi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67726 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13284
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning Perturbations to Extrapolate Your LLM Zetai Cen, Chenfei Gu, Jin Zhu, Ting Li, Yunxiao Chen, Chengchun Shi Abstract Recent advancements in large language models demonstrate that injecting perturbations can substantially enhance extrapolation performance. However, current approaches often rely on discrete perturbations with fixed designs, which limits their flexibility. In this work, we propose a framework where token prefixes are perturbed by a learnable transformation of a continuous latent vector within an embedding space. To overcome the challenge of an intractable marginal likelihood, we derive unbiased estimating equations for model parameters and optimize them via stochastic gradient descent. We establish the statistical properties of the resulting estimator in over-parameterized regimes. Empirical evaluations on both synthetic and real-world datasets demonstrate that our proposal yields significant gains in out-of-domain settings over a range of state-of-the-art baseline methods. Key words and phrases: Estimating equations, intractable marginal likelihoods, large language models, perturbations. 1 Introduction Training large language models (LLMs) such as GPT-5 and Qwen-3 ( 39 ; 51 ) on massive text corpora aims at capturing the underlying distribution of natural language. Yet, it remains challenging for the trained model to extrapolate to out-of-distribution or out-of-domain settings beyond the support of its training data. The literature has seen the development of various data perturbation techniques, such as synonym replacement, random insertion, deletion, and swap, that modify training instances into semantically similar variants to effectively expose LLMs to a broader range of inputs and improve their ability to generalize beyond the training data ( 7 ; 6 ; 19 ; 2 ) . However, their approach remains grounded in the discrete, word-level augmentation procedures mentioned previously, which may restrict its adaptivity across diverse domains. While discrete perturbations are simple to use, they could be too coarse and hard to refine due to the complexity of natural language ( 29 ; 18 ) . Meanwhile, fixed perturbations apply the same transformations to the data regardless of the contexts, thus failing to generalize appropriately ( 12 ) . This paper fills this gap. Our contribution is threefold: (i) A new continuous perturbation framework is proposed for training and prediction of LLMs, which perturbs token prefixes by a trainable transformation of a low-dimensional latent vector within an embedding space. This framework learns the perturbation model jointly with the LLM, enabling adaptivity across tokens and contexts. Extensive experiments show that the proposed framework achieves more stable and effective extrapolation performance across different datasets and base language models. (ii) A new training procedure is introduced, which tackles the challenge brought by the latency of the perturbation. That is, the marginal likelihood of an input prefix is intractable due to complex integrals over latent variables that determine the perturbation, making LLM training based on the likelihood function computationally infeasible. The proposed framework addresses the challenge by deriving an unbiased estimating function whose expectation vanishes at the true parameter and solving the resulting estimating equation by stochastic gradient descent. (iii) Theoretical guarantees are established for the trained model by establishing the statistical consistency of the parameter estimator (Theorem 1 ) and deriving its rate of convergence (Theorem 2 ). These results hold in an over-parametrized setting, where the true parameters are represented by a non-empty set with possibly more than one element. 1.1 Related Works Our work is closely related to: training in LLMs, perturbations in natural language processing (NLP), and score-based approaches. We next discuss the related works in these areas. Training in LLMs. Training LLMs typically consists of two stages. The first is a pre-training stage where the model is trained through autoregressive next - token prediction that learns the conditional distribution of each token given its preceding ones. This is implemented by minimizing the Kullback–Leibler (KL) divergence to the empirical data distribution ( 32 ; 1 ; 54 ) . Different training objectives have been proposed to improve the model robustness, such as the total variation distance ( 16 ) , Earth Mover’s distance ( 36 ) , and mixed forward-reverse cross-entropies ( 53 ) . The second stage is post-training that aligns the model’s behavior with complex human values ( 28 ; 34 ; 22 ; 46 ; 48 ; 45 ) and guides the model to address complex problems through step-by-step reasoning ( 38 ; 8 ; 10 ; 47 ) . This paper focuses on pre-training. Perturbations in NLP. Perturbing language inputs has become a common strategy in NLP. Discrete perturbations can be sentence-level ( 13 ; 37 ) , word-level ( 35 ; 44 ; 52 ; 2 ) , and character-level ( 31 ) , to name but a few types. These operations are mostly heuristic and agnostic to the data and the model, which limits their use in practice. Recent works have partially addressed this by introducing model-aware discrete perturbations ( 29 ) . This remains restricted in a discrete token space, making it difficult to impose smooth perturbations that can generalize across contexts. On the other hand, continuous perturbations are often random noises with a hyperparameter controlling their range ( 56 ; 18 ) , which remains ad-hoc. This work directly builds on this, in that we integrate any base LLM with a parameterized, continuous perturbation process in the embedding space which is jointly learned with the LLM parameters. Score-based approaches in machine learning. Score-based approaches have been used as a flexible alternative to maximum likelihood, particularly when the full likelihood is intractable ( 20 ) . In modern machine learning, they have been adapted to high-dimensional and transfer learning settings ( 40 ; 50 ) . Our construction is close to score - based methods that subtract an expectation under a model to obtain unbiased estimating functions ( 11 ; 9 ) , which have been extended to diffusion models and LLM alignment ( 41 ; 3 ; 24 ) . In our framework, the perturbation process plays a similar role, making the estimation procedure both computationally scalable and statistically principled. For optimization, we directly apply stochastic gradient descent (SGD) to solve the empirical loss derived from the estimating equation, without variational approximations. 2 Methodology We develop an autoregressive language model with a trainable perturbation process. Text tokens are represented by some embedding vectors with ambient dimension d . We assume any input sequence of length L , and shorter inputs are extended to this length by zero padding. The perturbation and next-token prediction proceed as follows. Model . We assume the access of some fixed functions that embed any input token O into a vector X\in\mathbb{R}^{d} , and a decoder that converts X back to O . An input prefix X_{<t}\in\mathbb{R}^{d\times(t-1)} is perturbed by adding a matrix W_{<t}=T_{\beta}(w|X_{<t}) , where w\in\mathbb{R}^{r} is a low - dimensional latent variable drawn from a known distribution (e.g. standard Gaussian) and T_{\beta}(\bullet|X_{<t}):\mathbb{R}^{r}\mapsto\mathbb{R}^{d\times(t-1)} is a neural network parameterized by \beta . The perturbed prefix \widetilde{X}_{<t}=X_{<t}+W_{<t} is then passed through a base autoregressive model P_{\theta} , parameterized by \theta , for predicting the next token embedding X_{t} . Let p be a known density function of w , which we set as standard Gaussian in experiments later. Note that this does not restrict the flexibility of our perturbation process, as the perturbation process will be learned