From Noise to Diversity: Random Embedding Injection in LLM Reasoning

Heejun Kim, Seungpil Lee, Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Taesup Kim, Sundong Kim, [email protected] · 2026

Read From Noise to Diversity: Random Embedding Injection in LLM Reasoning by Heejun Kim, Seungpil Lee, Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Taesup Kim, Sundong Kim, [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11936.txt.

Introduction

Heejun Kim and seven colleagues test whether soft prompt gains come from learned content or from injection itself. Their Random Soft Prompts append freshly drawn random embedding vectors to the input with no training at all, yet match tuned soft prompts on math reasoning tasks in several settings. Early tokens flatten and reasoning branches, then generation commits to one completion as the effect dilutes. The same trick widened Pass@N with temperature sampling and improved DAPO training.

Catalog record

CREATORHeejun Kim, Seungpil Lee, Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Taesup Kim, Sundong Kim, [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85487
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.11936

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Noise to Diversity: Random Embedding Injection in LLM Reasoning Heejun Kim, Seungpil Lee, Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Taesup Kim, Sundong Kim, [email protected] Abstract Recent soft prompt research has tried to improve reasoning by inserting trained vectors into LLM inputs, yet whether the gain comes from the learned content or from the act of injection itself has not been carefully separated. We study Random Soft Prompts (RSPs), which drop the training step entirely and append a freshly drawn sequence of random embedding vectors to the input. Each RSP vector is sampled from an isotropic Gaussian fitted to the entrywise mean and variance of the pretrained embedding table; the sequence carries no learned content, and yet reaches accuracy comparable to optimized soft prompts on math reasoning benchmarks in several settings. The mechanism unfolds in two stages: because attention has to absorb a never-seen-before random position, the distribution over the first few generated tokens flattens and reasoning trajectories branch , and as generation continues this influence dilutes naturally so the response commits to a single completion. We show that during inference RSPs lift early-stage token diversity and, combined with temperature sampling, widen Pass@ N N , the probability that at least one out of N N attempts is correct. Beyond inference, we carry the same effect into DAPO training and demonstrate practical gains. Our contributions are: (i) RSP isolates the simplest form of soft prompt — training-free, freshly resampled — providing a unified lens for the structural effect of injection that variants otherwise differing in training and form all share; (ii) a theoretical and empirical validation of the underlying mechanism; and (iii) an extension from inference to training. Code: github.com/heejunkim00/RSP . 1 Introduction As LLMs scale to billions of parameters, full fine-tuning is prohibitive, motivating parameter-efficient methods ( Hu et al. 2022 ; Houlsby et al. 2019 ) . Among these, soft prompts insert learnable continuous embeddings into the input and steer behavior through attention ( Li and Liang 2021 ; Lester et al. 2021 ) ; the paradigm is actively adopted for mathematical reasoning ( Kang et al. 2026 ; Xu et al. 2025 ; Ye et al. 2026 ; Hao et al. 2025 ; Zhang et al. 2026 ) . These methods diverge in injection position, training objective, and prompt form, and prior work attributes their gains to the learned content of the optimized vectors. The act of injecting embeddings into the input, common to all variants, has remained outside the analysis. Even though theory shows trained prefixes only bias attention in a fixed direction ( Petrov et al. 2024 ) , the source of the effect itself remains unsettled. Figure 1 : Conceptual overview of RSP-induced trajectory diversity. The hidden states shown are the final-layer outputs that drive the next-token distribution. (a) Early decoding: RSP perturbation reaches a branching state (red; formalized in § 3.1 ), whose top- K differs from the no-RSP baseline and induces a diverse output distribution. (b) Later decoding: as the KV cache grows, the bound on RSP attention mass narrows (Theorem 1 ); branching events become rare and the hidden state (blue) commits to the branch already opened. (c) Across rollouts, the trajectories opened by early RSP perturbation accumulate into independent paths. To separate learned content from the act of injection , we use Random Soft Prompts (RSPs) as a control — continuous vectors drawn from an isotropic Gaussian fitted to the embedding table’s mean and variance, with no learning. The control turns out to be non-neutral. Applying a chat template to a base model not fine-tuned for it degrades reasoning, yet appending RSPs recovers up to +29 pp on Qwen2.5-Math-1.5B; if simple noise were merely shaking the model, accuracy should have worsened instead. The injection itself, independent of any learned content, alters model behavior. We analyze training-free RSPs on LLM reasoning. With each rollout receiving an independent draw, a single RSP injection branches hidden states into diverse trajectories early on, which then stabilize as decoding proceeds. Empirically, RSPs share the early-decoding entropy signature of optimized methods (LTPO, TTSV, SoftCoT) despite carrying no learned content; Pass@ N accumulates only under independent resampling; and the same input-side diversity composes with DAPO ( Yu et al. 2025 ) training. Our contributions are: (i) RSP isolates the simplest form of soft prompt — training-free and freshly resampled per rollout — providing a unified lens for the structural effect of injection across variants that otherwise differ in training objective and prompt form, while still reaching accuracy comparable to optimized methods in several settings; (ii) a theoretical and empirical account of the mechanism: attention-mediated branch opening, isotropic coverage across all directions, and the automatic attenuation of RSP attention as the KV cache grows; and (iii) DAPO extension showing independent RSP resampling composes with rollout-diversity training. 2 Background 2.1 Soft Prompt Methods for LLM Reasoning Prior soft prompt methods share a common structure: they inject continuous vectors that do not correspond to discrete vocabulary tokens into the model’s representation stream and optimize them under task-specific objectives. As parameter-efficient alternatives to full fine-tuning, Prefix-Tuning ( Li and Liang 2021 ) , Prompt Tuning ( Lester et al. 2021 ) , and P-tuning ( Liu et al. 2024 ) reached near-fine-tuning performance using learnable continuous vectors, after which LLM-reasoning variants followed. TTSV ( Kang et al. 2026 ) optimizes prefix embeddings at test time via trajectory entropy minimization; SoftCoT ( Xu et al. 2025 ) replaces explicit chain-of-thought tokens with soft tokens generated by an assistant model; LTPO ( Ye et al. 2026 ) optimizes the soft prompt per problem to maximize its own confidence on that problem; COCONUT ( Hao et al. 2025 ) feeds its hidden states back as input embeddings; MemGen ( Zhang et al. 2026 ) uses embeddings as experience memory; and Pause tokens ( Goyal et al. 2024 ) insert learnable tokens to provide additional computation steps. These methods differ in position, training stage, and training target, yet share a common assumption: that the learned representations are the core driver of the effect. This paradigm faces several limitations. (i) Training is domain/benchmark-specific, requiring re-optimization for new settings and sensitive to seed/hyperparameter choices. (ii) Several have been observed to degrade on out-of-distribution challenging tasks ( Ye et al. 2026 ) . (iii) Evidence for the effect is largely empirical, with limited theoretical explanation beyond the training objective. Furthermore, soft prompting and prefix-tuning are theoretically restricted to biasing attention outputs in a fixed direction, without changing the relative attention pattern among content tokens ( Petrov et al. 2024 ) . Together, these findings cast doubt on whether the learned prompt vectors function as intended: they are suitable for eliciting or combining skills in the pretrained model, but they struggle to learn new capabilities. Moreover, prior works have not separated the contribution of injection itself from that of optimization . In light of limitations (i)–(iii), this paper focuses on injection itself : we introduce Random Soft Prompts (RSPs) — vectors drawn from an isotropic Gaussian fitted to the pretrained embedding statistics, with the training stage entirely removed — and show that they produce output-distribution shifts and accuracy comparable to those of learned soft prompts, thereby disentangling

Read the complete text