Improving Complex Reasoning with Dynamic Prompt Corruption: A soft prompt Optimization Approach
Sinan Fan, Liang Xie, Chen Shen, Ge Teng, Xiaosong Yuan, Xiaofeng Zhang, Chenxi Huang Wenxiao Wang, Xiaofei He, Jieping Ye · 2025
Read Improving Complex Reasoning with Dynamic Prompt Corruption: A soft prompt Optimization Approach by Sinan Fan, Liang Xie, Chen Shen, Ge Teng, Xiaosong Yuan, Xiaofeng Zhang, Chenxi Huang Wenxiao Wang, Xiaofei He, Jieping Ye (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13208.txt.
Introduction
Tuning a language model with a small learned prompt is cheap, but on hard reasoning tasks the trick can backfire and drag scores down. Sinan Fan and co-authors dig into why, in a 2025 paper. Tracing the model's inner flow, they find the soft prompt slowly gathers stray information that warps the deeper steps of a chain of thought.
Their fix watches the prompt's effect as reasoning unfolds. When the prompt starts to hurt, the method masks the prompt tokens that clash with the work in progress. Across math and logic test sets, the fix lifted prompt tuning scores by a fair margin on several models. The gain was steady rather than model-specific, which suggests the flaw they found is a general one in how soft prompts age during long reasoning.
Catalog record
| CREATOR | Sinan Fan, Liang Xie, Chen Shen, Ge Teng, Xiaosong Yuan, Xiaofeng Zhang, Chenxi Huang Wenxiao Wang, Xiaofei He, Jieping Ye |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-13208
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Improving Complex Reasoning with Dynamic Prompt Corruption: A soft prompt Optimization Approach Sinan Fan, Liang Xie, Chen Shen, Ge Teng, Xiaosong Yuan, Xiaofeng Zhang, Chenxi Huang Wenxiao Wang, Xiaofei He, Jieping Ye Abstract Prompt-tuning (PT) for large language models (LLMs) can facilitate the performance on various conventional NLP tasks with significantly fewer trainable parameters. However, our investigation reveals that PT provides limited improvement and may even degrade the primitive performance of LLMs on complex reasoning tasks. Such a phenomenon suggests that soft prompts can positively impact certain instances while negatively affecting others, particularly during the later phases of reasoning. To address these challenges, We first identify an information accumulation within the soft prompts. Through detailed analysis, we demonstrate that this phenomenon is often accompanied by erroneous information flow patterns in the deeper layers of the model, which ultimately lead to incorrect reasoning outcomes. we propose a novel method called D ynamic P rompt C orruption (DPC) to take better advantage of soft prompts in complex reasoning tasks, which dynamically adjusts the influence of soft prompts based on their impact on the reasoning process. Specifically, DPC consists of two stages: Dynamic Trigger and Dynamic Corruption. First, Dynamic Trigger measures the impact of soft prompts, identifying whether beneficial or detrimental. Then, Dynamic Corruption mitigates the negative effects of soft prompts by selectively masking key tokens that interfere with the reasoning process. We validate the proposed approach through extensive experiments on various LLMs and reasoning tasks, including GSM8K, MATH, and AQuA. Experimental results demonstrate that DPC can consistently enhance the performance of PT, achieving 4%-8% accuracy gains compared to vanilla prompt tuning, highlighting the effectiveness of our approach and its potential to enhance complex reasoning in LLMs. 1 Introduction Prompt Tuning (PT) ( Lester et al., 2021a ) as a promising parameter-efficient fine-tuning (PEFT) approach demands very few trainable continuous prompt vectors added to the input, achieving competitive performance compared to the full-parameter fine-tuning ( Qin et al., 2021 ; Ding et al., 2023a ) . Previous efforts have demonstrated the superiority of PT by generating high-quality soft prompts ( Shi & Lipani, 2024 ; Yang et al., 2023 ; Qin et al., 2024 ) . Nonetheless, soft prompting underperforms the foundation model on complex reasoning tasks. With a comprehensive investigation, we discover that vanilla prompt tuning provides little performance improvement on complex reasoning tasks and may even lead to negative performance outcomes. From the perspective of individual instances, soft prompts may have positive effects for some instances while potentially posing a negative impact on others. Figure 1 illustrates an example from the GSM8K ( Cobbe et al., 2021 ) : the model can answer the question correctly without a soft prompt, while a soft prompt guides the LLM to an incorrect answer through some intermediate reasoning steps. Therefore, it is crucial to determine whether the effect of soft prompts on reasoning is positive or negative. However, understanding why certain reasoning succeeds while others fail is difficult. To meet the challenge, we intend to explore the mechanisms of soft prompts on the Chain-of-Thought (CoT) ( Wei et al., 2022 ) through a lens of information flow. Neuron saliency score analysis is an important tool for analyzing the information flow during LLM inference ( Dai et al., 2022 ; Wang et al., 2023a ) , allowing to probe the models’ reasoning mechanisms. After a comprehensive investigation across several LLMs and tasks, we find: soft prompts lead to reaching the right answer when the latter reasoning steps put more emphasis on former tokens compared to soft tokens. In contrast, when the soft prompt significantly impacts the latter reasoning steps, the model may reach an incorrect answer. This phenomenon revealed by neuron saliency score analysis is similar to the thinking process of human beings to some extent: humans always recall the background knowledge related to the question and then answer it. As a carrier of task-related knowledge ( Xiao et al., 2023 ) ( Yu et al., 2024 ) , soft prompts can provide beneficial hints to help models understand the question and generate the initial reasoning steps, and then use the question and generated steps to complete the answer. However, if the remaining reasoning steps still attend to the hints, i.e., soft prompts, it may conflict with the question and existing reasoning steps. Furthermore, we discover that soft prompt tokens in certain positions have a strong impact on reasoning, and we can suppress the negative effects of the soft prompts on reasoning by corrupting them. Inspired by the above findings, we propose a novel two-stage method called Dynamic Prompt Corruption (DPC): Dynamic Trigger and Dynamic Corruption . Concretely, in the first stage, we measure the impact factor of the soft prompts on the latter part of the reasoning in the final layer, by which we can differentiate whether the soft prompts can lead to the correct answer; in the second stage, we locate the key token that influences both the question and reasoning steps most in shallow layers, then mitigate the negative impact with key token vector masking and other token sparse masking. To validate our analysis and evaluate the proposed DPC, we conduct comprehensive experiments with multiple LLMs on various tasks. Experimental results show that DPC can consistently improve the overall performance of LLMs such as LLaMA2-13B ( Touvron et al., 2023 ) , LLaMA3-8B ( Dubey et al., 2024 ) , and Mistral-0.2-7B ( Jiang et al., 2023 ) on various reasoning tasks with 4%-8% accuracy enhancement including GSM8K ( Cobbe et al., 2021 ) , MATH ( Hendrycks et al., 2021 ) , AQuA ( Ling et al., 2017 ) compared to pure prompt tuning. These results are in line with our observations and demonstrate the effectiveness of our DPC. Our contributions can be summarized as follows: • We investigate the relationship between the information accumulation in the soft prompts and erroneous information flow patterns through saliency score analysis. Our findings indicate that when the soft prompts influence the later stages of reasoning deeply, the model is more likely to reach an incorrect answer. • Based on the thorough analysis, we propose DPC, an instance-level prompt tuning optimization strategy, which can dynamically identify and mitigate the negative effects caused by the soft prompts. • Experiment results manifest our proposed DPC can improve the vanilla prompt tuning for various LLMs on complex reasoning tasks to a large margin, demonstrating the effectiveness of our findings and the superiority of the proposed method. Figure 1: Input the same question to guide the LLM to answer it. The model was originally able to provide the correct answer, but after adding the soft prompts, it produced an error in reasoning. 2 Information Flow Analysis for Soft Prompts To identify the influential factors in Chain-of-Thought (CoT) reasoning, we aim to analyze the model’s inference process. A complete CoT reasoning instance consists of three primary components: the Prompt \mathbf{P} , the Question \mathbf{Q} , and the Rationale \mathbf{R} . Therefore, we need to examine the interactions among these three components using an appropriate tool. The saliency score, a widely used interpretative analysis tool employed to study the interactions between tokens, integrates gradient and attention values and is particularly effective