Exploring LLM Reasoning Through Controlled Prompt Variations
Giannis Chatziveroglou MIT [email protected] & Richard Yun MIT [email protected] & Maura Kelleher MIT [email protected] · 2025
Read Exploring LLM Reasoning Through Controlled Prompt Variations by Giannis Chatziveroglou MIT [email protected] & Richard Yun MIT [email protected] & Maura Kelleher MIT [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-02111.txt.
Introduction
Giannis Chatziveroglou, Richard Yun, and Maura Kelleher test how well language models keep their logic when prompts get messy in a 2025 paper. Real prompts carry stray facts, odd orders, and side notes. A model that can not sort the wheat from the chaff will slip on such input, no matter how strong its base skills are.
The team bent grade-school math prompts in four set ways and ran 13 top models on them. Stray facts inside the prompt hurt scores the most by a wide gap. The harm did not grow with task steps and did not track model size. Some bends even sparked step-by-step logic on their own. The team urges tougher guards for noisy input.
Catalog record
| CREATOR | Giannis Chatziveroglou MIT [email protected] & Richard Yun MIT [email protected] & Maura Kelleher MIT [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-02111
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploring LLM Reasoning Through Controlled Prompt Variations Giannis Chatziveroglou MIT [email protected] & Richard Yun MIT [email protected] & Maura Kelleher MIT [email protected] Abstract This study investigates the reasoning robustness of large language models (LLMs) on mathematical problem-solving tasks under systematically introduced input perturbations. Using the GSM8K dataset as a controlled testbed, we evaluate how well state-of-the-art models maintain logical consistency and correctness when confronted with four categories of prompt perturbations: irrelevant context, pathological instructions, factually relevant but non-essential context, and a combination of the latter two. Our experiments, conducted on thirteen open-source and closed-source LLMs, reveal that introducing irrelevant context within the model’s context window significantly degrades performance, suggesting that distinguishing essential from extraneous details remains a pressing challenge. Surprisingly, performance regressions are relatively insensitive to the complexity of the reasoning task, as measured by the number of steps required, and are not strictly correlated with model size. Moreover, we observe that certain perturbations inadvertently trigger chain-of-thought-like reasoning behaviors, even without explicit prompting. Our findings highlight critical vulnerabilities in current LLMs and underscore the need for improved robustness against noisy, misleading, and contextually dense inputs, paving the way for more resilient and reliable reasoning in real-world applications. 1 Introduction Question: Claire makes a 3 egg omelet every morning for breakfast. How many dozens of eggs will she eat in 4 weeks? Answer: She eats 3 eggs every day and there are 7 days in a week so she eats 3*7 = <<3*7=21>>21 eggs a week. After 4 weeks she will have eaten 4*21 = <<4*21=84>>84 eggs. There are 12 eggs in 1 dozen and she’ll eat 84 eggs so that’s 84/12 = <<84/12=7>>7 dozen eggs #### 7 Figure 1: Sample question, answer data point from the GSM8K dataset Figure 2: Percentage difference in number of correct answers when evaluating various models with different perturbed prompts compared to baseline performance with the original prompt. The recent surge in large language model capabilities has dramatically advanced the state-of-the-art in natural language understanding and generation. Transformer-based architectures Vaswani 2017 when trained at scale have demonstrated remarkable competence in tasks ranging from machine translation and summarization to more challenging domains like mathematical problem solving, code generation and logical reasoning. However their growing sophistication highlights a crucial question: To what extent can these models reliably perform complex reasoning when confronted with challenging or noisy inputs Mathematical problem-solving has emerged as an important lens through which researchers can study and measure the reasoning abilities of LLMs. Unlike simple factual queries mathematical word problems require the model to decode problem statements, identify relevant information, apply logical and numerical reasoning steps and ultimately arrive at a precise answer. Benchmarks like GSM8K Cobbe 2021 have become central to this effort providing sets of grade-school-level mathematical problems that serve as standardized tests of a model’s multi-step reasoning performance. Additional similar benchmarks include MMLU ( Hendrycks 2021a ) , MATH ( Hendrycks 2021b ) and HellaSwag ( Zellers 2019 ) . Although many models now demonstrate commendable success on carefully constructed, “clean” benchmarks, their reasoning robustness, defined as their ability to handle perturbations, distractions and subtle nudges away from the correct solution path, remains less well understood. As large language models become integrated into real-world applications their resilience to perturbations becomes increasingly important. In the real world information rarely appears clean and isolated. It often arrives tangled in complex, irrelevant details, and sometimes even contains misleading clues. A robust model should be able not only to parse relevant information from distractors but also to maintain coherent logical chains of thought in the face of content noise. Prior work has shown that LLMs can struggle with compositional reasoning Lake 2017 and be sensitive to prompt formatting or small input changes Zhao 2021 . More recent studies on chain-of-thought prompting Wei 2022 have revealed that the structured breakdown of reasoning steps can substantially improve correctness suggesting that LLMs have latent reasoning capabilities that can be tapped with the right prompting strategies. Understanding how these models react to different forms of input perturbation can shed light on their internal reasoning processes and highlight directions for improving both their architectures and their training regimes. This study aims to systematically investigate how various perturbations affect the reasoning capabilities of large language models on established mathematical benchmarks. By employing the GSM8K dataset as a controlled environment, we introduce four categories of perturbations designed to stress-test reasoning under conditions that differ markedly from the neatly formatted baseline prompts. We explore the following perturbation types: 1. Irrelevant Context: Introducing large quantities of extraneous, information-rich text inspired by studies on the brittleness of model reasoning Shi 2023 which the model must ignore to maintain a correct solution path. 2. Pathological: Appending crafted instructions or deceptive linguistic cues to probe the model’s susceptibility to misdirection building on earlier work highlighting models’ sensitivity to misleading prompts Elazar 2021 . 3. Relevant Context: Integrating factually related yet non-essential details to see if and how these additions affect the solution process following prior research on contextual cue utilization in reasoning tasks Dua 2019 . 4. Combo: Assessing whether the simultaneous introduction of multiple perturbations magnifies their effects deepening our understanding of model robustness in more complex scenarios. This work contributes to the growing literature on interpretability, robustness and evaluation methodologies for LLMs. By providing detailed quantitative and qualitative analyses of model outputs under challenging conditions this study illuminates where today’s models falter and guides future efforts to bolster their logical consistency. Ultimately our findings inform both developers and researchers on how to better design, train and prompt large language models so that they excel not only on standard benchmarks but also in the more complex conditions characteristic of real-world applications. Our work can be found and reproduced in the following open-source repository 1 1 1 https://github.com/gchatz22/reasoning-perturbations-study . 2 Related Work 2.1 GSM-Symbolic The GSM-Symbolic benchmark ( Mirzadeh 2024 ) was developed to evaluate the mathematical reasoning capabilities of LLMs. The authors assesses reasoning performance through controlled variations in symbolic templates such as by modifying numerical values or adding/removing clauses in problem statements. Their findings indicate that LLMs often rely on pattern recognition rather than true logical reasoning. Introducing a single clause that appears relevant but does not contribute to the reasoning chain leads to substantial performance declines (up to 65%) across all state-of-the-art models. Thus the GSM-Symbolic benchmark effectively highlights a gap between pattern-based learning and genuine mathematical reasoning underscoring the need for more comprehensive benchmarks to measure reasoning in LLMs. The work on GSM-Symbolic has in many ways inspired the research in this paper. While both studies aim to illuminate the gap between pattern-based learning and authentic mathematical reasoning, our approach diverges by focusing on noise perturbations within the context window, whereas GSM-Symbolic primarily explores the impact of problem statement variations. 2.2 Large Language Models Can Be Easily Distracted by Relevant and Irrelevant Context Shi et. al Shi 2023 explores how LLMs are influenced by irrelevant context by introducing the GSM-IC dataset, an adaptation of GSM8K where extraneous, nonessential sentences are added to the problem statements. The authors found that model performance drops significantly when faced with irrelevant information even on problems whose unperturbed versions they correctly solve. Even one piece of irrelevant information