Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models

Tingchen Fu, Jiawei Gu, Yafu Li, Xiaoye Qu, Yu Cheng · 2025

Read Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models by Tingchen Fu, Jiawei Gu, Yafu Li, Xiaoye Qu, Yu Cheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14810.txt.

Introduction

Tingchen Fu and co-authors test in 2025 whether models that reason longer still obey the user's orders. Following instructions is core to making language models serve their users, yet training that boosts step-by-step reasoning may erode that obedience. The team built MathIF, a test set that pairs maths tasks with explicit directives.

Across models, stronger reasoning often came with weaker rule following, above all in long answers. Models trained on distilled long thought chains or on reasoning rewards slipped most. Simple fixes won back some obedience but cost reasoning power. The authors call for training that guards both skills at once.

Catalog record

CREATORTingchen Fu, Jiawei Gu, Yafu Li, Xiaoye Qu, Yu Cheng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-14810

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models Tingchen Fu, Jiawei Gu, Yafu Li, Xiaoye Qu, Yu Cheng Abstract Instruction-following is essential for aligning large language models (LLMs) with user intent. While recent reasoning-oriented models exhibit impressive performance on complex mathematical problems, their ability to adhere to natural language instructions remains underexplored. In this work, we introduce MathIF , a dedicated benchmark for evaluating instruction-following in mathematical reasoning tasks. Our empirical analysis reveals a consistent tension between scaling up reasoning capacity and maintaining controllability, as models that reason more effectively often struggle to comply with user directives. We find that models tuned on distilled long chains-of-thought or trained with reasoning-oriented reinforcement learning often degrade in instruction adherence, especially when generation length increases. Furthermore, we show that even simple interventions can partially recover obedience, though at the cost of reasoning performance. These findings highlight a fundamental tension in current LLM training paradigms and motivate the need for more instruction-aware reasoning models. We release the code and data at https://github.com/TingchenFu/MathIF . 1 Introduction Recent advancements in Large Reasoning Models (LRMs) qu2025survey , such as o3 and o4-mini o3 , DeepSeek-R1 deepseek-r1 , and K1.5 kimi , have demonstrated impressive capabilities in mathematical reasoning, including solving olympiad-level problems OlympiadBench ; math500 ; aime_1983_2024 and automating formal theorem proving deepseek-math-prover . These breakthroughs have sparked growing interest in scaling chain-of-thought (CoT) reasoning cot , where models produce explicit multi-step explanations to solve complex tasks. Typical approaches include imitation learning, e.g., supervised fine-tuning (SFT), and reinforcement learning with verifiable rewards rl-vr , both of which aim to strengthen model intelligence across various tasks and scales. Despite these advances, instruction following, i.e., the ability to accurately and reliably comply with user directives, has received comparatively little attention in the context of LRMs. Yet this ability is critical for real-world alignment and safety surveyllmasajudge . Our empirical evaluations on IFEval ifeval and FollowBench followbench reveal a consistent pattern: although LRMs excel at mathematical reasoning, they often fail to follow even simple instructions. This raises an important question: As reasoning scales, do models become more intelligent yet less controllable? Unfortunately, existing instruction-following benchmarks are ill-suited for answering this question. Most are designed for general-purpose language models and lack coverage of math-specific reasoning behaviors. In contrast, LRMs are typically trained on math-heavy datasets and optimized specifically for problem-solving capacity. This gap highlights the urgent need to evaluate whether increasing intelligence in specialized reasoning models inherently leads to diminishing control over their behavior, an issue at the heart of instruction alignment for advanced LRMs. To address this, we propose MathIF , the first benchmark specifically designed to evaluate the instruction-following capabilities of LRMs in math domains. MathIF introduces 15 Python-verifiable constraints across 4 categories, which are programmatically combined to create 30 double-constraint and 15 triple-constraint prompts. These are applied to math problems of varying difficulty, resulting in a total of 420 high-quality evaluation samples. Using MathIF, we evaluate 23 LRMs across a wide range of scales and architectures. Surprisingly, most models fail to reliably follow instructions, and performance does not consistently improve with larger model sizes. Even the best-performing model, Qwen3-14B, achieves only 50.71% accuracy on strict instruction-following. Furthermore, performance deteriorates with increasing task difficulty and constraint complexity, revealing substantial headroom for improvement. Our deeper analysis further uncovers a mutual interference between instruction-following and reasoning capabilities, observed at both training and inference stages. Common reasoning-oriented training strategies (e.g., SFT and RL) enhance reasoning ability but degrade instruction adherence. This degradation becomes more pronounced as the CoT length increases, likely because longer reasoning paths widen the contextual gap between the original instruction and the final answer, making it harder for the model to retain and execute directives. Conversely, enforcing brevity by limiting CoT length improves instruction-following performance, but at the cost of reasoning depth and accuracy. These observations reveal a consistent pattern: improving reasoning capability often comes at the cost of instruction adherence , suggesting an inherent trade-off between the two abilities. This trade-off highlights a crucial challenge in LRM development: training for stronger reasoning alone may undermine alignment, and future methods must account for this tension to build models that are both capable and controllable. To summarize, our contributions are three-fold: • We introduce MathIF, the first benchmark for systematically measuring instruction-following performance in large reasoning models within math domains. • We evaluate 23 recent LRMs and uncover a widespread inability to follow user constraints, especially on harder problems and multi-constraint settings. • We identify and empirically validate a trade-off between reasoning performance and instruction-following ability, with mutual interference observed during both training and inference. 2 Related Work 2.1 Large Reasoning Models (LRMs) Recent advances in enhancing the reasoning ability of language models and reimplementing large reasoning models generally fall into two paradigms. The first paradigm constructs high-quality long CoT by distilling from more capable LRMs or combining primitive reasoning actions 2025s1 ; deepseek-r1 , on which SFT and subsequent RL are conducted. For example, s1 2025s1 shows that even a small amount of CoT data could significantly promote the reasoning ability, and LIMO ye2025limoreasoning further put forward the less-is-more reasoning hypothesis, stating that sophisticated reasoning capabilities can emerge through minimal demonstration of cognitive processes if relevant domain knowledge is already encoded in the language model during pre-training. Similarly, yeo2025demystifying suggests that injecting more cognitive behavior into the long CoT during SFT brings promotion in mathematical benchmarks. On the other hand, cold-RL on base language models directly attracts more and more attention in the subfield because of the success of deepseek-R1-zero and the previous findings that the model tends to memorize training data during the SFT process chu2025sft . In contrast with SFT, cold-RL does not rely on long CoT data and provides supervision signals by rewards on the final outcome deepseek-r1 or the reasoning process liu2025oatzero . To simplify and accelerate the RL process, various techniques have been proposed, such as dynamic sampling yu2025dapo , process-reward prime , off-policy guidance luffy , and CoT preference optimization yang2025thinking . To compare two different pathways and evaluate the capacity of LRMs yeo2025demystifying , previous studies mostly focus on their complex reasoning performance on olympiad-level math problems, while their instruction-following ability when solving math problems is left unconsidered. 2.2 Instruction-following benchmarks As a crucial factor determining the practicality of a language model for real-world scenarios, the instruction-following ability is a core metric for language model evaluation, with numerous protocols and benchmarks being developed dubois2023alpacafarm ; chiang2023vicuna . Earlier benchmarks primarily focused on the completeness of user queries and depended on proprietary language models dubois2023alpacafarm ; chiang2023vicuna to measure the response quality by its win-rate over the baseline method, which is an oversimplification of real user queries. For a more comprehensive evaluation, sophisticated benchmarks have been developed to test the ability of a language model in following format constraints ifeval ; xia2024fofo ; tang2024struc , multi-turn instruction he2024multi ; li2025structflowbench ; han2025can ; sirdeshmukh2025multichallenge , long-context instruction wu2024lifbench , multi-lingual instruction he2024multi ; li2025xifbench , compositional instruction zhang2025iheval ; hayati2025chain ; han2025can and refutation instructions yan-etal-2024-refutebench ; yan2025refutebench20agentic . However, most instruction-following benchmarks concentrate on the general domain and relatively straightforward queries. The domain shift and the lack of long

Read the complete text