Reasoning Quality Emerges Early: Data Curation for Reasoning Models

Hongyi Henry Jin, Wenhan Yang, Meysam Ghaffari, Carlos Morato, Baharan Mirzasoleiman · 2026

Read Reasoning Quality Emerges Early: Data Curation for Reasoning Models by Hongyi Henry Jin, Wenhan Yang, Meysam Ghaffari, Carlos Morato, Baharan Mirzasoleiman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26797.txt.

Introduction

Hongyi Henry Jin and four colleagues show that the quality of reasoning training data reveals itself in its first few tokens. Fine-tuning on a small set of strong reasoning traces teaches language models to reason well. Curating that set usually needs expensive checks by stronger models. The first tokens of a trace, the authors find, already carry the signal.

Hard problems show up in the loss of the first hundred reasoning tokens at a slightly perturbed model checkpoint. Loss measures how surprised the model is. Examples with similar early loss patterns provoke similar learning gradients, a claim the authors prove. Fine-tuning two open models on medical and mathematics reasoning data followed. Their method beat existing curation baselines slightly while using far fewer tokens. The curation cost falls sharply as a result.

Catalog record

CREATORHongyi Henry Jin, Wenhan Yang, Meysam Ghaffari, Carlos Morato, Baharan Mirzasoleiman
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.26797

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Reasoning Quality Emerges Early: Data Curation for Reasoning Models Hongyi Henry Jin, Wenhan Yang, Meysam Ghaffari, Carlos Morato, Baharan Mirzasoleiman Abstract Supervised fine-tuning (SFT) on a small, high-quality set of long reasoning traces is an effective approach for eliciting strong reasoning capabilities in Large Language Models (LLMs). However, existing methods for curating high-quality SFT data rely heavily on strong reasoning models to filter examples based on diversity and difficulty, making the curation process costly while often yielding suboptimal data quality. In this work, we show that diverse and challenging reasoning examples can be identified using only the initial reasoning tokens. Specifically, we demonstrate that difficult problems can be reliably detected based on the loss of the first 100 reasoning tokens evaluated at a randomly perturbed checkpoint of the pretrained model. We further show that examples exhibiting similar loss patterns over their first 1k reasoning tokens across a small number of perturbed checkpoints extrapolating along the fine-tuning trajectory provably induce similar gradients. We validate our approach through extensive experiments on fine-tuning Qwen2.5-7B and Llama3.1-8B models on the M23K medical reasoning and OpenThoughts-Math datasets. Our method outperforms existing baselines by up to 1.7% while being 91% more token efficient. 1 1 1 Our repo is available at https://bigml-cs-ucla.github.io/TEMP-project-page/ \svgsetup inkscapelatex=false 1 Introduction Recent large language models, such as DeepSeek-R1 ( Guo et al. 2025 ) and o3 ( OpenAI 2024 ) , exhibit strong performance on reasoning-intensive benchmarks spanning mathematics, programming, and scientific domains. These capabilities are induced through post-training a strong pretrained model with either reinforcement learning (RL) on very large data, or supervised fine-tuning (SFT) on a smaller set of examples with long reasoning traces. This encourages the model to generate explicit intermediate reasoning steps at inference time. The resulting long-form reasoning traces, often referred to as chains of thought (CoT) or “thinking tokens,” have been shown to substantially improve solution accuracy ( Javanmard et al. 2026b ) . Figure 1 : Overview of TEMP: Token-Efficient Model Perturbation Reasoning Data Selection. First, using only the first 100 tokens of reasoning traces, we identify challenging examples as those exhibiting higher loss at a randomly perturbed checkpoint of the pretrained model (Sec. 3.1 ). Then, we cluster examples based on their first 1k token loss values, measured at a small number of noisy checkpoints extrapolating along the fine-tuning direction, and sample a diverse subset (Sec. 3.2 ). The curated difficult and diverse SFT dataset enables efficient training of high-performing reasoning models. In contrast to RL, where scale often plays a more significant role than data quality, SFT is highly sensitive to the quality of the reasoning traces. Indeed, naively scaling SFT data without quality control can substantially degrade performance ( Akter et al. 2025 ; Javanmard et al. 2026a ) . High-quality SFT datasets are commonly characterized by diversity and difficulty, i.e., examples that are challenging for the target model ( Muennighoff et al. 2025 ; Guha et al. 2025 ) . To meet these criteria, reasoning traces are either manually curated or generated by strong reasoning models, such as DeepSeek-R1, and subsequently filtered to promote diversity and difficulty ( Muennighoff et al. 2025 ; Guha et al. 2025 ; Huang et al. 2025 ) . Diversity is often enforced by categorizing data using more capable LLMs. Difficulty-based filtering prompts an LLM (e.g., GPT-4o-mini) to estimate the difficulty of each question and retains only the hardest instances, or selects questions that elicit the longest reasoning traces. However, the extensive LLM-based filtering is very expensive and often yields suboptimal results. In this work, we show that, perhaps surprisingly, both the diversity and difficulty of reasoning traces can be inferred from the initial portion of the chain-of-thought (CoT). This observation enables early identification and efficient elimination of non-informative reasoning traces, thereby obviating the need for expensive post hoc filtering. First, we demonstrate that, using only the first 100 tokens out of reasoning trace of length up to 91k tokens, challenging examples can be reliably identified as those exhibiting higher loss at a randomly perturbed checkpoint of the pretrained model. We then establish that, during SFT—where model parameters undergo relatively small changes—examples with similar loss values over their first 1k tokens, measured at a small number of noisy checkpoints extrapolating along the fine-tuning direction, induce similar gradients throughout training. Our Token-Efficient Model Perturbation (TEMP)-based data selection method curates an SFT dataset that is both diverse and challenging, enabling efficient training of high-performing reasoning models. We conduct extensive experiments for fine-tuning Qwen2.5-7B ( Qwen et al. 2025 ) and Llama3.1-8B ( Dubey et al. 2024 ) on M23K medical reasoning ( Huang et al. 2025 ) and OpenThoughts ( Guha et al. 2025 ) datasets and show that our approach outperforms existing baselines by up to 1.7% while being 91% more token efficient. 2 Related works LLM reasoning capabilities can be effectively induced using small, carefully curated sets of reasoning traces. This result has been verified in different settings, such as math reasoning ( Muennighoff et al. 2025 ; Ye et al. 2025 ) , medical reasoning ( Huang et al. 2025 ) , safety alignment ( Wang et al. 2025 ) , and instruction tuning ( Li et al. 2024b ; Liu et al. 2024 ) and with different model architectures ( Zhao et al. 2025 ) . In particular, difficulty and diversity are widely recognized as key attributes of high-quality reasoning data. ⬇ Q1 (Easy question): Write the decomposition of the vector \(\ vec { a }\) in terms of the vectors \[ \ vec { p }=\{1,2,4\}, \ quad \ vec { q }=\{1,-1,1\}, \ quad \ vec { r }=\{2,2,4\}, \ quad \ vec { a }=\{-1,-4,-2\}. \] A: Okay , so I need to decompose the vector \(\ vec { a } = \{-1, -4, -2\}\) in terms of the vectors \(\ vec { p } = \{1, 2, 4\}\), \(\ vec { q } = \{1, -1, 1\}\), and \(\ vec { r } = \{2, 2, 4\}\). ------------------------------ Q2 (Difficult Question): In a football tournament , each team is supposed to play one match against each of the other teams . However , during the tournament , half of the teams were disqualified and did not participate further . As a result , a total of 77 matches were played , and the disqualified teams managed to play all their matches against each other , with each disqualified team having played the same number of matches . How many teams were there at the beginning of the tournament ? A: Okay , let me try to figure out this football tournament problem . So , the question is : There was a tournament where each team was supposed to play against every other team once . But halfway through , half the teams got disqualified . In the end , a total of 77 matches were played . The important points are that the disqualified teams played all their matches against each other before being disqualified , and each disqualified team played the same number of matches . We need Figure 2 : Problem understanding phase in the first 100 response tokens. For the easy problem, the model only

Read the complete text