Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation
Kaiyuan Liu, Shaotian Yan, Rui Miao, Bing Wang, Chen Shen, Jun Zhang, Jieping Ye · 2025
Read Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation by Kaiyuan Liu, Shaotian Yan, Rui Miao, Bing Wang, Chen Shen, Jun Zhang, Jieping Ye (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-20908.txt.
Introduction
Kaiyuan Liu and six co-authors trace where a distilled model's sentences actually come from. In reasoning distillation, a big teacher model writes out solution paths and a small student trains on them. Whether the student truly absorbs the teacher's habits, or slips back to its old style on new problems, was unclear.
For every sentence the distilled model produced, the team compared probabilities from the teacher, the original student, and the distilled model. Sorting sentences by those comparisons showed genuine teacher-born behavior appearing in fresh test settings. The same tracing suggests which training examples to keep: those where teacher and student diverge most instructively.
Catalog record
| CREATOR | Kaiyuan Liu, Shaotian Yan, Rui Miao, Bing Wang, Chen Shen, Jun Zhang, Jieping Ye |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-20908
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation Kaiyuan Liu, Shaotian Yan, Rui Miao, Bing Wang, Chen Shen, Jun Zhang, Jieping Ye Abstract Reasoning distillation, a cost-effective approach for enhancing student model performance, has attracted increasing attention. It typically leverages a large teacher model to generate reasoning paths, which are then used to fine-tune a student model so that it mimics the teacher’s behavior in training contexts. However, previous approaches have lacked a detailed analysis of the origins of the distilled model’s capabilities. It remains unclear whether the student can maintain consistent behaviors with the teacher in novel test-time contexts, or whether it regresses to its original output patterns, raising concerns about the generalization of distillation models. To analyse this question, we introduce a cross-model Reasoning Distillation Provenance Tracing framework. For each action (e.g., a sentence) produced by the distilled model, we obtain the predictive probabilities assigned by the teacher, the original student, and the distilled model under the same context. By comparing these probabilities, we classify each action into four categories: (i) teacher-originated actions, (ii) student-originated actions, (iii) pre-existing actions in both models not enhanced by distillation, and (iv) pre-existing actions boosted through distillation. By systematically disentangling the provenance of each action, we experimentally demonstrate that, in test-time contexts, the distilled model can indeed generate teacher-originated actions, which correlate with and plausibly explain observed performance on distilled model. Building on this analysis, we further propose a teacher-guided data selection method. Unlike prior approach that rely on heuristics (e.g., selecting data most aligned with the student’s original distribution), our method directly compares teacher–student divergences on the training data, providing a principled selection criterion. We validate the effectiveness of our approach across multiple representative teacher models (Deepseek, QwQ, GPT-OSS-120B) and diverse student models (Qwen2.5-7B-Instruct, Qwen4-4B-Base, Qwen3-8B-Base, Qwen3-4B-Instruct-2507). The results highlight the utility of our provenance-tracing framework and underscore its promise for reasoning distillation. We hope to share Reasoning Distillation Provenance Tracing, along with our insights into reasoning distillation, with the community. 1 Introduction The rapid development of reinforcement learning (RL) techniques ( 29 ; 31 ; 1 ) and resulting large-scale reasoning models ( 4 ; 27 ; 34 ) has accelerated the growth of distillation researches, especially reasoning distillation ( 40 ; 8 ) . Early work on reasoning distillation focused on creating high-quality open-source datasets ( 40 ; 8 ; 26 ) , and recent studies have focused on curating and filtering distillation samples to improve efficiency and performance ( 39 ; 21 ) . However, these approaches primarily focus on model performance and fail to provide an explanatory analysis of the sources of the model’s output in test-time contexts. As a result, it remains unclear whether the student model has successfully inherited the knowledge and reasoning logic from the teacher model, raising concerns about the generalization of distillation models ( 12 ) . Concretely, as illustrated in Figure 1 , reasoning distillation involves two main steps. First, specific contexts (e.g., high-quality questions, or high-quality questions augmented with partially generated answers) are provided to the teacher model, which then produces the next action via sampling. Second, these context–action pairs are used to train the student model, encouraging it to reproduce the teacher’s actions under the same contexts. At test time, however, the distilled student faces new contexts. It remains unclear whether the student will continue to follow the teacher’s behavior instead of falling back on its original output distribution. Figure 1: Motivation. In reasoning distillation, the student model learns in the training stage to produce actions consistent with the teacher model within the training context. However, at test time, it remains unclear whether the distilled model will continue to output actions aligned with the teacher model instead of degrading to the outputs of the student model, raising concerns about the generalization of distillation models ( 12 ) . To analyse the above problem, we propose a cross-model Reasoning Distillation Provenance Tracing framework. Concretely, we evaluate three open-source distilled models: DeepSeek-Distill-Qwen-7B ( 4 ) , DeepSeek-R1-0528-Qwen3-8B ( 4 ) , and LIMO-v2 model ( 37 ) . To analyze the patterns in distilled models’ outputs under test contexts, we collect multiple responses from them on GPQA-D ( 28 ) and AIME24. Each response is re-input into the distilled model, the corresponding teacher model, and the original student model. This procedure allows us to obtain, under the same test scenario, the probability assigned by each model to the next action produced by the distilled model. By comparing these probabilities, we can naturally trace the provenance of every action. For example, if the teacher model assigns significantly higher probability to an action than the original student model does, we can attribute that action to teacher-originated actions, since the distilled model’s ability to produce it mainly derives from teacher model’s knowledge transferred during distillation. For further details about other action types, see Section 3 . To this end, we observe that distilled models can reproduce teacher-originated actions in new test contexts. Moreover, these actions are correlated with correct responses on the test set, which helps explain the generalization gains achieved through distillation. This observation further motivates our training design: we hypothesize that when the training data contains a higher proportion of teacher-originated actions, the distilled model attains better final performance. To validate this hypothesis, we propose a teacher-guided data selection strategy that compares the probabilities assigned by the teacher and student models on the training data and prioritizes examples that contain more teacher-originated actions. We then evaluate its effectiveness across multiple settings, including representative teacher models from different families (DeepSeek-R1-671B ( 4 ) , QwQ-32B ( 34 ) , GPT-OSS-120B ( 27 ) ) and various student models (Qwen2.5-7B-Instruct, Qwen4-4B-Base, Qwen3-8B-Base, Qwen3-4B-Instruct-2507). Our contributions can be summarised as follows: (1) We propose Reasoning Distillation Provenance Tracing, a systematic method to disentangle the origins of each action, through fine-grained categorization into teacher-originated, student-originated, shared, and boosted actions. This offers a principled approach for analyzing whether distillation genuinely transfers reasoning ability or merely reinforces pre-existing patterns. (2) Through analysis on reasoning benchmarks, we show that distilled models can generate teacher-originated actions even in unseen test scenarios. These actions are correlated with correctness, offering a quantitative explanation for why reasoning distillation improves generalization. (3) Building on the provenance analysis, we introduce a teacher-guided data selection strategy that prioritizes training samples rich in teacher-originated actions. Unlike heuristic method, our approach leverages explicit teacher–student divergence as a selection criterion. Experiments across diverse teacher–student pairs demonstrating performance gains in our settings. 2 Related Work 2.1 Reasoning Distillation Distilling the reasoning abilities of large reasoning models has been an important problem since their emergence ( 17 ; 13 ; 19 ; 18 ) . DeepSeek ( 4 ) pioneered this line of work by showing that supervised fine-tuning on the outputs of a reasoning teacher, which is also the approach we focus on in this paper, can dramatically enhance the reasoning abilities of smaller models. Numerous subsequent projects (e.g., HuggingFace OpenR1 ( 15 ) , OpenThoughts ( 8 ) , a-m-team ( 40 ) , NVIDIA AceReason ( 24 ) , Alibaba OmniThought ( 8 ) , LIMO ( 37 ) , Tencent DeepMath ( 10