Agentic-R1: Distilled Dual-Strategy Reasoning
Weihua Du, Pranjal Aggarwal, Sean Welleck, Yiming Yang · 2025
Read Agentic-R1: Distilled Dual-Strategy Reasoning by Weihua Du, Pranjal Aggarwal, Sean Welleck, Yiming Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05707.txt.
Introduction
Weihua Du, Pranjal Aggarwal, Sean Welleck, and Yiming Yang train a model that picks its reasoning style per problem in a paper from 2025. Long text reasoning chains do well on math but are slow and error-prone. Tool-using agents compute well yet stumble on abstract logic.
Their framework, DualDistill, merges both strategies from several teacher models into one student. The student, Agentic-R1, calls code tools for arithmetic and algorithmic tasks. For abstract tasks, it reasons in plain text. Accuracy rose across both heavy-compute and standard benchmarks. One flexible model beat single-strategy rivals. The approach hints at assistants that switch tactics like a good human solver.
Catalog record
| CREATOR | Weihua Du, Pranjal Aggarwal, Sean Welleck, Yiming Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-05707
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Agentic-R1: Distilled Dual-Strategy Reasoning Weihua Du, Pranjal Aggarwal, Sean Welleck, Yiming Yang Abstract Current long chain-of-thought (long-CoT) models excel at mathematical reasoning but rely on slow and error-prone natural language traces. Tool-augmented agents address arithmetic via code execution, but often falter on complex logical tasks. We introduce a fine-tuning framework, DualDistill , that distills complementary reasoning strategies from multiple teachers into a unified student model. Using this approach, we train Agentic-R1 , which dynamically selects the optimal strategy for each query, invoking tools for arithmetic and algorithmic problems, and using text-based reasoning for abstract ones. Our method improves accuracy across a range of tasks, including both computation-intensive and standard benchmarks, demonstrating the effectiveness of multi-strategy distillation in achieving robust and efficient reasoning. Our project is available at https://github.com/StigLidu/DualDistill . 1 Introduction A recently proposed reasoning paradigm for language models, long chain-of-thought (long-CoT) reasoning, has achieved state-of-the-art performance on challenging tasks such as mathematical problem solving Guo et al. 2025 ; Jaech et al. 2024 . By allocating a large inference budget, these models generate reasoning trajectories with iterative self-verification and refinement. Despite this progress, open-source long-CoT models remain limited: Their reasoning traces rely solely on natural language, which is both computationally expensive and error-prone without explicit verification. Figure 1: Overview of DualDistill . We distill knowledge from two complementary teacher models. Trajectories from teachers are composed based on correctness, enabling the student model to learn when and how to elect the appropriate strategy for each problem. Furthermore, the student internalizes these strategies through self-distillation. In contrast, tool-aided reasoning provides greater efficiency and reliability, particularly for large-scale numerical computations and tasks that require rigorous verification Gao et al. 2023 . Advanced agent frameworks, such as OpenHands Wang et al. 2024 , place language models in a multi-turn environment with a code interpreter and other tools. The resulting agentic trajectories are effective for tool-intensive tasks, but often fall short on abstract or conceptually complex reasoning problems Duan et al. 2024 . To leverage the strengths of both reasoning and tool-based strategies, we introduce DualDistill , a novel distillation framework (Fig. 1 ) that combines trajectories from two complementary teachers: one reasoning-oriented, the other tool-augmented, in a unified student. The resulting model, Agentic-R1 , learns to mix both strategies and dynamically selects the most appropriate one for each problem, executing code for arithmetic and algorithmic tasks and reasoning in natural language for abstract ones. Furthermore, the student can continue to improve via self-distillation, better calibrating its strategy selection to its actual capabilities. Our contributions are as follows. • DualDistill , a distillation framework that enables a language model to learn from multiple teacher models with complementary capabilities through solution trajectory composition. • Agentic-R1 , a distilled student model that achieves strong performance in mathematical tasks requiring both tool use and complex reasoning. 2 Related Work While prior efforts have integrated external tools into language models Gao et al. 2023 ; Schick et al. 2023 ; Nakano et al. 2022 , they are often specialized to either non-math domains or are confined to shorter reasoning chains. Concurrently, the paradigm of long chain-of-thought (long-CoT) reasoning or inference-time compute has demonstrated significant improvements Guo et al. 2025 ; Jaech et al. 2024 . However, these approaches can be difficult to control and may suffer from ‘overthinking’, particularly when applied to tool-use scenarios Cuadron et al. 2025 . Recent works have combined tool use with long CoT reasoning Feng et al. 2025 ; Song et al. 2025 , but these are often applied to different domains or rely on reinforcement learning, which can be less stable than our proposed distillation method. To the best of our knowledge, DualDistill is the first framework to employ distillation with trajectory composition from two heterogeneous teacher models, one specializing in agentic tool-use and the other in pure textual reasoning, creating a unified student model capable of adaptively leveraging both strategies. See Appendix B for a more detailed discussion. 3 Method As illustrated in Fig. 1 , DualDistill uses trajectory composition to distill the knowledge of the complementary teachers to the student model. The student model then applies self-distillation for a deeper understanding of the strategies. 3.1 Trajectory Composition Let \mathcal{D}=\{(x^{(i)},a^{(i)})\}_{i=1}^{N} be a training set, where x^{(i)} denotes the i -th problem and a^{(i)} is its corresponding reference answer. Let \pi_{A} and \pi_{R} be two distinct teacher policies, where \pi_{A} represents the agentic teacher and \pi_{R} the reasoning teacher. For each training instance (x,a) , we randomly select the initial teacher by sampling a binary indicator z\sim\text{Bernoulli}(0.5) and then produce solutions y_{1} and y_{2} as follows: \displaystyle y_{1} \displaystyle\sim z\pi_{A}(\cdot\mid x)+(1-z)\pi_{R}(\cdot\mid x), \displaystyle y_{2} \displaystyle\sim(1-z)\pi_{A}(\cdot\mid x,y_{1})+z\pi_{R}(\cdot\mid x,y_{1}). That is, after one teacher generates the initial solution y_{1} , the other teacher subsequently generates the second solution y_{2} , conditioned on both the original problem x and the preceding solution y_{1} . We evaluate the correctness of each solution using a rule-based grader, assigning binary correctness scores g_{1},g_{2}\in\{0,1\} to y_{1} and y_{2} , respectively. The distilled training trajectories are then composed based on these correctness scores. • g_{1}=0,g_{2}=1 : The first teacher produces an incorrect solution, and the second teacher successfully corrects it. The composed trajectory is structured as y_{1}\oplus t_{-+}\oplus y_{2} . (Here \oplus denotes concatenation and t_{-+} is a transition segment, described later). • g_{1}=1,g_{2}=1 : Both teachers provide correct solutions. We create a trajectory y_{1}\oplus t_{++}\oplus y_{2} to reflect complementary correct strategies. • g_{1}=1,g_{2}=0 : Only the initial teacher provides a correct solution. In this scenario, the composed trajectory includes only y_{1} . • g_{1}=0,g_{2}=0 : Both teachers do not solve the problem correctly. In this case, we just discard the problem without composing any trajectory. The transition segments t_{-+} and t_{++} are predefined sentences indicating strategy shifts (e.g., "Wait, using text reasoning is too tedious, let us try code reasoning." ). More examples and detailed descriptions can be found in Appendix A.4.1 . 3.2 Training Instance Selection We curate a training set with the instances for which one strategy has a clear advantage over the other in performance. Using an existing dataset such as GSM8K Cobbe et al. 2021 would be insufficient in this sense as most of the problems are relatively simple and can be solved by either strategy without a significant performance difference. Instead, we construct two contrasting subsets of Math problems from DeepMath He et al. 2025 : one can benefit more from tool-assisted reasoning, while the other can benefit more from pure text-based reasoning. After composition, we apply additional filtering to balance the training dataset, resulting in 2.6 k distilled trajectories. Detailed statistics can be found in Appendix A.3.2 . Further filtering details are provided in the Appendix A.3.1 . 3.3 Teacher and Student Models As the teacher of agentic reasoning, we utilize OpenHands Wang et al. 2024 , a tool-assisted agent built upon Claude-3.5-Sonnet Anthropic 2024 to employ human-designed problem-solving pipelines. As the teacher of text-based reasoning, we adopt Deepseek-R1 Guo et al. 2025 . The details can be found in Appendix A.4.2 . As for the student model, we adopt Deepseek-R1-Distill-7B , which has been fine-tuned on pure text-based reasoning trajectories and also exposed to code-related data during pretraining. We deliberately choose a