Hawkeye : Efficient Reasoning with Model Collaboration

Jianshu She, Zhuohao Li, Zhemin Huang, Qi Li, Peiran Xu, Haonan Li, Qirong Ho · 2025

Read Hawkeye : Efficient Reasoning with Model Collaboration by Jianshu She, Zhuohao Li, Zhemin Huang, Qi Li, Peiran Xu, Haonan Li, Qirong Ho (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-00424.txt.

Introduction

Long chains of thought help chat models solve hard problems, but most of the words in those chains are filler that costs time and money. Jianshu She and co-authors build a team approach in a 2025 paper: a large model writes a short, dense plan, and a small model expands it into the full reply. The plan keeps the steps that matter and drops the rest.

Training teaches the large model to squeeze its reasoning by scoring how much each part earns its place. With plans at about a third of the full length, reply quality held level while clarity rose. On hard math the team ran several times faster end to end and cut costs by more than half. The split plays to each model's strength: judgment up front, fluency after.

Catalog record

CREATORJianshu She, Zhuohao Li, Zhemin Huang, Qi Li, Peiran Xu, Haonan Li, Qirong Ho
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-00424

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hawkeye : Efficient Reasoning with Model Collaboration Jianshu She, Zhuohao Li, Zhemin Huang, Qi Li, Peiran Xu, Haonan Li, Qirong Ho Abstract Chain-of-Thought (CoT) reasoning has demonstrated remarkable effectiveness in enhancing the reasoning abilities of large language models (LLMs). However, its efficiency remains a challenge due to the generation of excessive intermediate reasoning tokens, which introduce semantic redundancy and overly detailed reasoning steps. Moreover, computational expense and latency are significant concerns, as the cost scales with the number of output tokens, including those intermediate steps. In this work, we observe that most CoT tokens are unnecessary, and retaining only a small portion of them is sufficient for producing high-quality responses. Inspired by this, we propose Hawkeye , a novel post-training and inference framework where a large model produces concise CoT instructions to guide a smaller model in response generation. Hawkeye quantifies redundancy in CoT reasoning and distills high-density information via reinforcement learning. By leveraging these concise CoTs, Hawkeye is able to expand responses while reducing token usage and computational cost significantly. Our evaluation shows that Hawkeye can achieve comparable response quality using only 35% of the full CoTs, while improving clarity, coherence, and conciseness by approximately 10%. Furthermore, Hawkeye can accelerate end-to-end reasoning by up to 3.4× on complex math tasks while reducing inference cost by up to 60%. Hawkeye will be open-sourced and the models will be available soon. † † † Equal contribution. 1 Introduction The emergence of reasoning-capable large language models (LLMs) ( OpenAI et al. 2024 ; DeepMind 2024 ; Guo et al. 2025 ) has recently made headlines. Equipped with Chain-of-Thought (CoT) reasoning ( Wei et al. 2023 ) , these models “think” by producing lengthy internal reasoning traces before generating a final response. This reasoning paradigm decomposes complex questions and applies multiple strategies to verify and refine answers, mimicking human-like problem solving. It has proven particularly effective for tasks that require fine-grained, step-by-step logical synthesis, such as mathematics and code generation ( Zhang et al. 2022 ; Sprague et al. 2024 ) . However, such test-time compute scaling is inefficient, as a large portion of the generated “thinking” tokens is redundant. While long CoTs can improve reasoning quality, generating them during inference introduces significant overhead. For example, the OpenAI o1 model ( OpenAI et al. 2024 ) uses up to 40K tokens, whereas GPT-4o averages around 4K tokens for the same query ( Patel 2024 ) , leading to a 10 \times increase in KV cache memory usage. Moreover, post-training and deployment of reasoning models are extremely costly. Knowledge distillation is often necessary to improve accessibility in reinforcement learning (RL)-based post-training. For example, the API cost of OpenAI o1 is approximately 6 \times higher than that of GPT-4o. As reasoning models continue to scale at inference time, improving their efficiency has become an increasingly urgent challenge. Figure 1 : Overview of Hawkeye and Comparison with Existing Reasoning Models. Hawkeye restructures the reasoning pipeline through model collaboration: a large model generates critical Chain-of-Thought (CoT) instructions, while a small model expands these instructions into complete responses to improve readability. The small model is fine-tuned using reinforcement learning on high-density CoTs. Hawkeye enables fast, efficient, and cost-effective reasoning by reducing token usage without sacrificing response quality. Although prior work has explored improving the test-time compute efficiency of reasoning, many approaches remain limited to self-reflection or multi-turn CoT tuning. For instance, Cheng & Durme 2024 reduce computational overhead by streamlining intermediate reasoning steps, while Wu et al. 2025 systematically examine the trade-off between efficiency and precision in large-scale reasoning. Additionally, Fu et al. 2024 introduce request tracking and scheduling based on model certainty, enabling early stopping to meet service-level objectives (SLOs) for latency-sensitive users. However, these methods do not address the fundamental challenge of overthinking in reasoning models: How can we measure reasoning redundancy and effectively exploit it to enable more efficient thinking and generation? In this work, we propose Hawkeye , a fast, efficient, and cost-effective CoT reasoning pipeline that is adaptable to a wide range of reasoning models. Our key insight spans both the inference and post-training stages. At inference time, we find that only the critical 20% of tokens meaningfully contribute to the final answer, while the remaining 80% introduce redundancy and can be safely removed. We provide a comprehensive assessment of this phenomenon across several popular reasoning models and diverse datasets. During post-training, we leverage reinforcement learning to guide models in identifying and retaining only the most informative tokens, enabling high-density reasoning. Hawkeye introduces model collaboration , wherein a powerful model generates concise reasoning instructions, and small models expand them into human-readable responses. By retaining only critical CoTs and discarding redundant tokens during inference, Hawkeye significantly reduces CoT reasoning complexity and cost, while maintaining—or even improving—response quality across datasets of varying difficulty levels (e.g., GSM8K ( Cobbe et al. 2021 ) , MATH ( Hendrycks et al. 2021 ) , MATH500 ( Lightman et al. 2023 ) , AIME). In our evaluation, Hawkeye achieves up to a 67.6% reduction in reasoning tokens, a 62% reduction in serving cost, and a 3.4 \times speedup in end-to-end reasoning compared to state-of-the-art reasoning models. Our contributions can be summarized as follows: 1. We propose a novel reasoning paradigm, Hawkeye , which utilizes model collaboration for efficient reasoning. In this setup, a large model generates concise reasoning instructions, which a small model then expands into full responses. Hawkeye significantly reduces both computational and financial costs while preserving the response quality. We provide both empirical and theoretical evidence demonstrating the effectiveness of this collaborative reasoning framework. 2. We present the first systematic study of CoT redundancy, showing that excessive reasoning tokens are a widespread phenomenon across various tasks. Our analysis reveals that a substantial portion of these tokens can be removed using a principled compression strategy without degrading output quality. 3. We curate a high-quality CoT dataset and fine-tune a reasoning model using reinforcement learning to optimize CoT generation. This approach reduces CoT length by over 75% compared to the original large model while maintaining performance, with only a 4% drop in accuracy on the evaluated dataset. 2 Related Work Figure 2 : An example of Hawkeye reasoning. The red highlights marks critical tokens that cannot be removed. Hawkeye generates a high density of critical tokens and reduces token usage by over 70% (600 \to 182 tokens). More examples can be found in Appendix C . Chain of Thought Chain-of-Thought (CoT) reasoning has become a dominant approach in recent reasoning models such as OpenAI o1 ( OpenAI et al. 2024 ) and DeepSeek-R1 ( Guo et al. 2025 ) . It generates rationales for questions and encourages models to think before responding. The quality of CoTs is critical, as errors can propagate due to the auto-regressive nature of generation. Several approaches have been proposed to optimize CoT reasoning, including self-consistency ( Wang et al. 2023 ) , least-to-most prompting ( Zhou et al. 2022 ) , Graph of Thoughts ( Yao et al. 2023a ) , and ReAct ( Yao et al. 2023b ) . Inefficiency in Reasoning Models Despite the success of reasoning models, step-by-step thinking is often lengthy and incurs substantial computational overhead. To address this issue,

Read the complete text