Fast Quiet-STaR: Thinking Without Thought Tokens

Wei Huang, Yizhe Xiong, Xin Ye, Zhijie Deng Hui Chen Zijia Lin Guiguang Ding · 2025

Read Fast Quiet-STaR: Thinking Without Thought Tokens by Wei Huang, Yizhe Xiong, Xin Ye, Zhijie Deng Hui Chen Zijia Lin Guiguang Ding (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17746.txt.

Introduction

Wei Huang and six co-authors present Fast Quiet-STaR in two thousand twenty-five. Quiet-STaR boosts reasoning by making thought traces at the token level. That gain costs a lot at run time. The new method keeps the gain while it cuts the cost.

Training slowly trims the count of thought tokens in stages. The model learns to hold its reasoning in a shorter, more abstract form. A second form folds the trick into plain next token training via reinforcement. It needs no thought tokens at run time at all. Tests on two models show higher scores at the same time budget.

Catalog record

CREATORWei Huang, Yizhe Xiong, Xin Ye, Zhijie Deng Hui Chen Zijia Lin Guiguang Ding
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17746

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Fast Quiet-STaR: Thinking Without Thought Tokens Wei Huang, Yizhe Xiong, Xin Ye, Zhijie Deng Hui Chen Zijia Lin Guiguang Ding Abstract Large Language Models (LLMs) have achieved impressive performance across a range of natural language processing tasks. However, recent advances demonstrate that further gains—particularly in complex reasoning tasks—require more than merely scaling up model sizes or training data. One promising direction is to enable models to “think” during the reasoning process. Recently, Quiet-STaR significantly improves reasoning by generating token-level thought traces, but incurs substantial inference overhead. In this work, we propose Fast Quiet-STaR, a more efficient reasoning framework that preserves the benefits of token-level reasoning while reducing computational cost. Our method introduces a curriculum-learning-based training strategy that gradually reduces the number of thought tokens, enabling the model to internalize more abstract and concise reasoning processes. We further extend this approach to the standard Next Token Prediction (NTP) setting through reinforcement learning-based fine-tuning, resulting in Fast Quiet-STaR NTP, which eliminates the need for explicit thought token generation during inference. Experiments on four benchmark datasets with Mistral 7B and Qwen2.5 7B demonstrate that Fast Quiet-STaR consistently outperforms Quiet-STaR in terms of average accuracy under the same inference time budget. Notably, Fast Quiet-STaR NTP achieves an average accuracy improvement of 9% on Mistral 7B and 5.7% on Qwen2.5 7B, while maintaining the same inference latency. Our code will be available at https://github.com/huangwei200012/Fast-Quiet-STaR. 1 Introduction Large Language Models (LLMs) Achiam et al. 2023 ; Grattafiori et al. 2024a have achieved remarkable progress in recent years by pretraining models with billions of parameters on massive datasets. However, merely scaling up model size or increasing the amount of training data is insufficient for enabling strong performance on tasks that require complex reasoning or long-term planning. To further enhance model capabilities, one promising direction is to enable models to engage in autonomous “thinking” before producing final answers. Recently, a growing body of research has explored this paradigm to strengthen the reasoning abilities of LLMs. Notably, models such as OpenAI o1 OpenAI et al. 2024 , DeepSeek-R1 Guo et al. 2025 , QwQ Zheng et al. 2024 , and Kimi-1.5 Team et al. 2025 have demonstrated impressive performance across a variety of challenging tasks, such as mathematical competition problems Hendrycks et al. 2021 ; Cobbe et al. 2021 . Figure 1: Performance comparison between Fast Quiet-STaR, Quiet-STaR and the pre-trained model (NTP). “Inference Time” represents the Time-to-First-Token (TTFT) of each model variant. Note that with the same number of thought tokens, Fast Quiet-STaR shares the same inference time with Quiet-STaR, but enjoys a significant performance boost. Additionally, Fast Quiet-STaR can be extended to the NTP setting, improving model performance without additional inference time overhead. Recently, Quiet-STaR (Quiet Self-Taught Reasoner) Zelikman et al. 2024 has been proposed as a novel reasoning paradigm that shifts the thinking process from the problem level to a finer, token-level granularity. In Quiet-STaR, before predicting the next token, the model first generates a sequence of intermediate thought trace (represented as <|start_of_thought|>,thought_token1, thought_token2, … <|end_of_thought|> ), based on which the model predicts the next token. Compared to other approaches, Quiet-STaR can significantly enhance the model’s reasoning ability through a lightweight unsupervised training process. For instance, it achieves a 10% performance gain on CommonsenseQA Talmor et al. 2018 for the Mistral 7B Jiang et al. 2023 by continue pre-training with only 0.2M tokens, demonstrating remarkable improvements through efficient training. Although Quiet-STaR significantly enhances the model’s reasoning capabilities, it substantially increases inference overhead due to the requirement of generating a thought trace for every token. As shown in Figure 1 , even when using only 8 thought tokens, the average Time-To-First-Token (TTFT) of Quiet-STaR remains over 10 times higher than that of conventional next token prediction (NTP) models. Despite the high inference costs, these thought tokens cannot be directly reduced or eliminated as they are the main contributor to performance improvements. For example, as shown in Figure 1 , halving the number of thought tokens from 16 to 8 leads to a 4.7% accuracy drop. Such dilemma on efficiency severely undermines the practical value of Quiet-STaR. Prior work has shown that LLMs are capable of skipping reasoning steps by omitting non-essential steps without sacrificing overall reasoning performance Liu et al. 2024 . Inspired by this, we believe that within the Quiet-STaR reasoning paradigm, the model can maintain its strong reasoning abilities obtained from long thought-trace training by compressing the number of thought tokens and keeping only a more abstract thought trace . To improve the efficiency of the Quiet-STaR reasoning paradigm with minimal performance degradation, we propose Fast Quiet-STaR . We employ a multi-stage training strategy that progresses from easy to hard. That is, we gradually guide the model from generating a detailed thought trace using more thought tokens to generating a concise thought trace using fewer thought tokens. To further accelerate the Quiet-STaR inference paradigm to NLP-level efficiency, we employ a reinforcement learning-based fine-tuning strategy for Fast Quiet-STaR model under the NTP setting. The resulting Fast Quiet-STaR NTP model preserves the original thinking abilities of Fast Quiet-STaR while eliminating reliance on generating explicit thought trace during inference. We evaluate our method on two open-source models, Mistral 7B Jiang et al. 2023 and Qwen2.5 7B Qwen et al. 2025 across four public datasets. Extensive experiments show that, given the same number of thought tokens(same inference time), Fast Quiet-STaR achieves substantial performance gains over Quiet-STaR. Furthermore, under equivalent inference time, Fast Quiet-STaR NTP improves the average accuracy by 9% on Mistral 7B and 5.7% on Qwen2.5 7B compared to the original pre-trained models. We summarize our contribution as follows: • We propose Fast Quiet-STaR, a novel training paradigm that compresses token-level thought traces to significantly reducing inference overhead while preserving the strong reasoning abilities imparted by the Quiet-STaR framework. • We introduce a curriculum learning-based multi-stage training strategy that progressively guides the model to learn a more concise thought trace, enabling it to internalize efficient reasoning patterns and express them compactly without performance degradation. We further accelerate Fast Quiet-STaR to the standard NTP-level setting via reinforcement learning-based fine-tuning, enabling implicit reasoning without explicit thought token generation. • Extensive experiments show that Fast Quiet-STaR achieves comparable or even better performance than standard Quiet-STaR while reducing thought tokens. Fast Quiet-STaR NTP significantly outperforms the pre-trained model without increasing the inference time. 2 Related Works 2.1 LLM Reasoning In recent years, enhancing the reasoning capabilities of large language models has become a major research focus Rajani et al. 2019 ; Zhang et al. 2025 ; Pan et al. 2025 . The Chain-of-Thought (CoT) Kojima et al. 2022 prompting technique explicitly guides models to generate intermediate reasoning steps. Tree of Thoughts (ToT) Yao et al. 2023 explores multiple reasoning paths through a tree-structured search. The CPO Zhang et al. 2024 method combines ToT with Direct Preference Optimization (DPO) Rafailov et al. 2023 , using reasoning paths generated by ToT as paired training data to directly optimize the model’s CoT abilities. Self-Consistency Wang et al. 2022 samples multiple reasoning paths for the same problem and selects the final answer through a voting mechanism. Methods based on Monte Carlo Tree Search (MCTS) Qi et al. 2024 introduce classical planning algorithms into the reasoning process.

Read the complete text