Diffuse Thinking: Exploring Diffusion Language Models as Efficient Thought Proposers for Reasoning

Chenyang Shao, Sijian Ren, Yong Li · 2025

Read Diffuse Thinking: Exploring Diffusion Language Models as Efficient Thought Proposers for Reasoning by Chenyang Shao, Sijian Ren, Yong Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27469.txt.

Introduction

Chenyang Shao, Sijian Ren, and two other co-authors pair two kinds of language models in this 2025 paper. Standard models write one word at a time, so proposing many candidate thoughts costs a lot. Diffusion language models work the other way: they clean a noisy draft in parallel, yielding varied samples in one pass. The team uses them as thought proposers.

A diffusion model drafts candidate steps; a standard model grades them. The split keeps quality while cutting the cost of brainstorming. Word-by-word drafting scales poorly as more compute is spent at test time. Parallel drafting dodges that trap. Across several hard reasoning benchmarks, the paired setup scored strongly. The authors offer it as a path to cheaper, deeper machine reasoning.

Catalog record

CREATORChenyang Shao, Sijian Ren, Yong Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-27469

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Diffuse Thinking: Exploring Diffusion Language Models as Efficient Thought Proposers for Reasoning Chenyang Shao, Sijian Ren, Yong Li Abstract In recent years, large language models (LLMs) have witnessed remarkable advancements, with the test-time scaling law consistently enhancing the reasoning capabilities. Through systematic evaluation and exploration of a diverse spectrum of intermediate “thoughts”, LLMs demonstrate the potential to generate deliberate reasoning steps, thereby substantially enhancing reasoning accuracy. However, LLMs’ autoregressive generation paradigm results in reasoning performance scaling sub-optimally with test-time computation, often requiring excessive computational overhead to propose thoughts while yielding only marginal performance gains. In contrast, diffusion language models (DLMs) can efficiently produce diverse samples through parallel denoising in a single forward pass, inspiring us to leverage them for proposing intermediate thoughts, thereby alleviating the computational burden associated with autoregressive generation while maintaining quality. In this work, we propose an efficient collaborative reasoning framework, leveraging DLMs to generate candidate thoughts and LLMs to evaluate their quality. Experiments across diverse benchmarks demonstrate that our framework achieves strong performance in complex reasoning tasks, offering a promising direction for future research. Our code is open-source at https://anonymous.4open.science/r/Diffuse-Thinking-EC60 . Introduction Since the explosive popularity of ChatGPT, transformer-based large language models (LLMs) have received unprecedented attention, becoming a research hotspot and sparking a wave of applications. Influential models such as GPT-4o ( OpenAI 2024a ) , Llama 3 ( Meta 2024 ) , and Deepseek-V3 ( Liu et al. 2024 ) , which adopt a decoder-only transformer, as well as encoder-decoder models like ChatGLM ( GLM et al. 2024 ) , all adhere to the autoregressive generation paradigm. This paradigm necessitates a forward pass computation sequentially for each token decoded, leading to significant inference cost concerns that remain a primary challenge. Moreover, many existing methods to enhance the reasoning accuracy of LLMs, such as Chain-of-Thought (CoT) ( Wei et al. 2022 ) and Tree-of-Thought (ToT) ( Yao et al. 2023 ) , can be seen as attempting to scale greater token reasoning spaces to achieve higher reasoning accuracy. Recent work by OpenAI ( OpenAI 2024c ) has brought test-time scaling into the spotlight, garnering significant attention within the academic and professional communities. This paradigm shifts the allocation of computational resources and time from the pretraining phase to the inference phase. Specifically, the reasoning models think more before the response, exploring a richer space of intermediate thoughts. By iteratively proposing diverse reasoning thoughts and coupling this process with systematic evaluation and optimization strategies, the models are able to improve their reasoning performance ( Xu et al. 2025 ; Kong et al. 2025 ) . However, this shift comes at a significant cost, as exemplified by OpenAI’s o1 model ( OpenAI 2024b ) , which requires a hundredfold increase in token consumption to solve a single problem, thereby exacerbating the already substantial computational and monetary costs associated with large-scale generative models. Since generating tokens is more computationally demanding than processing inputs, the process of proposing thoughts constitutes the primary computational load, motivating us to explore whether there exists a method capable of efficiently generating reasoning proposals. Figure 1: Comparison of generation in DLM and LLM: DLMs generate in parallel, producing multiple tokens simultaneously, while LLMs generate sequentially, one token at a time. The advancement of diffusion language models (DLMs) offers a promising direction, characterized by their efficient thought-proposing capabilities. As shown in Figure 1 , unlike LLMs which generate tokens sequentially and are constrained by their step-by-step, left-to-right nature, DLMs can produce multiple tokens simultaneously through reverse denoising. DLMs parallelize the denoising process effectively, enabling them to simultaneously generate multiple proposals. Moreover, while LLMs employ a uniform generation process for all tasks regardless of difficulty or quality requirements, DLMs can dynamically adjust their denoising steps, enabling a flexible trade-off between generation quality and speed tailored to the specific demands of the task. These advantages position the DLM as a promising approach for proposing diverse proposals efficiently. On the other hand, while LLMs incur high output costs, they demonstrate relatively efficient input processing, as any length of text input can be processed and stored through a single forward pass. Combined with their strong semantic understanding capabilities, LLMs are well-suited to serve as evaluators, assessing and filtering intermediate reasoning proposals. This collaborative approach between the two models mitigates information loss in DLMs while enhancing the overall reasoning process. In contradistinction to speculative decoding, which performs token-level mimicry followed by per-token verification, the DLM-LLM collaboration is instantiated at the level of entire reasoning thoughts, engendering a synergistic co-reasoning paradigm. However, directly employing existing DLMs as proposers for complex reasoning tasks presents several challenges. First, the collaboration mechanism between DLM and LLM is still unclear and the respective strengths of each model are not well understood. Second, while DLMs have certain advantages as proposers due to their ability to generate tokens in parallel, they are also prone to information loss during the generation process, as shown by information theory analysis. LLMs acting as an evaluator can mitigate the information loss that occurs when DLMs generate proposals. Informed by the aforementioned theoretical derivations, we propose a novel collaborative reasoning framework, “Diffuse Thinking” (DT). Specifically, we demonstrate that the parallel generation capabilities of DLMs, when employed as proposers, combined with the evaluation and filtering capabilities of LLMs acting as evaluators, result in a more efficient and accurate reasoning process. To optimize this collaborative dynamic, we meticulously design the interaction between DLMs and LLMs. By focusing on single-step reasoning, DLMs can produce more relevant and accurate proposals. These proposals are subsequently evaluated by LLMs in their capacity as evaluators, allowing them to select the most promising solutions. We further create single-step data specifically for targeted fine-tuning of DLMs in their role as proposers, thereby enhancing their ability to generate high-quality reasoning thoughts. We conduct extensive experiments on four logical and mathematical benchmarks. Across various benchmarks, the framework consistently outperforms baseline models in terms of accuracy while also achieving higher throughput. Specifically, the average accuracy and throughput of reasoning increase by 5% and 10%, respectively, compared to the most accurate baselines. Our contributions can be summarized in three points: • We design a novel framework that combines DLM and LLM, where DLM is used to efficiently propose many proposals, which are subsequently evaluated by LLM. • We theoretically reveal the sources of the advantages of our collaborative framework through information-theoretic and computational complexity analyses, demonstrating that it holds significant potential and is effective in practice. • We validate the proposed framework across diverse logical and mathematical benchmarks, demonstrating that its synergistic design can achieve both higher reasoning accuracy and increased throughput simultaneously. Related works Test-time scaling law in LLM reasoning The reasoning ability of large models is crucial for their application ( Shao et al. 2024 ) . Pre-training once ruled LLM reasoning; recent work ( OpenAI 2024c ; Zhong et al. 2024 ) shows inference-time compute, yields large gains. This has led to the emergence of the Test-Time Scaling Law . Through exploration and experimentation, researchers ( Wang et al. 2024 ; Zhang et al. 2024 ; Qin et al. 2024 ) have converged on a consistent implementation framework: Decompose problems, sample many thoughts, score a trained verifier (such as PRMs), and guide search

Read the complete text