InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models

Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, Yueting Zhuang · 2025

Read InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models by Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, Yueting Zhuang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-06692.txt.

Introduction

Yan and six co-authors put forth a way in twenty twenty-five to let large chat models think in long chains without a hard length cap. Long chains cost more as they grow and break past a set window. Their way cuts thought into short parts with brief sums of ground won between them.

Memory use then saws up and down in a small band in place of a steep climb. They re-built a known math set in that form, with more than three hundred thousand train cases. On math and science tests a tuned model gained by clear gaps. Cost fell at the same time, they report.

Catalog record

CREATORYuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, Yueting Zhuang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE101 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-06692

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, Yueting Zhuang Abstract Advanced reasoning in large language models has achieved remarkable performance on challenging tasks, but the prevailing long-context reasoning paradigm faces critical limitations: quadratic computational scaling with sequence length, reasoning constrained by maximum context boundaries, and performance degradation beyond pre-training context windows. Existing approaches primarily compress reasoning chains without addressing the fundamental scaling problem. To overcome these challenges, we introduce InftyThink, a paradigm that transforms monolithic reasoning into an iterative process with intermediate summarization. By interleaving short reasoning segments with concise progress summaries, our approach enables unbounded reasoning depth while maintaining bounded computational costs. This creates a characteristic sawtooth memory pattern that significantly reduces computational complexity compared to traditional approaches. Furthermore, we develop a methodology for reconstructing long-context reasoning datasets into our iterative format, transforming OpenR1-Math into 333K training instances. Experiments across multiple model architectures demonstrate that our approach reduces computational costs while improving performance, with Qwen2.5-Math-7B showing 3-11% improvements across MATH500, AIME24, and GPQA_diamond benchmarks. Our work challenges the assumed trade-off between reasoning depth and computational efficiency, providing a more scalable approach to complex reasoning without architectural modifications. Project Page: https://zju-real.github.io/InftyThink Figure 1: (a) Computational complexity comparison between vanilla long-context reasoning (blue, left) and InftyThink (orange, right). The sawtooth pattern of InftyThink demonstrates how periodic summarization creates a bounded memory footprint, substantially reducing computational costs (smaller area under curve) while enabling deeper reasoning. (b) InftyThink ( \star ) reduces reasoning latency while simultaneously improving reasoning performance compared to the vanilla long-context reasoning ( \bullet ). 1 Introduction Recent studies have demonstrated the remarkable reasoning capabilities of large language models (LLMs), with models like OpenAI o1 ( OpenAI, 2024 ) , DeepSeek-R1 ( Guo et al., 2025 ) , Gemini 2.0 Flash Thinking ( Google DeepMind, 2025 ) , QwQ ( Qwen Team, 2024 ) , and Kimi-1.5 ( Kimi Team et al., 2025 ) surpassing human performance on high-difficulty tasks including mathematical competitions ( Latif et al., 2024b ; Latif et al., 2024a ) . These advanced reasoning models are typically developed through methodical techniques such as test-time scaling ( Chen et al., 2024 ; Wan et al., 2024 ; Guan et al., 2025 ; Muennighoff et al., 2025 ) , post-training on long-thought trajectories ( Guo et al., 2025 ; Ye et al., 2025 ; Open Thoughts Team, 2025 ) , or large-scale reinforcement learning ( Guo et al., 2025 ; Shao et al., 2024 ) to generate effective reasoning paths that reach correct answers. A defining characteristic of these models is their ability to perform long-context reasoning, demonstrating advanced cognitive techniques including intent comprehension, multi-perspective analysis, self-reflection, and error correction ( Zeng et al., 2024 ; Zhong et al., 2024 ) . This evolution from simpler reasoning patterns to extensive deliberation has significantly improved problem-solving capabilities, particularly for complex challenges requiring multi-step inference. However, this substantial improvement in reasoning quality comes with significant computational costs ( OpenAI, 2024 ; Brown et al., 2024 ; Snell et al., 2024 ) . The computational complexity of decoder-based LLMs grows quadratically with sequence length, resulting in prohibitive resource requirements for long-form reasoning. This efficiency bottleneck manifests in three primary challenges: First , current reasoning models often generate thousands of tokens even for moderately complex problems ( OpenAI, 2024 ; Wang et al., 2025 ) , creating substantial memory and processing overhead during inference. Second , reasoning processes are constrained by the model’s maximum context length (aka. max_length ) ( Kuratov et al., 2024 ) , frequently resulting in truncated reasoning that fails to reach conclusive answers (See analysis in Appendix E ). Third , most LLM architectures are pre-trained with relatively small context windows (4k-8k tokens), causing performance degradation when reasoning extends beyond these boundaries ( Li et al., 2024b ; Yuan et al., 2025 ) . Existing approaches ( Pang et al., 2024 ; Han et al., 2025 ; Zhang et al., 2025b ) to address these limitations have explored various solutions with mixed success. Some methods attempt to compress reasoning chains post-generation ( Xiao et al., 2023 ; Zhang et al., 2023 ; Chen et al., 2025 ) , while others aim to train models to reason more concisely from the outset ( Kang et al., 2024 ; Arora & Zanette, 2025 ; Liu et al., 2024 ; Xia et al., 2025 ) . Chain-compression techniques like those employed in CoT-Valve ( Ma et al., 2025 ) show promise but require predefined compression ratios during training, limiting their flexibility at inference time. TokenSkip ( Xia et al., 2025 ) reduces redundant tokens by assessing each token’s significance, though this impacts the model’s reasoning performance. LightThinker ( Zhang et al., 2025b ) employs special tokens to dynamically compress the CoT process into a latent representation but lacks the ability to adaptively determine compression requirements for each step. Despite these advances, most approaches still operate within the traditional paradigm of generating a single, continuous reasoning chain, which merely attempting to make it more compact without addressing the fundamental computational scaling problem (A more detailed review of related work in Appendix C ). This raises a critical question: Instead of optimizing within the constraints of monolithic reasoning, could we improve the model’s accuracy and efficiency by altering its inherent reasoning paradigm? In this paper, we propose a fundamentally different approach to long-context reasoning. Rather than viewing reasoning as a single extended process, we introduce InftyThink, a novel paradigm that divides complex reasoning into multiple interrelated short reasoning segments. Each segment remains within a computationally efficient context length while maintaining the coherent flow of thought across iterations. This approach draws inspiration from human cognitive processes, where complex problem-solving frequently involves breaking problems into manageable parts and summarizing intermediate progress. The core mechanism of InftyThink is an iterative process where the model generates a partial reasoning chain, summarizes its current thinking, and builds upon these summaries in subsequent iterations. As illustrated in Figure 1 , traditional approaches (left, blue) face inevitable termination when context length reaches the model’s maximum limit, often before completing the reasoning. In contrast, InftyThink (right, orange) creates a sawtooth pattern through periodic summarization, enabling unbounded reasoning depth while maintaining a bounded memory footprint. This approach both reduces computational complexity (smaller area under the curve) and overcomes the fundamental ceiling on reasoning depth imposed by context length constraints. Beyond computational efficiency, InftyThink offers a crucial advantage: it enables reasoning of arbitrary depth without architectural changes to the underlying model. By summarizing and building upon previous reasoning in manageable segments, models can effectively navigate complex problem spaces that would otherwise exceed context limitations. To validate our approach, we reconstructed the existing SFT dataset OpenR1-Math, which was distilled from DeepSeek-R1, adapting it to conform to our proposed InftyThink paradigm. This reconstruction process transformed the original long-form reasoning examples into multiple interconnected reasoning segments with corresponding summaries. We then fine-tuned multiple base architectures on this reconstructed dataset and conducted comprehensive comparisons against traditional single-round long-context reasoning methods. Our experimental results demonstrate consistent improvements across various benchmarks, with Qwen2.5-Math-7B showing 2% improvement on MATH500, 11% improvement on AIME24,

Read the complete text