StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs
Qijun Luo, Mengqi Li, Lei Zhao, Xiao Li · 2025
Read StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs by Qijun Luo, Mengqi Li, Lei Zhao, Xiao Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-03077.txt.
Introduction
Qijun Luo and co-authors cut the memory cost of training language models in a 2025 paper. Long training sequences boost skill at hard, chained reasoning, but they flood memory. Backpropagation must store activation values for every step, and even checkpoint tricks leave a heavy bill.
Their method, StreamBP, splits the chain rule along the sequence in a layer-by-layer stream. Memory for activations and output scores falls sharply, while results stay exact, not approximate. The trick fits common training goals such as fine-tuning and preference methods. It also uses fewer operations and runs the backward pass faster. Long-context training becomes possible on smaller hardware fleets.
Catalog record
| CREATOR | Qijun Luo, Mengqi Li, Lei Zhao, Xiao Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-03077
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs Qijun Luo, Mengqi Li, Lei Zhao, Xiao Li Abstract Training language models on long sequence data is a demanding requirement for enhancing the model’s capability on complex tasks, e.g., long-chain reasoning. However, as the sequence length scales up, the memory cost for storing activation values becomes huge during the Backpropagation (BP) process, even with the application of gradient checkpointing technique. To tackle this challenge, we propose a memory-efficient and exact BP method called StreamBP , which performs a linear decomposition of the chain rule along the sequence dimension in a layer-wise manner, significantly reducing the memory cost of activation values and logits. The proposed method is applicable to common objectives such as SFT, GRPO, and DPO. From an implementation perspective, StreamBP achieves less computational FLOPs and faster BP speed by leveraging the causal structure of the language model. Compared to gradient checkpointing, StreamBP scales up the maximum sequence length of BP by 2.8 − 5.5 × 2.8-5.5\times larger, while using comparable or even less BP time. Note that StreamBP’s sequence length scaling ability can be directly transferred to batch size scaling for accelerating training. We further develop a communication-efficient distributed StreamBP to effectively support multi-GPU training and broaden its applicability. Our code can be easily integrated into the training pipeline of any transformer models and is available at https://github.com/Ledzy/StreamBP . 1 Introduction Large language models (LLMs) achiam2023gpt ; llama3 ; yang2024qwen2 have been regarded as a powerful approach toward general artificial intelligence bubeck2023sparks . Recently, there has been growing interest in training LLMs for reasoning tasks, leading to significant improvements on math and code benchmarks as shown by OpenAI-o1 and DeepSeek-R1 jaech2024openai ; guo2025deepseek . However, such models often require extremely long input sequences due to the inclusion of detailed reasoning traces guo2025deepseek ; yu2025dapo ; muennighoff2025s1 ; ye2025limo ; yeo2025demystifying , which results in substantial GPU memory consumption during backpropagation (BP) when training reasoning models using either reinforcement learning (RL) or supervised finetuing (SFT). This considerable memory usage mainly stems from storing intermediate activations during the BP process. This work aims to provide an efficient solution to such a memory issue. Background and related works. Due to rapid growth of research in this field, we provide a non-exhaustive background overview on reasoning models and memory-efficient training methods. Training LLM on long reasoning traces. To incentivize long chain of thought (CoT) wei2022chain reasoning capability, RL is one of most popular approaches jaech2024openai ; guo2025deepseek . We also refer to, e.g., openr1 ; tinyzero ; deepscaler2025 ; yu2025dapo ; liu2025understanding ; zeng2025simplerl ; hu2025open ; wang2025reinforcement , for replicating the "aha moment" of reasoning described in DeepSeek-R1. The core step of RL for incentivizing reasoning ability is to use a rule-based reward, e.g., the correct answer of the math question, to provide training signal towards fitting the correctly generated answer by the model itself with reasoning traces. It has been observed that as the RL training progresses, the sequence length will be increased guo2025deepseek . Apart from RL techniques, it has been shown that the simple SFT training pipeline with reasoning data distilled from strong reasoning models such as R1 can also incentivize reasoning ability of LLMs; see, e.g., muennighoff2025s1 ; ye2025limo ; li2025llms ; wen2025light . The length of SFT reasoning traces varies widely, ranging from 8k to 32k, and in some cases exceeding 100k. Training models on these long traces requires significant memory cost in storing activation values. The essential roles of RL and SFT for incentivizing reasoning ability are still under active discussion yue2025does . Memory-efficient training methods. Existing memory-efficient solution for training LLMs mainly focus on designing parameter-efficient or low-dimensional version of Adam kingma2014adam ; loshchilovdecoupled , significantly reducing the memory usage of storing the gradient and optimizer states. Typical algorithms include LoRA hu2022lora and its variants, e.g., qlora ; liu2024dora , GaLore galore , BAdam luo2024badam , etc. However, in the context of training reasoning models with long reasoning sequences, the dominant source of memory consumption arises from the BP process for storing intermediate activations. Gradient checkpointing technique chen2016training reduces the memory cost by recomputing the activation during the backward process. However, it still stores the full activation of the reforwarded layer and the full logits of the output layer; see Figure 1 for an illustration. MsT luo2024mini reduces the memory cost of SFT logits by iteratively processing mini-sequence. However, it still requires the full storage of layer activations during reforward and does not apply to RL objectives. Sequence parallel approaches increase the maximum sequence length by partitioning sequence across GPUs li2021sequence ; jacobs2023deepspeed ; megatron-seqparallel , which requires access of multiple GPUs. Main contributions. Motivated by the above observations, this work aims to address the memory issue occurred during the BP process. Our main contributions are summarized below. (C.1) We propose a memory-efficient and exact backpropagation algorithm called StreamBP , which significantly reduces memory usage of storing the intermediate activation values when training LLMs on ultra-long sequences, e.g., training reasoning models. StreamBP is based on a linear decomposition of the chain rule and involves nontrivial and intricate developments for the language modeling head and transformer layers. As a result, StreamBP is compatible with common training objectives, including SFT, GRPO, and DPO. Moreover, by leveraging the causal structure of LLM, StreamBP is able to save computational FLOPs compared to the standard BP, achieving faster BP speed than the gradient checkpointing baseline. To support multi-GPU training, we also develop a distributed implementation of StreamBP with special attention to gradient and parameter communication, significantly improving training efficiency and broadening its applicability. (C.2) Empirically, we measure the maximum sequence length and time cost of StreamBP under both single GPU and distribuetd training settings. Specifically, we measure the memory and time cost of BP in Section 4.1 . StreamBP substantially scales up the maximum sequence length by 2.8-5.5\times compared to the gradient checkpointing baseline across different model scales while achieving comparable or even less BP time. It is worth mentioning that StreamBP’s sequence length scaling ability can be directly transferred to batch size scaling for accelerating the training speed, as its memory cost can be linearly related to the sequence length. In Section 4.2 , we verify the effectiveness of StreamBP under various training objectives, showing that it significantly increases maximum sequence length under the objective of SFT, GRPO, and DPO. In Section 4.3 , we verify the effectiveness of StreamBP under Deepspeed ZeRO-2 distributed training scheme, where StreamBP achieves 5-5.6\times larger sequence length than gradient checkpointing. 2 Preliminary: Peak Memory Cost during Backpropagation Figure 1 : Memory profile during backpropagation of Qwen 3-4B with gradient checkpointing, visualized using PyTorch’s memory profiler. Sequence length is set to 8192. Optimizer states are excluded as we focus on the BP process. As a concrete example, we utilize PyTorch CUDA memory snapshot tool to record detailed GPU memory usage across 2 forward and backward processes of Qwen 3-4B model under the sequence length of 8192, where the gradient checkpointing technique is applied. The result is shown in Figure 1 . We exclude the memory cost of optimizer states and focus solely on the BP process. Peak memory cost. The peak memory cost occurs