FastForward Pruning: Efficient LLM Pruning via Single-Step Reinforcement Learning

Xin Yuan 1∗ , Siqi Li 1∗ , Jiateng Wei 1 , Chengrui Zhu 1 , Yanming Wu 1 Qingpeng Li 1 , Jiajun Lv 1 , Xiaoke Lan 3 , Jun Chen 2† , Yong Liu 1† · 2025

Read FastForward Pruning: Efficient LLM Pruning via Single-Step Reinforcement Learning by Xin Yuan 1∗ , Siqi Li 1∗ , Jiateng Wei 1 , Chengrui Zhu 1 , Yanming Wu 1 Qingpeng Li 1 , Jiajun Lv 1 , Xiaoke Lan 3 , Jun Chen 2† , Yong Liu 1† (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-18977.txt.

Introduction

Xin Yuan and nine co-authors present FastForward Pruning, a faster way to decide how much of each layer of a large language model can be cut. Pruning shrinks models, but the best cut is uneven across layers, and searching for it with reinforcement learning is powerful yet prohibitively slow at this scale. Their core move is to decouple the search: a single-step learning framework optimises the policy apart from the separate problem of meeting the overall size budget.

A curriculum that starts with cheap, simple tasks and grows harder cuts the search cost further. Across three model families, the framework finds pruning plans that beat rival methods on the balance of size and performance. The work targets teams that must compress frontier-scale models on a realistic compute budget.

Catalog record

CREATORXin Yuan 1∗ , Siqi Li 1∗ , Jiateng Wei 1 , Chengrui Zhu 1 , Yanming Wu 1 Qingpeng Li 1 , Jiajun Lv 1 , Xiaoke Lan 3 , Jun Chen 2† , Yong Liu 1†
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-18977

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FastForward Pruning: Efficient LLM Pruning via Single-Step Reinforcement Learning Xin Yuan 1∗ , Siqi Li 1∗ , Jiateng Wei 1 , Chengrui Zhu 1 , Yanming Wu 1 Qingpeng Li 1 , Jiajun Lv 1 , Xiaoke Lan 3 , Jun Chen 2† , Yong Liu 1† Abstract Pruning is an effective method for compressing Large Language Models (LLMs), but finding an optimal, non-uniform layer-wise sparsity allocation remains a key challenge. While heuristic methods are fast but yield suboptimal performance, more powerful search-based approaches like Reinforcement Learning (RL) are often hindered by prohibitive computational costs on large-scale models. To overcome this efficiency barrier, we propose FastForward Pruning. Its core is a decoupled, single-step RL framework that separates policy optimization from the complex budget satisfaction problem. Such a decoupling is crucial for efficiently searching the vast policy space of LLMs. This curriculum-based strategy begins with low-cost, simple tasks and gradually increases in complexity, significantly reducing the search’s computational overhead. Evaluated on the LLaMA, Mistral, and OPT model families, our framework discovers pruning policies that achieve superior performance over strong heuristic baselines. Crucially, when compared to other search-based algorithms, our method achieves competitive or superior results at a fraction of the computational cost, demonstrating a clear advantage in search efficiency. † † address: 1 APRIL Lab, Zhejiang University 2 Zhejiang Normal University 3 College of Internet of Things Technology, Hangzhou Polytechnic {theoyuan,lsq4747}@zju.edu.cn , [email protected] , [email protected] 1 Introduction Large Language Models (LLMs) have demonstrated exceptional performance across a wide range of tasks [ 1 , 2 , 3 , 4 , 5 , 6 ] . However, their substantial computational and storage requirements pose considerable challenges to practical deployment. Pruning provides an effective way to address this issue by reducing model size and computation. Among various pruning methodologies, structured pruning is regarded as a practical technique due to its ability to facilitate direct inference acceleration on general-purpose hardware [ 7 ] . This technique is implemented by setting a sparsity ratio for the model’s prunable units, with the most straightforward approach being uniform pruning, where a single ratio is applied across all layers. However, studies indicate that layers in LLMs exhibit significant heterogeneity in pruning sensitivity [ 8 , 9 , 10 ] . This transforms the task of pruning into a challenging optimization problem: how to determine an optimal, non-uniform sparsity allocation that maximizes model performance under a global budget. Figure 1 : Our single-step RL paradigm (d) avoids the key drawbacks of legacy approaches for LLMs: (a) sparse rewards in sequential decision-making [ 11 ] , (b) complex state representations [ 12 ] , and (c) multi-agent coordination challenges [ 13 ] . The pursuit of this goal has primarily led to two categories of pruning methods: heuristic and search-based approaches. Heuristic strategies, such as SliceGPT [ 14 ] , FLAP [ 15 ] , and SVD-LLM [ 16 ] , determine layer-wise sparsity or parameter reduction based on predefined, handcrafted rules. While these methods are computationally efficient and have evolved into strong performance baselines, they are often limited by their fixed, local assumptions, which may fail to capture the complex inter-layer dependencies that are crucial for preserving model capabilities [ 17 , 18 ] . To overcome these limitations, search-based approaches were introduced to globally explore the vast policy space. These search-based methods can be divided into two main paradigms: zero-order optimization [ 19 , 20 ] and gradient-guided methods [ 21 , 11 , 13 ] . Zero-order methods like Evolutionary Algorithms (EAS) [ 19 , 20 ] suffer from immense computational cost and instability, often failing to outperform strong heuristics. Gradient-guided methods like Reinforcement Learning (RL) [ 22 , 23 , 24 ] theoretically offer higher sample efficiency by using policy gradients to guide the search. However, pioneering RL frameworks are less efficient for LLMs. As illustrated in Figure 1 , these legacy paradigms suffer from flaws that render them inefficient for large-scale models. These flaws include prohibitively slow sequential decision-making, computationally expensive state representations, and complex multi-agent coordination. Figure 2 : An overview of our FastForward Pruning framework, consisting of two main stages. Stage 1: Efficient Search , where the search process is accelerated by our Progressive Scheduling (ProgSched) mechanism. It uses a unified schedule \alpha(t) to govern a Sparsity Curriculum (SC) for task difficulty and a Fidelity Curriculum (FC) for evaluation cost. An RL Agent outputs raw retention scores, which are then converted into a budget-compliant policy by the Policy Mapping module. Stage 2: Calibration , where the final pruned weights undergo a retraining-free calibration via Ridge Regression to recover performance. To resolve these shortcomings, we propose FastForward Pruning. Its core innovation is a decoupled design that separates the learning of layer importance from the mechanics of budget satisfaction. In this design, the RL agent learns a high-level vector of unconstrained retention scores, while a deterministic function maps these scores to a budget-compliant policy. This fundamental separation enables the formulation of the search as a stable, single-step policy generation task, avoiding the complexities of sequential RL on LLMs. To further manage the cost of this now-practical search, we introduce a Progressive Scheduling mechanism that dynamically adjusts task difficulty and evaluation fidelity. Our framework thus makes automated pruning search not just powerful, but also practical for large-scale models. 2 Methodology Our framework addresses the challenge of designing an efficient search strategy for discovering near-optimal, layer-wise retention policies \mathbf{p}=[p_{1},\dots,p_{N}] . As illustrated in Fig. 2 , our methodology is a two-stage pipeline: (1) an efficient search stage, where we formulate the problem as a direct policy optimization task, akin to a contextual bandit problem, which is accelerated by our novel Progressive Scheduling (ProgSched) mechanism; and (2) a performance recovery stage, which employs a lightweight, retraining-free weight calibration to compensate for pruning-induced performance loss. 2.1 Reinforcement Learning Formulation for Pruning A key challenge in policy learning is the credit assignment problem, which is exacerbated in a constrained action space. The global budget imposes a tight coupling among the retention rates of all layers; an increase in one layer’s retention must be compensated by a decrease elsewhere. Consequently, a single policy update is a composite action, and the resulting reward signal cannot be uniquely attributed to any single dimension of the action. This ambiguity blurs the policy gradient, leading to an unstable and inefficient search. We resolve this by decoupling policy learning from budget satisfaction. The RL agent learns a vector of unconstrained importance scores, where independent gradients provide a clear learning signal. The complex budget constraint is offloaded to a deterministic mapping function (Alg. 1 ), significantly improving search stability and sample efficiency. RL Environment. We construct a focused RL environment with the sole purpose of enabling the agent to efficiently learn the relative importance of each layer. The state ( S_{t} ) is defined solely by the global target sparsity ratio \sigma_{t} . This deliberate simplification allows the agent to learn a policy structure that is highly effective for transfer learning to new sparsity targets within a given model architecture, conditioned only on the global budget. This state is varied to form a learning curriculum (Sec. 2.2 ). The action ( A_{t} ) is a vector of

Read the complete text