Prune, Update and Trim: Robust Structured Pruning for Large Language Models Diego Coello de Portugal Mecke, Tom Hanika, Lars Schmidth-Thieme Abstract Large Language Models (LLMs) have experienced significant growth and development in recent years. However, performing inference on LLMs remains costly, especially for long-context inference or in resource-constrained devices. This motivates the development of new post-training pruning (PTP) methods. These methods reduce LLMs’ requirements by removing a substantial part of the model’s parameters. The discarded weights are selected depending on their impact on the models performance. Current PTP methods prune the models by removing the less informative hidden nodes from the FFN layers, and the least important attention layers. We propose Putri , a PTP method that introduces three changes to the State-of-the-art. First, we update the un-pruned weights of the FFN to compensate for the introduced pruning error. Second, the FFN layers are pruned sequentially, taking into account the updates done to the previous layers. Third, instead of removing full attention layers, we remove individual attention-heads. We extend this method such that it can also address Grouped-Query Attention. In summary, Putri is a structure pruning method which remains simple while showing SOTA performance. Pruning experiments on multiple models with a wide variety of sparsity ranges and on different datasets, validate the generality of Putri. Notably, we demonstrate that, unlike previous methods, Putri can prune LLMs on extreme sparsity ratios. The code is available at: https://github.com/Coello-dev/Putri. 1 Introduction Large Language Models (LLMs) 27 ; 28 ; 5 show consistent improvements when increasing model scale, training data and compute. This enhanced performance has allowed LLMs to tackle a variety of tasks such as agentic workflow 10 ; 17 ; 37 and robotics 4 ; 8 ; 3 . Nevertheless, these tasks require the model to either run on a large number of tokens, which drastically increases the required memory, or run on a smaller device with less memory and computing capacity than data centers. These requirements have motivated the development of multiple post-training techniques that reduce the memory and computing requirements of LLMs by compressing model size. For example, Knowledge distillation 15 aims to train a smaller model ( student ) that mimics the prediction of the already trained bigger model ( teacher ). Even though Knowledge distillation achieves a highly optimized smaller model, it requires the computational effort to run inference on the teacher model and train the student model at the same time. Another approach is Quantization 36 , which reduces the memory requirements by reducing the floating-point precision of the model’s parameters. However, this technique is limited by the number of precision bits to store the weights. This sets a lower bound of how much it can compress the model. Another prominent method is Pruning 19 , which aims to remove the less informative parameters of a model while maintaining the original model performance. This allows Pruning to reduce the model size with a comparatively small computational effort (with respect to Knowledge Distillation). At the same time it is (in theory) capable of reducing the model size as much as required for a certain task (in contrast to Quantization). Nevertheless, in practice Pruning methods are not used due to its inability to meaningfully reduce the model’s size. The first Pruning methods aimed to remove the model’s weights without considering the model’s structure ( Unstructured pruning 44 ; 11 ). These methods result in models with highly sparse weight matrices, i.e., weight matrices where most of the values are zero. Although these procedures allow for pruning at high sparsity levels without significantly reducing the model’s performance, they do not always result in memory or speed gains proportional to the sparsity ratio 38 ; 42 . This inability to turn the sparsity into meaningful memory and speed-up gains is a result from current software and hardware not being able to benefit from sparse matrices. In particular, current computation libraries such as (Pytorch 25 , Keras 6 , etc) and accelerator hardware (GPUs, TPUs, etc.) are highly optimized for dense matrix operation and only partially benefit from sparse matrices. Even with specialized frameworks 24 , these Unstructured Pruning methods do not lead to relevant speed-up or memory gains. Notably, these marginal gains are not proportional to the model’s sparsity 11 . Structured Pruning methods 20 take a different approach. Their sparsification methods take into account the underlying model structure. This guarantees both a reduction of the memory footprint and a substantial speed-up for inference. A principled way to do this is to remove rows or columns from the weight matrices ( Width pruning 2 ). This reduces the number of computations required while maintaining matrices dense for faster computation. Another Structured Pruning approach, it to remove full transformer layers ( Depth pruning 22 ) or any other important substructure such as attention heads 16 . State-of-the-art pruning methods such as 2SSP 30 achieve efficient post-training compression by applying simple criteria. However, these methods leave several design choices unexplored. First, they do not update the remaining weights after a pruning step. This can lead to a significant reconstruction error at high sparsity. Second, when pruning attention layers, their approach works with a coarse granularity, i.e., full entire attention modules are removed instead of removing individual attention heads. Third, pruning multiple FFN layers independently ignores the distribution shift introduced by earlier pruning steps. We address these limitations by introducing Putri , a robust structured pruning method with the following contributions: • We show the importance of updating the unpruned weights. In particular, we demonstrate how some of the state-of-the-art (SOTA) pruning methods underperform for this reason. • Instead of discarding entire attention layers, we introduce a fine-grained attention head removal. Previous attention head pruning methods did only consider standard Multi-Head Attention (MHA). To the best of our knowledge, our approach is the first method to tackle Grouped-Query Attention (GQA), which is a standard component in many modern LLMs. • We sequentially prune the FFN layers, allowing each pruning decision to account for the perturbations introduced by the layers that were pruned earlier. • Our method not only achieves SOTA performance, but it also demonstrates that achieving 95% sparsity is possible with Structured Pruning. This opens the door to explore in the future Structured Pruning methods that could compete with other model compression approaches such as Quantization or Knowledge Distillation. 2 Related Work Pruning is a model compression technique that aims to remove the unimportant weights. This can be done in an unstructured manner where you remove some of the weights’ parameters depending on an importance score. For example, using the weights’ magnitude 44 , or given a calibration dataset, it can measure some statistics of the weights and use it to score and prune the weights’ parameters. This includes the expected squared error 34 , the standard deviation of the output 21 or minimize the error reconstruction using the Hessian to approximate the loss 11 . Even though Unstructured Pruning methods can maintain good performance even for high sparsity, they require specialized software to benefit from the sparse matrices 24 , they don’t have reliable speed-ups 2 and they do not reduce the memory requirements. This motivates the idea of pruning sub-structures of the model that can reliably lead to memory and speed-up improvements. 2.1 Structured Pruning Structured Pruning prunes the model while taking into account the underlying structure. The different pruning methods can be grouped as follows: Width pruning These pruning methods focus on reducing the number of columns and rows of the weights’ matrices 2 ; 30 . Intuitively, it prunes some of the hidden neurons and/or channels from the model. Depth pruning In this case, given a model with skip connections ( x_{l+1}=x_{l}+f_{l}(x_{l}) ) it aims to prune the model by removing the processing blocks f_{l}(x_{l}) such that x_{l+1}=x_{l} . This can be applied to the full transformer layer 22 ; 13 or to the individual sub-blocks, i.e., the FeedForward Layer and the Multi-Head Attention 43 ; 32 ; 30 . Head Pruning It prunes the attention layer by removing individual heads, which allows to remove multiple columns from the query, key and value matrices as well as multiple rows from the output matrix. To the best of our knowledge, existing methods can only be applied to standard Multi-Head Attention (MHA) 16 and not to Grouped-Query Attention (GQA). Therefore, they can not be used to prune LLMs with GQA such as Llama3 or Qwen3. 2.2 Grouped-Query Attention When performing inference with a transformer model, the process is done in an auto-regressive manner. This process recomputes many calculations, in particular, the key and values of previous tokens in the attention layer is recomputed for each auto-regressive step. In order to avoid that, LLMs store the previous tokens’ keys and values for faster inference (KV-cache). However, this only speed-ups the process but does not reduce the memory requirements. To improve the memory efficiency, Multi-Query Attention (MQA 31 ) was proposed. GQA is a variation of MHA that uses only one key and value head that matches all the query-heads. This also reduces the KV-cache required, making the inference process even faster. Nevertheless, MQA is not able to perform comparably to standard MHA. Grouped-Query Attention (GQA 1 ) is introduced as an option in between MHA and MQA, where each key-value head matches to multiple query heads. However, contrary to MQA there are multiple key-value heads instead of a single one. This allows GQA to benefit from the speedup by having less key-value heads than MHA and therefore less KV-cache, while still achieving the same performance as MHA. 3 Methodology Following 2SSP 30 and we split LLM pruning process in 2 steps. First, we prune the MLP by removing some intermediate nodes. However, we do this process in a sequential manner while additionally update the remaining weights in the process. Second, we prune the Attention layers, but instead of removing full attention layers, we remove individual heads from MHA or KV-heads from GQA. 3.1 MLP pruning Given a target sparsity for the MLP layers we prune all the MLP layers uniformly. Achieving the same sparsity level on all of them. For each MLP layer, the pruning process aims to remove all the connections related to the less important intermediate nodes/hidden representations. In order to prune the intermediate nodes, first a score is assigned to each individual node as follows: score(node_{i}^{(l)})=\|z_{i}^{(l)}\|_{2}^{2} (1) where i is the node index, l refers to the layer and z_{i}^{(l)} represents the output of the intermediate node or analogously, the input to the mlp.fc2 or mlp.down_proj . After removing the nodes with the least score, each un-pruned connection from the intermediate weights is reconstructed by minimizing the L_{2} -loss, i.e.: arg\min_{\hat{W}_{P}}\|XW-X_{P}\hat{W}_{P}\|_{2}^{2}=(X_{P}^{T}X_{P})^{-1}X_{P}^{T}XW (2) where X\in\mathbb{R}^{N\times M_{1}} and W\in\mathbb{R}^{M_{1}\times M_{2}} are the original inputs and weight parameters, while X_{P}\in\mathbb{R}^{N\times P} and \hat{W}_{P}\in\mathbb{R}^{P\times M_{2}} are the inputs of only the P unpruned intermediate nodes and their corresponding updated weights. N refers to the number of instances in the dataset, M_{1} the number of input features and M_{2} the number of output features. Applying this update, allows the remaining weights to reduce some of the error obtained from pruning some intermediate representations. In order to further improve performance, the MLP layers are not pruned in parallel but sequentially. This allows each MLP layer to be pruned taking into account the output of the previously pruned layers. Figure 1 : Diagram of the pruning process for SPU-H. First the FFN layers are pruned sequentially. Then the attention/kv-heads are pruned in an iterative manner. 3.2 Attention pruning Previous works 32 showed that deciding which full attention layers to remove using the perplexity on 1 sequence achieves comparable performance to using multiple sequences. This reduces dramatically the time required to prune. However, we argue that a more fine-grained approach, such as pruning each head, could lead to a better performance. Notice that some of the modern LLMs pruned use Grouped-Head Attention instead of Multi-Head Attention (MHA). We address this challenge by selecting the individual key-value heads to prune and all their associated q-heads. In the case of MHA, each key-value head has one unique query head associated, while for GQA there would be multiple query heads for each key-value head. In particular we select the key-value head to remove as follows: arg\min_{a\in\theta_{Attn}}perplexity(f(X;\theta_{FFN},\theta_{Attn}\backslash\{a\}),Y) (3) where X\in\mathbb{R}^{N\times M_{1}} are the inputs, \theta_{FFN} are the weight parameters from the FFN layers, \theta_{Attn} are the weight parameters from the Attention layers and a refers to a grouped head from \theta_{Attn} , including multiple query heads, a key head and a value head. We notice that selecting one grouped head at a time and recalculating the perplexity each time requires a lot of compute. We address this limitation by removing the H less informative heads each iteration, where H is the number of grouped heads in each of the original attention layer. 3.3 Attention vs FFN Pruning ratio Previous methods 30 showed that the ideal pruning ratio between the Attention and FFN layers is obtained by pruning a number of attention layers equal to: N_{Attn}=round(L\cdot s^{\frac{|\theta_{FFN}|}{\alpha|\theta_{Attn}|}}) (4) where L is the number of layers, s is the target sparsity, |\theta_{i}| is the number of parameters that the i part of the model has (with i\in\{FFN,Attn\} ) and \alpha is the hyperparameter controlling the ratio ( \alpha=1.5 by default). In our case, since we are pruning key heads instead of attention layers, we modify Equation 4 slightly to calculate the number of key-value heads as follows: N_{KV-Heads}=K\cdot round(L\cdot s^{\frac{|\theta_{FFN}|}{\alpha|\theta_{Attn}|}}) (5) where K is the number of key heads in each Grouped-Query Attention layer. 4 Experiments Previous Structured Pruning methods evaluate at a moderate sparsity range with maximum ratio of 50% sparsity. However, the final objective of pruning is to significantly reduce the number of parameters from the model, i.e., reduce the model size by at least an order of magnitude. This reduction would require of a sparsity ratio of at least 90%. Nevertheless, this scenario remains unexplored in the literature with no discussion or experimentation. In this section we will aim to not only show the performance of our method ( Putri ) but also fill in the gap in the literature regarding this sparsity range. 4.1 Set-Up In order to do a fair comparison between all the pruning method, we use an extensive set-up with a wide variety of model families, model sizes, datasets with different tasks, pruning methods and sparsities ranges. The specific configurations used in the experiments consider the following options: Models We evaluate the Llama-3 9 (including 3.0, 3.1 and 3.2 versions) and Qwen3 39 family of models across the different sizes available. Both of them are SOTA open-sourced modern LLMs with GQA 1 , RMSNorm 41 and RoPE embeddings 33 . Datasets For calibration during the pruning process we use the C4 dataset 29 . The pruned models’ perplexity is evaluated on the datasets C4 29 , Wikitext-2 (Wiki 23 ) and FineWeb-Edu (FW 26 ). Additionally, we also measure the zero-shot capabilities using LM Eval Harness 12 . We evaluate on multiple benchmarks such as Winogrande 18 , ARC (easy and challenge, "ARC-E" and "ARC-C" respectively) 7 , Hellaswag 40 , PiQA 35 and MMLU 14 . Baselines We evaluate against multiple post-training Strcutured Pruning methods, which include width-wise pruning approaches ( 2SSP 30 ) and depth-wise pruning on the transformer blocks ( ShortGPT 22 , UIDL 13 ) or the FFN and Attention sub-blocks ( Blockpruner 43 , Evopress 32 , 2SSP 30 ). Since most new open-sourced LLMs do not use MHA but GQA, we do not evaluate previous head pruning methods since they are not suitable for GQA. Sparsity As previously mentioned, we will focus on higher sparsity ranges. Therefore we would start from 50% sparsity and prune until a sparsity of 99%. Due to space constrains, we delegate to the Appendix the following results. First, the perplexity results on the test partition of the C4 dataset. We consider Wikitext-2 and FineWeb-Edu more interesting datasets since the models are pruned with the train partition of C4. Therefore, this other datasets would better show the generalization to text not following the distribution of the calibration dataset C4. Second, the smaller models from the Llama3 family. Qwen3 has shown in general better capabilities than Llama3 and therefore we only show results on the bigger Llama3 models we are able to run. Lastly, we measure for completeness the performance of the pruning methods in lower sparsities such as 25% and 37.5%. Table 1 : Perplexity comparison of multiple pruning methods on Qwen3 and LLama3 models. We add some notations to specify the following cases: ‘OOM’ is an out-of-memory error, ‘nan’ refers to an invalid pruned model output or an error during the pruning process, ‘ \infty ’ is a successfully pruned model that returns infinite when the perplexity is evaluated. Sparsity Method Llama-3.1-8B Qwen3-14B Qwen3-32B Wiki FW Wiki FW Wiki FW 0.0 - 6.406 8.375 8.641 9.43 7.594 8.32 0.5 UIDL 52736 38656 \infty \infty \infty \infty ShortGPT 14208 10432 1318 622.5 \infty \infty BlockPruner 163 163 221 131.5 39.25 35.03 EvoPress 148 96 86.94 73.75 OOM OOM 2SSP 34.75 41.25 23.53 25 19.39 21.67 Putri (ours) 39.25 41.75 18.19 19.97 15.16 16.95 0.75 UIDL 344064 196608 \infty \infty \infty \infty ShortGPT 49664 49664 \infty \infty 64736 \infty BlockPruner 98816 28288 5524 4336 5884 3224 EvoPress 17152 8640 7208 4840 OOM OOM 2SSP 278 260 212.6 160.5 122.1 99.62 Putri (ours) 163 185 124.5 68.75 44.66 41.78 0.9 UIDL 1859584 21364736 \infty \infty nan nan ShortGPT 1859584 21364736 \infty \infty nan nan BlockPruner 501760 684032 nan nan 26992 27840 EvoPress 28288 14208 9928 7208 OOM OOM 2SSP 1544 908 5316 2662 2866 1476 Putri (ours) 644 416 1307 716.5 370.2 215.1 0.95 UIDL 501760 501760 \infty \infty \infty \infty ShortGPT 532480 391168 \infty \infty \infty \infty BlockPruner 1204224 995328 nan nan nan nan EvoPress 135168 98816 10816 8360 OOM OOM 2SSP 8096 2976 nan nan 29872 23264 Putri (ours) 1696 1096 4916 1880 2714 871 0.99 UIDL nan nan nan nan nan nan ShortGPT 532480 501760 \infty \infty \infty \infty BlockPruner 532480 501760 nan nan nan nan EvoPress nan nan nan nan OOM OOM 2SSP 34048 17152 nan nan nan nan Putri (ours) 11776 5216 46624 39584 \infty \infty 4.2 Main experiment Even though there are speed-up and memory benefits when pruning a model up to 50%, those benefits need to be of at least an order of magnitude in order to be deployed in real world scenarios. A speed-up or memory reduction of an order of magnitude can only be achieved with a sparsity 90% or higher. Given this reason, we focus our evaluation of the pruning methods on sparsity levels from 50% sparsity up to 99%. To accurately measure the generalization capabilities of the pruning methods, we evaluate the perplexity over many datasets and multiple models in Table 1 . The consistent improvements in Table 1 show the robustness of our method compared to the baselines. Additionally, we also show that most pruning methods are not able to properly prune on high sparsity degrees. In particular, they either return models with infinite perplexity (denoted with \infty ) or the pruning method is not able to successfully finish due to not being able to handle such a high sparsity (denoted with ‘ nan ’). To further investigate the collapse of the pruning methods, we evaluate them on a wider range of model sizes from the Qwen3 family of models. We particularly focus on the smaller models since achieving a higher sparsity degree on them is an even harder task compared to the bigger models. The results in Table 2 show that not only Putri improvements are consistent across the model sizes, but also it is able to successfully prune to a much higher sparsity degree than previous methods. Nevertheless, pruning the smaller models with a pruning sparsity of 99% is not even possible for Putri. This further highlights the difficulty of pruning in this highly sparse ranges. The complete list of results for all the different models and set-ups can be viewed in the Appendix. Table 2 : Perplexity comparison across different model sizes. We add some notations to specify the following cases: ‘OOM’ is an out-of-memory error, ‘nan’ refers to an invalid pruned model output or an error during the pruning process, ‘ \infty ’ is a successfully pruned model that returns infinite when the perplexity is evaluated. Spars. Method Qwen3-0.6B Qwen3-1.7B Qwen3-4B Qwen3-8B Wiki FW Wiki FW Wiki FW Wiki FW 0 - 20.97 20.69 16.75 16.11 13.75 13.54 9.734 10.48 0.5 UIDL \infty \infty \infty \infty 3768 2946 \infty \infty ShortGPT \infty \infty 58944 11072 2570 1499 1787 639.5 BlockPruner 409.8 328 923.5 446.5 143.2 120.7 202.9 108.2 EvoPress 356 272 7792 5976 117.9 100 211 120.7 2SSP 371.8 255.4 166.2 109.9 63.84 56.56 41.94 40.09 Putri (ours) 108.2 97.69 65.62 55.47 40.97 36.44 28.44 27.3 0.75 UIDL \infty \infty \infty \infty \infty \infty \infty \infty ShortGPT \infty \infty \infty \infty \infty \infty \infty \infty BlockPruner 20528 13256 \infty \infty 14560 10088 13464 5884 EvoPress 6716 4952 9544 6984 23264 14904 5112 3768 2SSP 2866 2198 1080 768.5 1596 1063 536.5 395.8 Putri (ours) 1441 1038 884.5 647 309.2 207.6 164.2 140 0.9 UIDL \infty \infty \infty \infty \infty \infty \infty \infty ShortGPT \infty \infty \infty \infty \infty \infty \infty \infty BlockPruner nan nan nan nan nan nan nan nan EvoPress \infty 64736 51616 23824 58496 54528 34112 27408 2SSP nan nan 30816 57568 nan nan 12352 9256 Putri (ours) 4440 2414 2610 1685 916.5 705.5 1030 617.5 0.95 UIDL nan nan nan nan nan nan nan nan ShortGPT \infty \infty \infty \infty \infty \infty \infty \infty BlockPruner nan nan nan nan nan nan nan nan EvoPress \infty \infty \infty \infty \infty \infty \infty \infty 2SSP nan nan nan nan nan nan \infty \infty Putri (ours) \infty \infty 9400 6068 3596 2258 3124 1540 4.3 Zero-shot Even though perplexity is a good measure on the model’s performance, the main goal is for the LLM to be able to correctly answer and solve different tasks. To evaluate that, we compare the model on a list of benchmarks: Winogrande (WG), ARC-E, ARC-C, HellaSwag (HS), PiQA and MMLU. The results on Qwen3-14B are shown in Table 3 . In Table 3 , it can be observed that Putri is overall the best method. However, in those cases were it is the second best (90% and 99%) it can be observed in Table 1 , that the best method (UIDL) has actually infinite perplexity. Meaning, that the “good” performance in the zero-shot benchmark is a lucky guess rather than an informed decision. These can also be observed by the fact that at high perplexities all the methods converge to a result close to random. Notice that most of these benchmarks and questions with a set of options to choose from. Therefore, even a relatively high accuracy of 33% or 25% is nothing more than a random guess out of 3 or 4 options. This observation highlights again the big gap in the literature when pruning at high sparsities. Table 3 : Zero-shot accuracy comparison with Qwen3-14B. ‘Avg. Acc.’ refers to the average accuracy over all the zero-shot tasks and ‘nan’ refers to an invalid pruned model output or an error during the pruning process Sparsity Method WG ARC-E ARC-C HS PiQA MMLU Avg. Acc. 0 - 72.8 84.2 58.7 61.0 80.0 77.3 72.3 0.5 UIDL 52.8 28.3 24.4 28.0 56.7 23.0 35.5 ShortGPT 50.7 30.5 19.3 26.4 53.6 23.5 34.0 BlockPruner 53.0 38.3 22.0 30.2 56.4 25.5 37.6 EvoPress 48.5 41.1 22.2 31.8 61.3 23.9 38.1 2SSP 58.4 46.6 22.7 39.6 66.9 23.6 43.0 Putri (ours) 62.0 48.8 24.0 42.3 69.7 23.8 45.1 0.75 UIDL 48.8 26.3 22.1 25.8 54.0 25.9 33.8 ShortGPT 47.8 24.8 22.4 25.7 51.7 25.3 33.0 BlockPruner 49.0 26.3 23.4 25.9 53.0 24.7 33.7 EvoPress 51.6 24.6 22.0 25.6 53.4 23.9 33.5 2SSP 49.9 28.9 17.1 28.3 56.3 24.2 34.1 Putri (ours) 49.0 33.1 17.7 28.6 56.3 23.0 34.6 0.9 UIDL 53.4 25.6 23.2 25.5 53.8 24.3 34.3 ShortGPT 50.3 24.7 22.3 25.7 52.6 22.9 33.1 BlockPruner nan nan nan nan nan nan nan EvoPress 48.3 24.6 21.3 25.8 52.2 23.2 32.6 2SSP 50.3 26.3 21.4 26.1 52.6 23.4 33.3 Putri (ours) 51.6 26.5 20.6 26.2 53.0 25.4 33.9 0.95 UIDL 51.1 25.8 21.5 25.5 52.3 25.1 33.6 ShortGPT 49.4 24.5 22.8 25.2 52.6 25.2 33.3 BlockPruner nan nan nan nan nan nan nan EvoPress 51.1 25.6 22.8 25.4 51.8 24.6 33.6 2SSP nan nan nan nan nan nan nan Putri (ours) 49.5 25.3 23.2 26.1 53.5 24.5 33.7 0.99 UIDL nan nan nan nan nan nan nan ShortGPT 50.8 24.3 22.8 25.4 53.0 26.9 33.9 BlockPruner nan nan nan nan nan nan nan EvoPress nan nan nan nan nan nan nan 2SSP nan nan nan nan nan nan nan Putri (ours) 50.8 25.7 20.8 25.7 53.3 24.4 33.4 4.4 Ablation study Our method Putri incorporates 3 main components when compared to previous methods such as 2SSP: updating the weights of the FFN layers, pruning the FFN layers sequentially to take into account the previous layers pruning, and pruning attention heads instead of the full attention layer. To validate the usefulness of these additions, we perform the ablation experiments in Figures 2 and 3 where each individual component is removed. We denote ‘Putri (no FFN upd.)’ to not updating the un-pruned FFN weights after removing the other weights, ‘Putri (full Attn.)’ to remove the full attention layer instead of the heads, and ‘Putri (parallel upd.)’ to not taking into account the result of the previously pruned FFN layers by using the output of the original model as an input, instead of the already pruned previous layers. Whenever the pruning method was not able to properly prune the model and resulted in a ‘nan’ error, it is not plotted in the graph. This happens for some methods at 95% and/or 99% sparsity. We observe in Figures 2 and 3 that overall, removing any of these changes results in a decrease in performance. In particular, not updating the FFN weights significantly reduces the performance, especially for higher sparsities. We also observe that at even though some of the other combinations can in particular scenarios compete and sometimes even beat the full method, the overall trend is that the complete method performs better than removing any of the individual parts. Figure 2 : Ablation study on Qwen3-14B about the individual components of Putri. Figure 3 : Ablation study on Qwen3-8B about the individual components of Putri. 5 Conclusion In this paper we introduce Putri, a post-training structure pruning approach. Our method can even be applied in high-sparsity regimes. In contrast to previous SOTA methods, Putriintroduces three main changes. First, it updates the un-pruned parameters in the last linear layer from the FFN to reduce errors induced by earlier steps. Second, the FFN pruning is done sequentially so that each layer takes into account the input from the previously pruned FFN layers. This not only improves performance but also reduces the memory requirements since only one layer at a time needs to be in memory. Third, our method takes into account the fact some attention layers depend highly on specific heads rather than in the combination of all of them. Therefore, Putri prunes on a attention head level and extends this approach to tackle Grouped-Query Attention. Multiple experiments on the Llama3 and Qwen3 family models validate our approach. We compare against multiple pruning methods in a wide range of sparsities and multiple datasets using perplexity performance. Additionally, the zero-shot performance is evaluated on multiple benchmarks. All of these experiments validate Putri and demonstrate the broad scope of enhanced performance across various settings. In particular, we show that Putri does not only improve in pruning performance, but it is also able to prune the models to a higher sparsity levels than other methods. This robustness to high sparsities degrees is especially interesting since model compression starts being useful when the reduction is at least one order of magnitude, i.e., at least 90% sparsity. Lower sparsities (25% or 37.5%) even though interesting from a research perspective, are less applicable for real world scenarios, since the memory and speed-up benefits are not sufficient. This paper explores the short comings of current structured pruning methods in high sparsity regimes. Even though, it is a new field, we hope that by exploring higher sparsity regimes future research can develop pruning methods with gains comparable to Quantization and Knowledge Distillation. This would open a new avenue to reduce the requirements of LLMs or possibly, improving current techniques by augmenting it with Structured Pruning. Appendix A Appendix A.1 Additional results As mentioned in the main paper, we delegate some of the results to this section due to space constrains. We will in this section add an extensive list of all the experiment results. To not extend the tables over the margin constrains we abbreviate sparsity with S., EvoPress with E.V., ShortGPT with Short. and BlockPruner with B.P. Additionally, results over 10000 would be rounded to 10K notation. Spars. Method Qwen3-0.6B Qwen3-1.7B Qwen3-4B Wiki FW C4 Wiki FW C4 Wiki FW C4 0 - 20.97 20.69 29.45 16.75 16.11 22.28 13.75 13.54 20.05 0.25 UIDL 23K 21K 33K \infty \infty \infty 1994 555.5 771.5 Short 189.1 146.6 187.6 104.8 103.2 134.1 48.56 43.28 62.34 B.P. 41.28 38.41 52 30.86 28.94 39.03 20.41 21.02 29.56 E.P. 41.12 39.88 53.75 28.33 27.67 37.91 22.67 22.02 30.5 2SSP 48 42.19 56.56 26.41 26.66 34.38 20.48 18.44 24.75 Putri 35.03 32.66 42.59 22.11 23.77 28.83 15.77 17.38 22.36 0.375 UIDL \infty \infty \infty 4544 2704 4688 828 733.5 831 Short 5568 2790 3710 54944 23440 47744 198.1 189.1 236.2 B.P. 110.8 102 131 159.2 90.38 125.9 51.78 51 70.94 E.P. 125.4 103.2 136.2 106.1 74.31 101.6 44.31 42.03 58.34 2SSP 112.4 94.69 118.3 50.59 46.69 54.19 35.19 28.72 35.75 Putri 52.19 51.28 62.59 32.78 32.28 36.22 23.86 26.14 30.62 0.5 UIDL \infty \infty \infty \infty \infty \infty 3768 2946 3950 Short \infty \infty \infty 58944 11072 12848 2570 1499 1780 B.P. 409.8 328 398.8 923.5 446.5 566.5 143.2 120.7 163.6 E.P. 356 272 358.8 7792 5976 6664 117.9 100 136.2 2SSP 371.8 255.4 311.8 166.2 109.9 112.4 63.84 56.56 62.59 Putri 108.2 97.69 112 65.62 55.47 58.34 40.97 36.44 42.19 0.75 UIDL \infty \infty \infty \infty \infty \infty \infty \infty \infty Short \infty \infty \infty \infty \infty \infty \infty \infty \infty B.P. 21K 13K 14K \infty \infty \infty 15K 10K 15K E.P. 6716 4952 5792 9544 6984 7852 23K 15K 23K 2SSP 2866 2198 2294 1080 768.5 790 1596 1063 1042 Putri 1441 1038 1114 884.5 647 649.5 309.2 207.6 224.5 0.9 UIDL \infty \infty \infty \infty \infty \infty \infty \infty \infty Short \infty \infty \infty \infty \infty \infty \infty \infty \infty B.P. nan nan nan nan nan nan nan nan nan E.P. \infty 65K 64K 52K 24K 30K 58K 55K 61K 2SSP nan nan nan 30816 58K 42K nan nan nan Putri 4440 2414 2404 2610 1685 1705 916.5 705.5 711 0.95 UIDL \infty \infty \infty \infty \infty \infty \infty \infty \infty Short. \infty \infty \infty \infty \infty \infty \infty \infty \infty B.P. nan nan nan nan nan nan nan nan nan E.P. \infty \infty \infty \infty \infty \infty \infty \infty \infty 2SSP nan nan nan nan nan nan nan nan nan Putri \infty \infty \infty 9400 6068 6116 3596 2258 2156 0.99 UIDL \infty \infty \infty \infty \infty \infty \infty \infty \infty Short \infty \infty \infty \infty \infty \infty \infty \infty \infty B.P. nan nan nan nan nan nan nan nan nan E.P. nan nan nan nan nan nan nan nan nan 2SSP nan nan nan nan nan nan nan nan nan Putri 64K 29K 27K \infty \infty \infty \infty \infty \infty Spars. Meth. Qwen3-8B Qwen3-14B Qwen3-32B Wiki FW C4 Wiki FW C4 Wiki FW C4 0 - 9.734 10.48 15.19 8.641 9.43 13.73 7.594 8.32 12.26 0.25 UIDL 5072 739 551.5 23.03 24.28 34.5 \infty \infty \infty Short. 38.56 33.91 46.16 39.56 38.12 52.19 14.16 15.31 23.2 B.P. 22.67 18.66 25.59 16.98 17.28 24.23 10.06 11.34 16.14 E.P. 22.02 18.66 26 14.95 15.19 21.3 OOM OOM OOM 2SSP 14.49 16.17 20.92 11.51 12.91 16.88 10.02 11.31 15.34 Putri 13.54 15.83 19.2 10.62 12.28 15.46 9.13 13.91 10.48 0.375 UIDL 36320 \infty \infty 30336 15376 16368 \infty \infty \infty Short. 246.6 145 182.5 151.4 131 161.8 28.44 29.8 42.09 B.P. 47.53 39.31 53.03 46.78 39.47 54.81 14.22 15.58 22.06 E.P. 42.59 38.25 51.19 25.34 24.19 34.03 OOM OOM OOM 2SSP 21.12 24.42 28.66 15.25 16.98 21.05 13.43 15.31 19.47 Putri 18.25 20.2 23.3 13.51 15.61 18.28 11.03 15.46 12.28 0.5 UIDL \infty \infty \infty \infty \infty \infty \infty \infty \infty Short. 1787 639.5 655 1318 622.5 697 \infty \infty \infty B.P. 202.9 108.2 143.2 221 131.5 181.9 39.25 35.03 48 E.P. 211 120.7 151.9 86.94 73.75 101.6 OOM OOM OOM 2SSP 41.94 40.09 44.22 23.53 25 29.75 19.39 21.67 26.56 Putri 28.44 27.3 30.09 18.19 19.97 22.98 15.16 16.95 20.2 0.75 UIDL \infty \infty \infty \infty \infty \infty \infty \infty \infty Short. \infty \infty \infty \infty \infty \infty 65K \infty \infty B.P. 13K 5884 5884 5524 4336 4544 5884 3224 3568 E.P. 5112 3768 4042 7208 4840 5316 OOM OOM OOM 2SSP 536.5 395.8 385 212.6 160.5 151.9 122.1 99.62 94.31 Putri 164.2 140 149.6 124.5 68.75 72.31 44.66 41.78 44.56 0.9 UIDL \infty \infty \infty \infty \infty \infty nan nan nan Short. \infty \infty \infty \infty \infty \infty nan nan nan B.P. nan nan nan nan nan nan 27K 28K 28K E.P. 34K 27K 27K 9928 7208 7976 OOM OOM OOM 2SSP 12K 9256 7436 5316 2662 2432 2866 1476 1282 Putri 1030 617.5 634.5 1307 716.5 673 370.2 215.1 214.2 0.95 UIDL \infty \infty \infty \infty \infty \infty \infty \infty \infty Short. \infty \infty \infty \infty \infty \infty \infty \infty \infty B.P. nan nan nan nan nan nan nan nan nan E.P. \infty \infty \infty 11K 8360 9328 OOM OOM OOM 2SSP \infty \infty \infty nan nan nan 30K 23K 25K Putri 3124 1540 1571 4916 1880 1583 2714 831 871 0.99 UIDL \infty \infty \infty nan nan nan nan nan nan Short. \infty \infty \infty \infty \infty \infty \infty \infty \infty B.P. nan nan nan nan nan nan nan nan nan E.P. nan nan nan nan nan nan OOM OOM OOM 2SSP nan nan nan nan nan nan nan nan nan Putri nan nan nan 47K 40K 39K \infty \infty \infty Sparsity Method Meta-Llama-3-8B Llama-3.1-8B Wiki FW C4 Wiki FW C4 0 - 6.312 8.5 8.125 6.406 8.375 8.25 0.25 UIDL 148 96 153 2976 3168 2976 Short. 223 174 191 260 296 278 B.P. 14.44 17.5 20.38 14.69 18 21 E.P. 12.94 14.44 18.88 13.56 15.62 20.12 2SSP 11.81 14 16.62 12 14.44 16.62 Putri 12.38 19.75 21.75 11.81 13.38 14.94 0.375 UIDL 392 286 368 111616 20736 15168 Short. 1408 856 968 8640 18304 14208 B.P. 32 32 43.25 27 29.25 37.5 E.P. 31.12 29.25 39.25 26.25 27 34.25 2SSP 21.38 22 24.25 20.38 21.75 23.5 Putri 16.88 19.5 20.75 17.5 19.75 21.75 0.5 UIDL 1000 688 732 52736 38656 34048 Short. 56320 49664 46592 14208 10432 8640 B.P. 131 99 108.5 163 163 185 E.P. 96 99 108.5 148 96 105 2SSP 34.75 40.5 46.75 34.75 41.25 47.5 Putri 32 45.25 45.25 39.25 41.75 46 0.75 UIDL 17152 15168 17152 344064 196608 391168 Short. 1204224 1359872 2113536 49664 49664 56320 B.P. 7136 8640 6720 98816 28288 26624 E.P. 20736 8640 7616 17152 8640 8640 2SSP 346 268 268 278 260 252 Putri 216 237 223 163 185 203 0.9 UIDL 880640 8912896 8912896 1859584 21364736 10092544 Short. 323584 152576 162816 1859584 21364736 10092544 B.P. 119296 184320 126976 501760 684032 643072 E.P. 46592 41216 38656 28288 14208 20736 2SSP 1592 1032 940 1544 908 908 Putri 568 458 472 644 416 472 0.95 UIDL 569344 501760 442368 501760 501760 415744 Short. 251904 143360 184320 532480 391168 471040 B.P. 827392 778240 827392 1204224 995328 995328 E.P. 92672 68096 72192 135168 98816 98816 2SSP 4928 2976 2976 8096 2976 2976 Putri 2048 828 880 1696 1096 968 0.99 UIDL \infty \infty \infty – – – Short. 569344 569344 569344 532480 501760 442368 B.P. 569344 569344 569344 532480 501760 442368 E.P. nan nan nan nan nan nan 2SSP 22016 14208 17152 34048 17152 17152 Putri 24960 9152 8096 11776 5216 3824 Sparsity Method Llama-3.2-1B Llama-3.2-3B Wiki FW C4 Wiki FW C4 0 - 10.25 12 12.94 8.375 10.12 10.75 0.25 UIDL 7616 2976 4608 1496 908 1280 Short. 245 237 245 158 168 203 B.P. 42.5 38.75 54.5 23.5 22.75 29.25 E.P. 51.25 48.25 54.5 22.75 21.38 26.62 2SSP 18.62 20.12 25.38 16.12 19.5 20.75 Putri 16.38 19.5 21.75 16.88 23.12 27.5 0.375 UIDL 52736 34048 34048 7616 5568 5568 Short. 260 191 209 458 416 486 B.P. 203 174 191 53.75 51.25 54.5 E.P. 163 119.5 148 50.5 46.75 54.5 2SSP 31.62 35.75 50.5 26.62 29.25 32.5 Putri 24.62 26.62 30.12 31.62 45.25 54.5 0.5 UIDL 222208 162816 173056 46592 34048 36352 Short. 3168 1592 1592 11072 9792 11776 B.P. 1592 968 1032 139 127 153 E.P. 536 392 404 163 135 153 2SSP 56.25 79.5 77 49.75 49 54.5 Putri 35.75 38.75 53.75 40 40.5 48.25 0.75 UIDL 778240 684032 880640 152576 126976 126976 Short. 442368 1359872 6094848 38656 14208 19456 B.P. 14208 8096 7136 4928 2976 2976 E.P. 28288 18304 28288 7616 5216 5568 2SSP 472 368 404 520 378 378 Putri 158 209 216 191 209 216 0.9 UIDL \infty \infty \infty \infty \infty \infty Short. \infty \infty \infty \infty \infty \infty B.P. \infty \infty \infty 152576 173056 173056 E.P. 49664 56320 59904 12544 8640 8096 2SSP 2976 1408 1456 2464 1368 1320 Putri 1096 856 752 752 604 664 0.95 UIDL \infty \infty \infty \infty \infty \infty Short. \infty \infty \infty \infty \infty \infty B.P. nan nan nan \infty \infty \infty E.P. nan nan nan 501760 143360 126976 2SSP 17152 19456 14208 5568 2896 2976 Putri 2624 1320 1240 1984 1240 1000 0.99 UIDL \infty \infty \infty \infty \infty \infty Short. \infty \infty \infty \infty \infty \infty B.P. nan nan nan \infty \infty \infty E.P. nan nan nan nan nan nan 2SSP nan nan nan 13376 11776 11776 Putri nan nan nan 10432 5920 5920