SPAP: Structured Pruning via Alternating Optimization and Penalty Methods

Hanyu Hu, Xiaoming Yuan · 2025

Read SPAP: Structured Pruning via Alternating Optimization and Penalty Methods by Hanyu Hu, Xiaoming Yuan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-03373.txt.

Description

Hanyu Hu and Xiaoming Yuan propose in 2025 a structured pruning method for large language models that cuts whole parts of a network at once. Large models cost much memory and computing to run. Cutting whole parts can speed them up, but old methods lose quality, use rough rules, or need costly retraining.

Their method sets pruning as an optimization problem with whole number choices, uses a penalty to push choices toward low error, and alternates updates to recover quality. Tests on several known model families gave faster inference and lower memory at set sparsity, beating leading methods. Readers get a theory based route to smaller models that still perform well.

Catalog record

CREATORHanyu Hu, Xiaoming Yuan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-03373

Browse the full library · About the Tower of Babel