Lightweight and Post-Training Structured Pruning for On-Device Large Lanaguage Models

Zihuai Xu Yang Xu Hongli Xu Yunming Liao Zhiwei Yao Zuan Xie · 2025

Read Lightweight and Post-Training Structured Pruning for On-Device Large Lanaguage Models by Zihuai Xu Yang Xu Hongli Xu Yunming Liao Zhiwei Yao Zuan Xie (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-15255.txt.

Description

Zihuai Xu and co-authors present COMP, a light pruning method that shrinks large language models after training in 2025. Trimming weights in a structured way suits small devices. Old methods need costly fine tuning to regain lost skill. That cost bars on device use. Other post training methods demand special parts or shapes.

COMP first drops whole layers by importance, then trims neurons in what remains. A new matrix based score judges neuron worth. Mask tuning then restores accuracy without full fine tuning. Memory use falls sharply. On a known seven billion weight model at twenty percent pruning, COMP beat a rival method by a clear margin. The method keeps quality while cutting size.

Catalog record

CREATORZihuai Xu Yang Xu Hongli Xu Yunming Liao Zhiwei Yao Zuan Xie
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-15255

Browse the full library · About the Tower of Babel