Beyond One-Size-Fits-All Pruning via Evolutionary Metric Search for Large Language Models

Shuqi Liu, Bowei He, Han Wu †, Linqi Song † · 2025

Read Beyond One-Size-Fits-All Pruning via Evolutionary Metric Search for Large Language Models by Shuqi Liu, Bowei He, Han Wu †, Linqi Song † (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10735.txt.

Description

Shuqi Liu and three co-authors make pruning of large language models adaptive instead of one-size-fits-all. Their framework appeared in twenty twenty-five. Pruning trims a trained model's weights for speed, but models differ so much in weight patterns that fixed recipes fail across them.

Their system, OptiShear, searches for the right scoring metric for each model. A fast error measure lets it test many candidates cheaply. A multi-goal genetic algorithm tunes both the metric and how much each layer is cut. On LLaMA and Mistral models from seven to seventy billion parameters, the adaptive metrics beat fixed ones across benchmarks. The learned layer patterns also boosted rival methods. The authors show the gains transfer across tasks and models.

Catalog record

CREATORShuqi Liu, Bowei He, Han Wu †, Linqi Song †
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10735

Browse the full library · About the Tower of Babel