Systematic Weight Evaluation for Pruning Large Language Models: Enhancing Performance and Sustainability Journal: Nuclear Physics B

Ashhadul Islam, Samir Brahim Belhaouari, Amine Bermak · 2025

Read Systematic Weight Evaluation for Pruning Large Language Models: Enhancing Performance and Sustainability Journal: Nuclear Physics B by Ashhadul Islam, Samir Brahim Belhaouari, Amine Bermak (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-17071.txt.

Description

Ashhadul Islam, Samir Brahim Belhaouari, and Amine Bermak wrote a 2025 paper on shrinking large language models in a smarter way. These models power well-known chat tools, but training and running them uses huge amounts of energy and water. Pruning, the removal of unneeded connections inside a model, can cut that cost. Done carelessly, it also wrecks the model's skill.

The authors track how each weight inside the model changes during training. Weights that barely matter over time become the first candidates for removal. Tests on a small language model and a large model that also handles images showed a clear pattern. Moderate pruning made the models leaner and even slightly better. Heavy pruning, by contrast, caused a sharp drop in quality. The authors argue that careful, measured pruning is a path to AI that is both strong and more sustainable.

Catalog record

CREATORAshhadul Islam, Samir Brahim Belhaouari, Amine Bermak
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-17071

Browse the full library · About the Tower of Babel