Rethinking Layer Redundancy in Large Language Models: Calibration Objectives and Search for Depth Pruning

Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim, YoungJin Heo, Suin Cho, Seong-hun Kim, Woosang Lim, Gaeul Kwon · 2026

Read Rethinking Layer Redundancy in Large Language Models: Calibration Objectives and Search for Depth Pruning by Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim, YoungJin Heo, Suin Cho, Seong-hun Kim, Woosang Lim, Gaeul Kwon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-24938.txt.

Description

Kim and five colleagues study depth pruning in large language models from a functional view. Depth pruning improves run speed by removing Transformer blocks. Evaluating typical model families across diverse calibration setups and multiple search methods, they show different setups produce different pruned sets. Under a fixed calibration setup, complex search methods yield only small gains over simple one-shot methods, landing on similar pruned sets. Overall, calibration setup matters far more than search method choice for pruning patterns and calibration scores, and mattering comparably for downstream reasoning score spread.

Catalog record

CREATORMinkyu Kim, Vincent-Daniel Yun, Youngrae Kim, YoungJin Heo, Suin Cho, Seong-hun Kim, Woosang Lim, Gaeul Kwon
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25672
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2604.24938

Browse the full library · About the Tower of Babel