Rethinking Layer Redundancy in Large Language Models: Calibration Objectives and Search for Depth Pruning
Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim, YoungJin Heo, Suin Cho, Seong-hun Kim, Woosang Lim, Gaeul Kwon · 2026
Read Rethinking Layer Redundancy in Large Language Models: Calibration Objectives and Search for Depth Pruning by Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim, YoungJin Heo, Suin Cho, Seong-hun Kim, Woosang Lim, Gaeul Kwon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-24938.txt.
Description
Kim and five colleagues study depth pruning in large language models from a functional view. Depth pruning improves run speed by removing Transformer blocks. Evaluating typical model families across diverse calibration setups and multiple search methods, they show different setups produce different pruned sets. Under a fixed calibration setup, complex search methods yield only small gains over simple one-shot methods, landing on similar pruned sets. Overall, calibration setup matters far more than search method choice for pruning patterns and calibration scores, and mattering comparably for downstream reasoning score spread.
Catalog record
| CREATOR | Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim, YoungJin Heo, Suin Cho, Seong-hun Kim, Woosang Lim, Gaeul Kwon |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25672 |
| LICENSE | http://creativecommons.org/publicdomain/zero/1.0/ |
| RIGHTS | CC0 |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.24938