TraceNAS: Zero-shot LLM Pruning via Gradient Trace Correlation
Prajna G. Malettira, Manish Nagaraj, Arjun Roy, Shubham Negi, Kaushik Roy · 2026
Read TraceNAS: Zero-shot LLM Pruning via Gradient Trace Correlation by Prajna G. Malettira, Manish Nagaraj, Arjun Roy, Shubham Negi, Kaushik Roy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02891.txt.
Description
Pruning large language models without any training is the goal of Prajna G. Malettira and 4 co-authors in a 2026 paper. Pruning removes layers, heads, or channels to make models cheaper to run. Judging each part alone misses how parts interact across the whole model, while training-aware search is expensive.
Their framework, TraceNAS, searches depth and width together. A zero-shot score checks how closely a pruned model's loss landscape matches the original. Discovery took hours on a single graphics processor, far less than training-aware methods. Tests on the Llama and Qwen model families followed. TraceNAS stayed competitive with training-aware baselines on commonsense and reasoning benchmarks.
Catalog record
| CREATOR | Prajna G. Malettira, Manish Nagaraj, Arjun Roy, Shubham Negi, Kaushik Roy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-02891