TraceNAS: Zero-shot LLM Pruning via Gradient Trace Correlation

Prajna G. Malettira, Manish Nagaraj, Arjun Roy, Shubham Negi, Kaushik Roy · 2026

Read TraceNAS: Zero-shot LLM Pruning via Gradient Trace Correlation by Prajna G. Malettira, Manish Nagaraj, Arjun Roy, Shubham Negi, Kaushik Roy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02891.txt.

Description

Pruning large language models without any training is the goal of Prajna G. Malettira and 4 co-authors in a 2026 paper. Pruning removes layers, heads, or channels to make models cheaper to run. Judging each part alone misses how parts interact across the whole model, while training-aware search is expensive.

Their framework, TraceNAS, searches depth and width together. A zero-shot score checks how closely a pruned model's loss landscape matches the original. Discovery took hours on a single graphics processor, far less than training-aware methods. Tests on the Llama and Qwen model families followed. TraceNAS stayed competitive with training-aware baselines on commonsense and reasoning benchmarks.

Catalog record

CREATORPrajna G. Malettira, Manish Nagaraj, Arjun Roy, Shubham Negi, Kaushik Roy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02891

Browse the full library · About the Tower of Babel