FlashHead: Efficient Drop-In Replacement for the Classification Head in Language Model Inference
Wilhelm Tranheden Shahnawaz Ahmed Devdatt Dubhashi Jonna Matthiesen Hannes von Essen Embedl Gamla Almedalsvägen 21, 412 63 Gothenburg, Sweden https://www.embedl.com · 2026
Read FlashHead: Efficient Drop-In Replacement for the Classification Head in Language Model Inference by Wilhelm Tranheden Shahnawaz Ahmed Devdatt Dubhashi Jonna Matthiesen Hannes von Essen Embedl Gamla Almedalsvägen 21, 412 63 Gothenburg, Sweden https://www.embedl.com (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14591.txt.
Description
Tranheden and co-authors replace the output head of language models with a lean retrieval design. Growing vocabularies make the dense head a heavy cost. It can hold up to 60% of model parameters and half of inference compute. FlashHead needs no training and treats head scoring as retrieval. Balanced clusters pack the vocabulary into hardware-friendly tensors. Multiprobe scoring checks thousands of clusters at once. A sampling step and selective quantization complete the design, with speedups shown on Llama, Gemma and Qwen models.
Catalog record
| CREATOR | Wilhelm Tranheden Shahnawaz Ahmed Devdatt Dubhashi Jonna Matthiesen Hannes von Essen Embedl Gamla Almedalsvägen 21, 412 63 Gothenburg, Sweden https://www.embedl.com |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54059 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.14591