FlashHead: Efficient Drop-In Replacement for the Classification Head in Language Model Inference

Wilhelm Tranheden Shahnawaz Ahmed Devdatt Dubhashi Jonna Matthiesen Hannes von Essen Embedl Gamla Almedalsvägen 21, 412 63 Gothenburg, Sweden https://www.embedl.com · 2026

Read FlashHead: Efficient Drop-In Replacement for the Classification Head in Language Model Inference by Wilhelm Tranheden Shahnawaz Ahmed Devdatt Dubhashi Jonna Matthiesen Hannes von Essen Embedl Gamla Almedalsvägen 21, 412 63 Gothenburg, Sweden https://www.embedl.com (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14591.txt.

Description

Tranheden and co-authors replace the output head of language models with a lean retrieval design. Growing vocabularies make the dense head a heavy cost. It can hold up to 60% of model parameters and half of inference compute. FlashHead needs no training and treats head scoring as retrieval. Balanced clusters pack the vocabulary into hardware-friendly tensors. Multiprobe scoring checks thousands of clusters at once. A sampling step and selective quantization complete the design, with speedups shown on Llama, Gemma and Qwen models.

Catalog record

CREATORWilhelm Tranheden Shahnawaz Ahmed Devdatt Dubhashi Jonna Matthiesen Hannes von Essen Embedl Gamla Almedalsvägen 21, 412 63 Gothenburg, Sweden https://www.embedl.com
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54059
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.14591

Browse the full library · About the Tower of Babel