LinguDistill : Recovering Linguistic Ability in Vision-Language Models via Selective Cross-Modal Distillation

Patrick Amadeus Irawan, Erland Hilman Fuadi, Shanu Kumar · 2026

Read LinguDistill : Recovering Linguistic Ability in Vision-Language Models via Selective Cross-Modal Distillation by Patrick Amadeus Irawan, Erland Hilman Fuadi, Shanu Kumar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-00829.txt.

Description

Irawan and colleagues propose LinguDistill, an adapter-free distillation method. It recovers language ability lost when a pretrained language model is turned into a vision-language model. A frozen original model teaches during multimodal training through layer-wise KV-cache sharing, with distillation applied only on language-heavy data. The method matches the original vision-language model on text-only benchmarks on average, beats it on ScienceQA, and adds no parameters or inference cost.

Catalog record

CREATORPatrick Amadeus Irawan, Erland Hilman Fuadi, Shanu Kumar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46906
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.00829

Browse the full library · About the Tower of Babel