LinguaMap: Which Layers of LLMs Speak Your Language and How to Tune Them?
J. Ben Tamo 1 , Daniel Carlander-Reuterfelt 2 , Jonathan Rubin 2 , Dezhi Hong 2 Mingxian Wang 2 , Oleg Poliannikov 2 1 Georgia Institute of Technology 2 Amazon · 2026
Read LinguaMap: Which Layers of LLMs Speak Your Language and How to Tune Them? by J. Ben Tamo 1 , Daniel Carlander-Reuterfelt 2 , Jonathan Rubin 2 , Dezhi Hong 2 Mingxian Wang 2 , Oleg Poliannikov 2 1 Georgia Institute of Technology 2 Amazon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20009.txt.
Description
J. Ben Tamo and colleagues ask which layers of a language model handle which language. Despite multilingual pretraining, models often stumble on non-English tasks. The authors separate two failure modes. In the transfer bottleneck, the model replies in the right language but gets the task wrong. In the consistency bottleneck, the answer is right but in the wrong language.
Their study, LinguaMap, characterises where each failure lives in the network. Knowing the responsible layers tells practitioners how to tune them. Targeted tuning could fix language control without retraining everything. Most users of these models do not work in English. Work on language control is therefore central, not peripheral, to the field.
Catalog record
| CREATOR | J. Ben Tamo 1 , Daniel Carlander-Reuterfelt 2 , Jonathan Rubin 2 , Dezhi Hong 2 Mingxian Wang 2 , Oleg Poliannikov 2 1 Georgia Institute of Technology 2 Amazon |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-20009