TransMLA: MLA Is All You Need
Fanxu Meng, Pingzhi Tang, Xiaojuan Tang, Zengwei Yao, Xing Sun, Muhan Zhang · 2025
Read TransMLA: MLA Is All You Need by Fanxu Meng, Pingzhi Tang, Xiaojuan Tang, Zengwei Yao, Xing Sun, Muhan Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-07864.txt.
Description
Fanxu Meng and five co-authors present TransMLA, a frame that turns models built with one attention style into another called multi-head latent attention. The target style is used by DeepSeek and unlocks its speed tools. Their work came out in twenty twenty-five.
The shift squeezes the stored key and value cache by a vast share. On a known seven billion weight model, that gave a more than ten fold speed up at long text lengths while output stayed sound. Only a modest pool of training text was needed to bring scores back to par across test sets. The authors pitch the frame as a hands on path for moving older models to the new style.
Catalog record
| CREATOR | Fanxu Meng, Pingzhi Tang, Xiaojuan Tang, Zengwei Yao, Xing Sun, Muhan Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-07864