TransMLA: MLA Is All You Need

Fanxu Meng, Pingzhi Tang, Xiaojuan Tang, Zengwei Yao, Xing Sun, Muhan Zhang · 2025

Read TransMLA: MLA Is All You Need by Fanxu Meng, Pingzhi Tang, Xiaojuan Tang, Zengwei Yao, Xing Sun, Muhan Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-07864.txt.

Description

Fanxu Meng and five co-authors present TransMLA, a frame that turns models built with one attention style into another called multi-head latent attention. The target style is used by DeepSeek and unlocks its speed tools. Their work came out in twenty twenty-five.

The shift squeezes the stored key and value cache by a vast share. On a known seven billion weight model, that gave a more than ten fold speed up at long text lengths while output stayed sound. Only a modest pool of training text was needed to bring scores back to par across test sets. The authors pitch the frame as a hands on path for moving older models to the new style.

Catalog record

CREATORFanxu Meng, Pingzhi Tang, Xiaojuan Tang, Zengwei Yao, Xing Sun, Muhan Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-07864

Browse the full library · About the Tower of Babel