LV-XAttn : Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models
Tzu-Tao Chang, Shivaram Venkataraman · 2025
Read LV-XAttn : Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models by Tzu-Tao Chang, Shivaram Venkataraman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-02406.txt.
Description
Tzu-Tao Chang and Shivaram Venkataraman speed up multimodal language models that must look at very long visual inputs, such as videos. These models let text attend to picture tokens through cross-attention. With huge numbers of visual tokens, memory blows up and the work is split across graphics processors, where moving data between chips becomes the bottleneck.
Their method, LV-XAttn, exploits an imbalance: the text side of the attention is small, the visual side enormous. So the big visual blocks stay put on each processor while only the small query blocks travel. A recomputation trick stretches the visual context further. The authors prove the communication savings add up across model sizes. On three real multimodal models, the system ran up to 10.62 times faster end to end than existing distributed approaches.
Catalog record
| CREATOR | Tzu-Tao Chang, Shivaram Venkataraman |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-02406