LV-XAttn : Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models

Tzu-Tao Chang, Shivaram Venkataraman · 2025

Read LV-XAttn : Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models by Tzu-Tao Chang, Shivaram Venkataraman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-02406.txt.

Description

Tzu-Tao Chang and Shivaram Venkataraman speed up multimodal language models that must look at very long visual inputs, such as videos. These models let text attend to picture tokens through cross-attention. With huge numbers of visual tokens, memory blows up and the work is split across graphics processors, where moving data between chips becomes the bottleneck.

Their method, LV-XAttn, exploits an imbalance: the text side of the attention is small, the visual side enormous. So the big visual blocks stay put on each processor while only the small query blocks travel. A recomputation trick stretches the visual context further. The authors prove the communication savings add up across model sizes. On three real multimodal models, the system ran up to 10.62 times faster end to end than existing distributed approaches.

Catalog record

CREATORTzu-Tao Chang, Shivaram Venkataraman
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-02406

Browse the full library · About the Tower of Babel