SageAttention2++: A More Efficient Implementation of SageAttention2

Jintao Zhang, Xiaoming Xu, Jia Wei, Haofeng Huang, Pengle Zhang, Chendong Xiang, Jun Zhu, Jianfei Chen · 2025

Read SageAttention2++: A More Efficient Implementation of SageAttention2 by Jintao Zhang, Xiaoming Xu, Jia Wei, Haofeng Huang, Pengle Zhang, Chendong Xiang, Jun Zhu, Jianfei Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21136.txt.

Description

Jintao Zhang and co-authors speed up the attention step that dominates transformer model costs. Their 2025 paper improves SageAttention2, a method that uses low-precision math to accelerate attention. Attention time grows with the square of sequence length, so every gain matters for long texts and videos.

The new version, SageAttention2++, swaps in a faster chip instruction that multiplies in FP8 while adding up in FP16. That instruction runs twice as fast as the one used before. The result was a near fourfold speedup over FlashAttention with the same accuracy as SageAttention2. Gains carried across language, image, and video models with negligible loss in end metrics.

Catalog record

CREATORJintao Zhang, Xiaoming Xu, Jia Wei, Haofeng Huang, Pengle Zhang, Chendong Xiang, Jun Zhu, Jianfei Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE15 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-21136

Browse the full library · About the Tower of Babel