Fused3S: Fast Sparse Attention on Tensor Cores Conference: 2025 International Conference on Supercomputing; June 8–11, 2025; Salt Lake City, UT, USA 2025 International Conference on Supercomputing (ICS ’25), June 8–11, 2025, Salt Lake City, UT, USA DOI: 10.1145/3721145.3730430 ISBN: 979-8-4007-1537-
Zitong Li, Aparna Chandramowlishwaran · 2025
Read Fused3S: Fast Sparse Attention on Tensor Cores Conference: 2025 International Conference on Supercomputing; June 8–11, 2025; Salt Lake City, UT, USA 2025 International Conference on Supercomputing (ICS ’25), June 8–11, 2025, Salt Lake City, UT, USA DOI: 10.1145/3721145.3730430 ISBN: 979-8-4007-1537- by Zitong Li, Aparna Chandramowlishwaran (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08098.txt.
Description
Zitong Li and Aparna Chandramowlishwaran introduce, in a 2025 paper, a faster way to run sparse attention on modern graphics processors. Sparse attention is a core step in many leading neural models, from graph learning to sparse sequence work. It breaks into three sparse matrix operations in a row, with a softmax step in the middle.
Running that pattern well is hard for two reasons. The scattered sparsity clashes with tensor cores, the chip parts built for dense math. And moving data around costs much time. Past work optimized the steps one by one, or tackled only one of the two problems. Their method, Fused3S, is the first to fuse all three steps into one pass that uses tensor cores well and moves little data. On real graph data sets, it ran many times faster than the best prior methods.
Catalog record
| CREATOR | Zitong Li, Aparna Chandramowlishwaran |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-08098