SPAT: Sensitivity-based Multihead-attention Pruning on Time Series Forecasting Models
Suhan Guo, Jiahong Deng, Mengjun YiSchool of Artificial Intelligence, Nanjing University, ChinaNational Key Laboratory for Novel Software Technology, Nanjing University, China {shguo, jiahongdeng, mengjunyi}@smail.nju.edu.cn Furao shenSchool of Artificial Intelligence, Nanjing University, ChinaNatio · 2025
Read SPAT: Sensitivity-based Multihead-attention Pruning on Time Series Forecasting Models by Suhan Guo, Jiahong Deng, Mengjun YiSchool of Artificial Intelligence, Nanjing University, ChinaNational Key Laboratory for Novel Software Technology, Nanjing University, China {shguo, jiahongdeng, mengjunyi}@smail.nju.edu.cn Furao shenSchool of Artificial Intelligence, Nanjing University, ChinaNatio (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08768.txt.
Description
Suhan Guo and co-authors prune attention in forecasting models in 2025. Attention models predict time series well but cost much to run. Most trimming methods cut small parts inside a unit and still risk fitting noise in the data.
Their method, SPAT, removes whole attention units judged least useful by a new score. That score ranks each unit during a pre-training phase by how much its output varies. Pruned models cut a key error measure and slashed computing cost by about a third. They also beat lean rivals, including newer model types, on standard and zero-shot tests, the authors report.
Catalog record
| CREATOR | Suhan Guo, Jiahong Deng, Mengjun YiSchool of Artificial Intelligence, Nanjing University, ChinaNational Key Laboratory for Novel Software Technology, Nanjing University, China {shguo, jiahongdeng, mengjunyi}@smail.nju.edu.cn Furao shenSchool of Artificial Intelligence, Nanjing University, ChinaNatio |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-08768