SPAT: Sensitivity-based Multihead-attention Pruning on Time Series Forecasting Models

Suhan Guo, Jiahong Deng, Mengjun YiSchool of Artificial Intelligence, Nanjing University, ChinaNational Key Laboratory for Novel Software Technology, Nanjing University, China {shguo, jiahongdeng, mengjunyi}@smail.nju.edu.cn Furao shenSchool of Artificial Intelligence, Nanjing University, ChinaNatio · 2025

Read SPAT: Sensitivity-based Multihead-attention Pruning on Time Series Forecasting Models by Suhan Guo, Jiahong Deng, Mengjun YiSchool of Artificial Intelligence, Nanjing University, ChinaNational Key Laboratory for Novel Software Technology, Nanjing University, China {shguo, jiahongdeng, mengjunyi}@smail.nju.edu.cn Furao shenSchool of Artificial Intelligence, Nanjing University, ChinaNatio (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08768.txt.

Description

Suhan Guo and co-authors prune attention in forecasting models in 2025. Attention models predict time series well but cost much to run. Most trimming methods cut small parts inside a unit and still risk fitting noise in the data.

Their method, SPAT, removes whole attention units judged least useful by a new score. That score ranks each unit during a pre-training phase by how much its output varies. Pruned models cut a key error measure and slashed computing cost by about a third. They also beat lean rivals, including newer model types, on standard and zero-shot tests, the authors report.

Catalog record

CREATORSuhan Guo, Jiahong Deng, Mengjun YiSchool of Artificial Intelligence, Nanjing University, ChinaNational Key Laboratory for Novel Software Technology, Nanjing University, China {shguo, jiahongdeng, mengjunyi}@smail.nju.edu.cn Furao shenSchool of Artificial Intelligence, Nanjing University, ChinaNatio
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-08768

Browse the full library · About the Tower of Babel