Online Pseudo-average Shifting Attention(PASA) for Robust Low-precision LLM Inference: Algorithms and Numerical Analysis
Long Cheng*, Qichen Liao, Fan Wu, Junlin Mu, Tengfei Han, Zhe Qiu, Lianqiang Li, Tianyi Liu, Fangzheng Miao, Keming Gao, Liang Wang, Zhen Zhang, Qiande Yin · 2025
Read Online Pseudo-average Shifting Attention(PASA) for Robust Low-precision LLM Inference: Algorithms and Numerical Analysis by Long Cheng*, Qichen Liao, Fan Wu, Junlin Mu, Tengfei Han, Zhe Qiu, Lianqiang Li, Tianyi Liu, Fangzheng Miao, Keming Gao, Liang Wang, Zhen Zhang, Qiande Yin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01873.txt.
Description
Thirteen authors led by Long Cheng speed the focus step of large AI models with a low-precision trick that keeps the maths whole. The focus step, called attention, eats time on long text and film jobs. On lean chips, half-precision sums can spill past their cap and wreck the run. Their method, PASA, shifts values by a live pseudo-mean as it goes, then heals the shift at the end.
Big bias and swing in focus data are named as the spill drivers in two model kinds. Tests ran on both random rigs and real large models to vet the fix. Data moves drop and chip work rises on cramped AI gear with the trick in place. Long jobs thus run on small chips with less spill fear.
Catalog record
| CREATOR | Long Cheng*, Qichen Liao, Fan Wu, Junlin Mu, Tengfei Han, Zhe Qiu, Lianqiang Li, Tianyi Liu, Fangzheng Miao, Keming Gao, Liang Wang, Zhen Zhang, Qiande Yin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01873