Quantizing Small-Scale State-Space Models for Edge AI
Leo Zhao *, Tristan Torchet *, Melika Payvand, Laura Kriener §, Filippo Moro § · 2025
Read Quantizing Small-Scale State-Space Models for Edge AI by Leo Zhao *, Tristan Torchet *, Melika Payvand, Laura Kriener §, Filippo Moro § (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-12480.txt.
Description
Leo Zhao and colleagues shrank state-space sequence models for edge devices in a 2025 paper. State-space models handle long sequences efficiently, making them attractive for small hardware, but their weights still need quantization, storing numbers in fewer bits, to fit tight memory. Using the S4D design, the team found which parts suffer most under quantization: the state matrix and the internal state itself.
Simple post-training quantization crushed accuracy on a sequential digit task. Training with quantization in the loop restored nearly all of it at eight-bit precision, and even allowed going below eight bits with the right parameterization. Giving sensitive parts more bits and robust parts fewer cut total memory further. Edge AI gains a recipe for tiny sequence models.
Catalog record
| CREATOR | Leo Zhao *, Tristan Torchet *, Melika Payvand, Laura Kriener §, Filippo Moro § |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-12480