SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation
Yixian Shen, Qi Bi, Jia-Hong Huang, Hongyi Zhu, Andy D. Pimentel, Anuj Pathania · 2025
Read SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation by Yixian Shen, Qi Bi, Jia-Hong Huang, Hongyi Zhu, Andy D. Pimentel, Anuj Pathania (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05539.txt.
Description
Yixian Shen and co-authors introduced SSH in 2025, a leaner way to fine-tune large foundation models. Low-rank adaptation already cuts the number of trainable weights, but cost and memory still bite as models grow. SSH works in the frequency domain instead. A discrete Hartley transform turns the weights into a spectrum, and only the most informative parts are trained.
Guided by the starting weights, the method picks spectral components across all layers. A light inverse transform then maps updates back for use. Tests covered language understanding and generation, plus video-text tasks. SSH beat rival parameter-efficient methods while cutting compute and memory needs sharply. The authors pitch it as a practical route for adapting very large models on modest hardware.
Catalog record
| CREATOR | Yixian Shen, Qi Bi, Jia-Hong Huang, Hongyi Zhu, Andy D. Pimentel, Anuj Pathania |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-05539