SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation

Yixian Shen, Qi Bi, Jia-Hong Huang, Hongyi Zhu, Andy D. Pimentel, Anuj Pathania · 2025

Read SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation by Yixian Shen, Qi Bi, Jia-Hong Huang, Hongyi Zhu, Andy D. Pimentel, Anuj Pathania (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05539.txt.

Description

Yixian Shen and co-authors introduced SSH in 2025, a leaner way to fine-tune large foundation models. Low-rank adaptation already cuts the number of trainable weights, but cost and memory still bite as models grow. SSH works in the frequency domain instead. A discrete Hartley transform turns the weights into a spectrum, and only the most informative parts are trained.

Guided by the starting weights, the method picks spectral components across all layers. A light inverse transform then maps updates back for use. Tests covered language understanding and generation, plus video-text tasks. SSH beat rival parameter-efficient methods while cutting compute and memory needs sharply. The authors pitch it as a practical route for adapting very large models on modest hardware.

Catalog record

CREATORYixian Shen, Qi Bi, Jia-Hong Huang, Hongyi Zhu, Andy D. Pimentel, Anuj Pathania
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-05539

Browse the full library · About the Tower of Babel