Efficient and Effective Prompt Tuning via Prompt Decomposition and Compressed Outer Product
Pengxiang Lan, Haoyu Xu, Enneng Yang, Yuliang Liang, Guibing Guo, Jianzhe Zhao, Xingwei Wang · 2025
Read Efficient and Effective Prompt Tuning via Prompt Decomposition and Compressed Outer Product by Pengxiang Lan, Haoyu Xu, Enneng Yang, Yuliang Liang, Guibing Guo, Jianzhe Zhao, Xingwei Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12200.txt.
Description
Lan and co-authors present a 2025 paper on LAMP, a leaner way to prompt-tune large language models. Prompt tuning trains only a few soft tokens, yet today's versions ignore how those tokens relate. They also grow long for hard tasks, eating memory. LAMP first shrinks the prompt space with a truncated singular value split.
A compressed outer product then lets the tokens interact many times, surfacing their hidden links. Average pooling finishes the design. The result trains far fewer parameters while keeping task quality. Tests target the twin goals in the title: efficiency and effectiveness together. The authors offer LAMP for teams tuning big models on tight budgets.
Catalog record
| CREATOR | Pengxiang Lan, Haoyu Xu, Enneng Yang, Yuliang Liang, Guibing Guo, Jianzhe Zhao, Xingwei Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-12200