Long Exposure : Accelerating Parameter-Efficient Fine-Tuning for LLMs under Shadowy Sparsity
Tuowei Wang 1 2 3 4 , Kun Li 1 4 5 , Zixu Hao 1 2 , Donglin Bai 1 , Ju Ren 2 5 , Yaoxue Zhang 2 , Ting Cao 1 , Mao Yang 1 · 2025
Read Long Exposure : Accelerating Parameter-Efficient Fine-Tuning for LLMs under Shadowy Sparsity by Tuowei Wang 1 2 3 4 , Kun Li 1 4 5 , Zixu Hao 1 2 , Donglin Bai 1 , Ju Ren 2 5 , Yaoxue Zhang 2 , Ting Cao 1 , Mao Yang 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-15964.txt.
Description
Tuowei Wang leads seven co-authors in speeding up parameter-efficient fine-tuning of large language models. Such tuning adapts a pre-trained model by changing few added parameters. It still costs much time and money in practice. The team names a neglected pattern in that process. They call it Shadowy Sparsity, a subtle form of sparsity that is specific to fine-tuning.
Their system, Long Exposure, attacks the pattern in three parts. An exposer senses over a longer range to catch more hidden sparsity. A sequence-oriented predictor handles long inputs and shifting parameters. A dynamic-aware operator structures the sparse math for cleaner memory access. Across tests, the system beats leading prior methods. End-to-end fine-tuning speedups reach about two and a half times at best. The work targets cheaper adaptation of large models to new tasks.
Catalog record
| CREATOR | Tuowei Wang 1 2 3 4 , Kun Li 1 4 5 , Zixu Hao 1 2 , Donglin Bai 1 , Ju Ren 2 5 , Yaoxue Zhang 2 , Ting Cao 1 , Mao Yang 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-15964