DiaBlo: Diagonal Blocks Are Sufficient For Finetuning

Selcuk Gurses Aozhong Zhang Yanxia Deng Xun Dong Xin Li Naigang Wang, Penghang Yin Zi Yang · 2025

Read DiaBlo: Diagonal Blocks Are Sufficient For Finetuning by Selcuk Gurses Aozhong Zhang Yanxia Deng Xun Dong Xin Li Naigang Wang, Penghang Yin Zi Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-03230.txt.

Description

Tuning a large language model for a new task can cost a great deal of time and memory. A 2025 paper by Selcuk Gurses and co-authors offers a leaner way, called DiaBlo. Rather than retrain the whole model, the method updates only the square blocks that sit on the diagonal of chosen weight tables. It skips the extra low-rank steps that a well-known method, LoRA, needs to start and train well.

Tests cover common sense tasks, math tasks, code writing, and safety tuning. The team reports that this small change trains in a steady way and matches the speed and memory use of LoRA. They also give proofs that the method can express more than LoRA in a simple case and can settle at a sound end point.

Catalog record

CREATORSelcuk Gurses Aozhong Zhang Yanxia Deng Xun Dong Xin Li Naigang Wang, Penghang Yin Zi Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-03230

Browse the full library · About the Tower of Babel