SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning
Jinpeng Chen, Runmin Cong, Yuzhi Zhao, Hongzheng Yang, Guangneng Hu, Horace Ho Shing Ip, Sam Kwong · 2025
Read SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning by Jinpeng Chen, Runmin Cong, Yuzhi Zhao, Hongzheng Yang, Guangneng Hu, Horace Ho Shing Ip, Sam Kwong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-02486.txt.
Description
Jinpeng Chen and six co-authors split forgetting in ever learning vision language models into two kinds in a 2025 paper. As such models take new tasks in turns, old skills fade. Some fading is skin deep: the facts stay, but the answer style drifts to the newest task's dress, which makes old answers unusable. The rest is true loss, where well dressed answers turn wrong.
Their fix works in order. First, training sets are restyled to one shared varied dress, which stops the style drift. Then a regularized low rank method, RegLoRA, locks the weights where old knowledge chiefly sits. The pair, named SEFE, topped past methods on the standard runs. The authors say style must be cleared before true loss can even be seen.
Catalog record
| CREATOR | Jinpeng Chen, Runmin Cong, Yuzhi Zhao, Hongzheng Yang, Guangneng Hu, Horace Ho Shing Ip, Sam Kwong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-02486