DisTaC : Conditioning Task Vectors via Distillation for Robust Model Merging
Kotaro Yoshida, Yuji Naraki, Takafumi Horie, Ryotaro Shimizu, Hiroki Naganuma · 2025
Read DisTaC : Conditioning Task Vectors via Distillation for Robust Model Merging by Kotaro Yoshida, Yuji Naraki, Takafumi Horie, Ryotaro Shimizu, Hiroki Naganuma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01148.txt.
Description
Kotaro Yoshida and four co-authors fortify, in 2025 work, the merging of trained models into one multi-task whole. Merges are cheap and flexy, yet past checks ran on sets that flatter the trick. On harder ground two flaws bite: task shift spans of unlike size, and source models that speak with low trust.
Their pre-step runs each source's lore through a teacher-student squeeze. The squeeze re-scales the shift span and lifts trust, while the task's core lore stays. Top merge tools then took in models with the bad traits — models they once failed on — and gained clear ground. Prep the parts, they show, and the merge holds.
Catalog record
| CREATOR | Kotaro Yoshida, Yuji Naraki, Takafumi Horie, Ryotaro Shimizu, Hiroki Naganuma |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-01148