Rethinking Knowledge Distillation in Collaborative Machine Learning: Memory, Knowledge, and Their Interactions
Pengchao Han, Xi Huang, Yi Fang, Guojun Han · 2025
Read Rethinking Knowledge Distillation in Collaborative Machine Learning: Memory, Knowledge, and Their Interactions by Pengchao Han, Xi Huang, Yi Fang, Guojun Han (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-19972.txt.
Description
Pengchao Han, Xi Huang, Yi Fang, and Guojun Han review how knowledge distillation behaves when the students teach each other. Collaborative learning lets distributed agents train together without surrendering private data, and distillation, transferring knowledge between models, is its central courier. What memory and knowledge each agent holds, and how they interact during transfer, had not been systematically mapped.
The review defines and categorizes memory and knowledge inside the distillation process, then examines distributed, hierarchical, and decentralized collaboration patterns. Task heterogeneity gets extended treatment, spanning federated learning, multi-agent domain adaptation, multimodal and continual federated variants, multi-task learning, and graph knowledge embedding, alongside model, data, and resource heterogeneity and privacy concerns. Existing work is sorted by how it handles memory and knowledge, and open challenges close the survey.
Catalog record
| CREATOR | Pengchao Han, Xi Huang, Yi Fang, Guojun Han |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 140 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-19972