Beyond Student: An Asymmetric Network for Neural Network Inheritance
Yiyun Zhou Jingwei Shi Mingjing Xu Zhonghua Jiang Jingyuan Chen · 2026
Read Beyond Student: An Asymmetric Network for Neural Network Inheritance by Yiyun Zhou Jingwei Shi Mingjing Xu Zhonghua Jiang Jingyuan Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-09509.txt.
Description
Knowledge distillation compresses a large teacher network into a smaller student, but the capacity gap caps what the student can absorb. Yiyun Zhou and four co-authors ask a sharper question in their 2026 paper. Can a network inherit not just the teacher's structure but as much of its knowledge as possible, and how would it compare with ordinary students from the same teacher? Their answer is InherNet, a network built by inheritance rather than imitation.
InherNet applies asymmetric low-rank decomposition to the teacher's weights, rebuilding a lightweight yet expressive network without major architectural disruption. Singular value decomposition initializes the process, preserving principal knowledge while balancing depth, width, and compression. Across unimodal and multimodal tasks, InherNet beats student networks of similar size. The work points model compression beyond classic distillation toward inheritance.
Catalog record
| CREATOR | Yiyun Zhou Jingwei Shi Mingjing Xu Zhonghua Jiang Jingyuan Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 105 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-09509