Dual-level knowledge distillation from vision transformer to Swin transformer for fine-grained brain tumor classification: A 44-class MRI benchmark
Eram Mahamud, Md Assaduzzaman, Nafiz Fahad, Tze Hui Liew, Ohidujjaman · 2026
Read Dual-level knowledge distillation from vision transformer to Swin transformer for fine-grained brain tumor classification: A 44-class MRI benchmark by Eram Mahamud, Md Assaduzzaman, Nafiz Fahad, Tze Hui Liew, Ohidujjaman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/PMC13572084.txt.
Description
Mahamud and co-authors shrink a large vision model into a smaller one that classifies brain tumours better. Their 2026 paper proposes dual-level knowledge distillation, transferring both the soft answer patterns and the internal feature representations of a frozen vision transformer teacher into a compact Swin transformer student. The benchmark covers 44 classes of MRI scans, combining fifteen tumour types across three imaging modes, with 4,479 images in total.
Averaged over five runs, the student reached about 98 percent test accuracy, beating its own teacher, which scored about 95 percent, and a student trained without distillation. Logit-only and feature-only distillation each helped, but together they added the largest gain, showing the two signals complement each other. Four standard convolutional networks, from about 92 to 94 percent accuracy, all trailed the distilled student under identical training conditions.
Catalog record
| CREATOR | Eram Mahamud, Md Assaduzzaman, Nafiz Fahad, Tze Hui Liew, Ohidujjaman |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 87 KB |
| LICENSE | https://creativecommons.org/licenses/by/4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |