KD4MT: A Survey of Knowledge Distillation for Machine Translation

Ona de Gibert 1, Joseph Attieh ∗1, Timothee Mickus 1, Yves Scherrer 2, Jörg Tiedemann 1 · 2026

Read KD4MT: A Survey of Knowledge Distillation for Machine Translation by Ona de Gibert 1, Joseph Attieh ∗1, Timothee Mickus 1, Yves Scherrer 2, Jörg Tiedemann 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-15845.txt.

Description

Gibert and colleagues survey knowledge distillation for machine translation, or KD4MT, across 105 papers through October 1, 2025. Knowledge distillation has gained traction as a compression tool for ever-larger models in NLP. In the field, however, it also functions as a general-purpose knowledge transfer mechanism shaping supervision and translation quality as well as efficiency.

Advances are then grouped by method gains and by useful uses. Noted and counted reviews identify common trends, key research gaps, and the absence of unified testing practice for methods in the field. Practical tips for selecting a method in real cases are provided. Potential risks for machine translation include increased hallucination and bias growth. Finally, the role of large language models in reshaping the KD4MT field is discussed. An open base sums main traits of surveyed methods, and a list of key terms aids further research.

Catalog record

CREATOROna de Gibert 1, Joseph Attieh ∗1, Timothee Mickus 1, Yves Scherrer 2, Jörg Tiedemann 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-15845

Browse the full library · About the Tower of Babel