Life Cycle-Aware Evaluation of Knowledge Distillation for Machine Translation: Environmental Impact and Translation Quality Trade-offs
Joseph Attieh, Timothee Mickus, Anne-Laure Ligozat, Aurélie Névéol, Jörg Tiedemann · 2026
Read Life Cycle-Aware Evaluation of Knowledge Distillation for Machine Translation: Environmental Impact and Translation Quality Trade-offs by Joseph Attieh, Timothee Mickus, Anne-Laure Ligozat, Aurélie Névéol, Jörg Tiedemann (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-09691.txt.
Introduction
Knowledge boiling shrinks a larger teacher system into a smaller student, and in machine rendering it is often judged by render quality and run speed. Joseph Attieh and four co-authors argue that this counting ignores the green costs of making and serving the boiled system. Their 2026 case study tests typical boiling methods on both bespoke render models and large language models. Quality is weighed against compute cost with a life-cycle check spanning the full model life cycle.
The key finding is that the serving volume needed to repay boiling is serving-tied and can shift by several orders of scale under batching. Boiling is not alone greener: whether it pays off green-wise depends on how much the boiled system is truly served. Usable guidance follows for picking, building, and testing boiling methods under quality and compute-induced limits.
Catalog record
| CREATOR | Joseph Attieh, Timothee Mickus, Anne-Laure Ligozat, Aurélie Névéol, Jörg Tiedemann |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-09691
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Life Cycle-Aware Evaluation of Knowledge Distillation for Machine Translation: Environmental Impact and Translation Quality Trade-offs Joseph Attieh, Timothee Mickus, Anne-Laure Ligozat, Aurélie Névéol, Jörg Tiedemann Abstract Knowledge distillation (KD) is a tool to compress a larger system (teacher) into a smaller one (student). In machine translation, studies typically report only the translation quality of the student and omit the computational complexity of performing KD, making it difficult to select among the many available KD choices under compute-induced constraints. In this study, we evaluate representative KD methods by considering both translation quality and computational cost. We express computational cost as a carbon footprint using the machine learning life cycle assessment (MLCA) tool. This assessment accounts for runtime operational emissions and amortized hardware production costs throughout the KD model life cycle (teacher training, distillation, and inference). We find that (i) distillation overhead dominates the total footprint at small deployment volumes, (ii) inference dominates at scale, making KD beneficial only beyond a task-dependent usage threshold, and (iii) word-level distillation typically offers more favorable footprint–quality trade-offs than sequence-level distillation. Our protocol provides reproducible guidance for selecting KD methods under explicit quality and compute-induced constraints. 1 Introduction Knowledge distillation (KD) is a standard family of algorithms that transfers the knowledge of larger teacher models to smaller student models ( Hinton et al. 2015 ) . Training a smaller model results in a lighter model that can run in resource-constrained environments, requiring less energy at inference time ( Luccioni et al. 2024 ) . However, KD introduces two fundamental trade-offs: First, reducing model size can degrade translation quality relative to the teacher ( Gumma et al. 2023 ) . Second, KD incurs additional computational overhead beyond standard training ( Zhang et al. 2018 ) : sequence-level KD requires inference from the teacher over the full dataset, whereas logit-level approaches requires repeated teacher forward passes. Despite these costs, most prior work reports only improvements in task performance relative to baselines or the quality gap between teacher and student, while largely ignoring environmental costs incurred during the distillation process. As a result, existing evaluations provide an incomplete picture of the process, making it difficult to decide whether to apply KD, or which KD method to use, in settings where computational resources and energy consumption are important. In this work, we focus on machine translation (MT), an end-user application in which responsive edge-device applications are desirable. MT systems are commonly used in high-throughput settings, such as translation platforms, document processing pipelines, and customer-support pipelines, where costs scale with the number of translated tokens. Importantly, this computational overhead is tied to KD broadly ( Mansourian et al. 2025 ) , as these overheads incur in any sequence-generation task that relies on beam search or repeated teacher inference ( Chan et al. 2025 ) . MT is specifically suitable and relevant for this study, thus why we focus on this application. In this paper, we argue that model performance has to be assessed through the lens of environmental impact, and not just generation quality. We adopt the machine learning life cycle assessment (MLCA) framework ( Morand et al. 2024 ) to evaluate the environmental impact of performing KD and to compare the impact of using different KD methods. We quantify the impact in terms of carbon footprint. Our analysis decomposes the assessment from a software point of view (teacher training, distillation and inference) and from a hardware point of view (hardware manufacturing and usage). We show that whether KD is worth applying in MT cannot be answered from translation quality metrics alone: it depends on how much translation the system will serve, which KD variant is used, and what translation quality target is required. Through experiments on English-to-Icelandic translation, we provide the following contributions: 1. We benchmark representative KD methods for MT by evaluating both translation quality using the COMET metric and carbon impact measured in CO 2 g equivalent. This dual focus allows us to give a more complete assessment of KD methods, emphasizing environmental implications alongside performance metrics. 2. We instantiate an MLCA-style accounting that decomposes emissions into teacher training, distillation, and inference, and includes both operational and amortized embodied emissions under transparent assumptions. 3. We model inference-phase emissions and identify break-even regimes in which distillation overhead is offset by per-request savings. 4. We evaluate two compression setups (Transformer-Base and Transformer-Tiny students distilled from a Transformer-Big teacher) to analyze how KD trade-offs change with the teacher–student capacity gap. We leverage the test case to provide guidelines for a detailed life cycle-aware assessment of KD in MT. 2 Preliminary Technical Background 2.1 Knowledge Distillation for Machine Translation Notations. We consider autoregressive MT: given a source sentence \mathbf{x} , a model produces a target sequence \mathbf{y}=(y_{1},\dots,y_{T}) by predicting each token y_{i} conditioned on the source x and the prefix \mathbf{y}_{<i} . Let D=\{(\mathbf{x}^{(i)},{\mathbf{y}}^{(i)})\}_{i=1}^{N} be a parallel corpus, where x^{i} is a sentence in the source language and y^{i} is the translation in the target language. Knowledge distillation (KD) transfers knowledge from a larger teacher to a smaller student ( Buciluǎ et al. 2006 ; Hinton et al. 2015 ) ; in MT, the two standard KD families are word-level (distribution matching) and sequence-level (synthetic targets) ( Kim & Rush 2016 ) . We denote by \theta_{\mathfrak{T}} and \theta_{\mathfrak{S}} the parameter vector of teacher and student. For a model of parameters \theta , the model’s next-token distribution at time step t for a specific example (\mathbf{x},\mathbf{y}) can be expressed as: p_{\theta}\;:=\;p(y_{t}\mid\mathbf{x},\mathbf{y}_{<t};\theta) . Sequence-level KD (Seq-KD). Seq-KD ( Kim & Rush 2016 ) trains the student on teacher-generated translations. For each source sentence \mathbf{x} , we decode a single high-probability teacher output \hat{\mathbf{y}} and form a synthetic corpus \hat{D}=\{(\mathbf{x}^{(i)},\hat{\mathbf{y}}^{(i)})\}_{i=1}^{N} . The synthetic target approximates the teacher argmax, \hat{\mathbf{y}}\approx\arg\max_{\mathbf{y}}p_{\theta_{\mathfrak{T}}}(\mathbf{y}\mid\mathbf{x}) , typically computed with beam search. At each time step t , beam search maintains a set of B partial hypotheses, expands them using the teacher next-token distribution, and keeps the top- B candidates by accumulated log-probability (optionally with a length penalty); the final output \hat{\mathbf{y}} is the highest-scoring completed hypothesis. Since decoding cost scales with B , this stage can dominate the distillation compute. Training is done through standard cross-entropy with teacher forcing: \mathcal{L}_{\mathrm{CE}}(\theta_{\mathfrak{S}};\hat{D})\;=\;\sum_{(\mathbf{x},\hat{\mathbf{y}})\in\hat{D}}\sum_{t=1}^{|\hat{\mathbf{y}}|}\mathcal{H}\!(\delta_{\hat{y}_{t}},p_{\theta_{\mathfrak{S}}}) Seq-KD is therefore an offline hard-distillation regime: the teacher supplies one-hot synthetic targets, and additional compute is primarily incurred during teacher-side decoding. Word-level KD (Word-KD). Word-KD ( Kim & Rush 2016 ) trains the student to match the teacher’s token-level distribution at each time step. The dataset-level KD objective can be described as: \mathcal{L}_{\mathrm{KD}}(\theta_{\mathfrak{S}};\theta_{\mathfrak{T}})\;=\;\sum_{(\mathbf{x},\mathbf{y})\in D}\sum_{t=1}^{|\mathbf{y}|}D_{\mathrm{KL}}(p_{\theta_{\mathfrak{T}}}\;\|\;p_{\theta_{\mathfrak{S}}}) In practice, student training combines teacher supervision with standard likelihood training on ground-truth targets using an interpolation hyperparameter \alpha\in[0,1] : \mathcal{L}_{\textsc{Word-KD}}(\theta_{\mathfrak{S}};\theta_{\mathfrak{T}})\;=\;(1-\alpha)\,\mathcal{L}_{\mathrm{CE}}(\theta_{\mathfrak{S}})\;+\;\alpha\,\mathcal{L}_{\mathrm{KD}}(\theta_{\mathfrak{S}};\theta_{\mathfrak{T}}) Because teacher distributions are produced on the fly during student training, Word-KD is often described as online and soft distillation. It requires additional teacher forward passes during student training, increasing compute and memory. 2.2 Machine Learning Life Cycle Assessment (MLCA) MLCA Methodology. Estimating the environmental impact of ML solutions requires assessing the impact of the solution throughout its entire life cycle. This can be done by adopting the Machine Learning Life Cycle Assessment (MLCA) framing of Morand et al. 2024 , which adapts the life cycle assessment methodology (LCA) for ML systems. The MLCA