Lingua Custodia’s participation at the WMT 2025 Terminology shared task
Jingshu Liu, Raheel Qader, Gaëtan Caillaut, Mariam Nakhlé · 2025
Read Lingua Custodia’s participation at the WMT 2025 Terminology shared task by Jingshu Liu, Raheel Qader, Gaëtan Caillaut, Mariam Nakhlé (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17504.txt.
Introduction
Machine translation systems often stumble over specialised terms, which must be rendered the same way every time in technical and legal texts. Jingshu Liu, Raheel Qader, Gaëtan Caillaut, and Mariam Nakhlé describe the entry that the Lingua Custodia team built for the WMT 2025 shared task on terminology. The task covered three language pairs: English into German, English into Spanish, and English into Russian.
The team started from two recent instruction tuned language models, Qwen3 and Gemma3 in their smaller sizes, and fine tuned them further in a two stage process aimed at honouring supplied terminology. A system description of this kind lets other builders see which base models and training stages a competitive terminology entry used. Note that the arXiv listing abstract for this record does not match the paper itself, so this description follows the hosted text.
Catalog record
| CREATOR | Jingshu Liu, Raheel Qader, Gaëtan Caillaut, Mariam Nakhlé |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-17504
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Lingua Custodia’s participation at the WMT 2025 Terminology shared task Jingshu Liu, Raheel Qader, Gaëtan Caillaut, Mariam Nakhlé Abstract This paper presents Lingua Custodia’s submission to the WMT25 shared task on Terminology shared task targeting three language pairs: English-German, English-Spanish, and English-Russian. Our approach builds on two recent instruction-tuned large language models Qwen3-4b and Gemma3-4b-it which we further finetune in a two-stage process. In the first stage, we perform supervised fine-tuning (SFT) on parallel data augmented with explicit terminology constraints. In the second stage, we apply Group Relative Policy Optimization (GRPO) with a custom reward function designed to encourage correct terminology usage while preserving general translation quality. Experimental results on English-German and English-Spanish language pairs show > = 10 >=10 points of accuracy gain in term match compared to the their baseline counterparts and > = 5 >=5 points for English-Russian, while maintaining similar Comet-Kiwi scores. 1 Introduction Nowadays both classical encode-decoder ( Sutskever et al. 2014 ; Bahdanau et al. 2014 ; Luong et al. 2015 ; Vaswani et al. 2017 ) and decoder only LLM based neural machine translation systems ( Alves et al. 2024 ; Cheng et al. 2025 ; Qwen et al. 2025 ) have demonstrated strong performance across a wide range of translation tasks, with the latter increasingly favored due to their scalability, multilingual generalization, and ability to leverage instruction-tuning for domain adaptation. We leverage LLM-based MT systems due to their ease of fine-tuning and scalability, which make them well-suited for rapidly adapting to new domains and tasks. This paper describes our submission to the WMT25 Terminology translation task in English to German, English to Spanish and English to Russian. The task aims to assess the extent to which machine translation models can utilize additional information regarding the translation of terminologies. Previous works on machine translation with terminology control can be grouped into two categories according to whether the method needs training the model with terminology information. One group incorporates the constraint during the inference time ( Hokamp and Liu 2017 ; Post and Vilar 2018 ; Susanto et al. 2020 ) . An LLM trained without any terminology constraint can also be seen as this category with zero or few shot prompting during the inference. These methods can typically satisfy most of the constraints but suffer from high computational cost and sometimes low translation quality because they always try to strictly apply the terminology constraint regardless of the correctness of the whole sentence. The other group integrates lexical constraints during training ( Dinu et al. 2019 ; Crego et al. 2016 ; Song et al. 2019 ; Alves et al. 2024 ) by annotating terminology in order to guide the model to learn the enforcement of the translation constraints. The main disadvantage of these methods is the lack of guarantee of all constraints in the translations. Another limitation of these works is that they usually requires extra resources such as a term dictionary, an annotator(a big LLM) to augment the data. Our work extends the prior approaches of Ailem et al. 2021 and Liu et al. 2023 , which follow the second line of methods. Furthermore, we build our systems by finetuning two of the most prominent open large language models: Qwen3 ( Qwen et al. 2025 ) and Gemma3 ( Team et al. 2025 ) . We evaluate our work on the WMT25 terminology track1 testsets. Since the reference is not released by the time of writing this paper, we evaluate our system by a simple naive strict match with respect to the target constraint completed by the Comet-Kiwi ( Rei et al. 2022 ) score to make sure that the translation ability is sustained. Results show that our proposed approach outperforms the baseline by an average of more than 10 points in term match accuracy. 2 Method In this section we present our systems for the terminology task. We focus on the sentence-level track and our all of our proposed systems are multilingual. Therefore one system is applied to all the language pairs during the inference time. 2.1 Background In order to annotate the Enligsh-German and English-Spanish data, we choose to exploit the dictionary extracted by Liu et al. 2023 which builds a large bilingual terminology in a unsupervised manner. Alternative approaches such as Hazem and Morin 2016 and Liu et al. 2018 can also achieve the same goal bu they require heavy computation, while Artetxe et al. 2016 learns only single word bilingual lexicon. We follow the same extraction setting as in Liu et al. 2023 to keep the process efficient: limit the candidate ngams to 5. Our systems are finetuned models from two popular instruction LLMs Qwen3-4b ( Yang et al. 2025 ) and Gemma3-4b-it ( Team et al. 2025 ) . Both models stand out as high-performing models with open weights and efficient inference capabilities. Besides, they have been optimized for instruction following and multilingual understanding, which fit perfectly our scenario. With 4 billion parameters, they are well-suited for fast fine-tuning and efficient inference in constrained environments, enabling manageable deployment in our production environment without the extensive resource demands of larger models. 2.2 Instruction data generation We use public open general bilingual data to finetune the LLMs. For English-German and English-Spanish pairs, we use Common Crawl ( Common Crawl 2023 ) as our database. As for the English-Russian pairs, we take the train dataset provided by WMT25 which is a collection of various open data sources 1 1 1 https://www2.statmt.org/wmt25/mtdata/mtdata.recipes.wmt25-constrained.yml . We will explain the details of the data cleaning and filtering in the Section 3.1. To annotate the instruction data with terminology control, we first employed Meta-Llama-3.1-405B-Instruct ( Grattafiori et al. 2024 ) to generate 10 diverse instruction templates. For example: Translate the following sentence from [src_lang] to [tgt_lang] while adhering to the specified terminology: [mapping_list] should be used as a reference. This variation is intended to improve model generalization across diverse instructions. The [mapping_list] is obtained by matching the sentence against our bilingual terminology dictionary, extracted using the method described in Section 2.1. To construct the final training set with terminology control, each parallel sentence is parsed using the dictionary, one instruction template is randomly selected, and the result is transformed into either the Qwen3 or Gemma3 chat format. Following the official recommendations of the Qwen3 authors, we disable the thinking mode by inserting empty content between the thinking tags and exclude the corresponding thinking tag tokens from the training loss computation. 2.3 Model finetuning Our finetuning process consists of two main phases: • Supervised fine-tuning (SFT) • Group Relative Policy Optimization (GRPO) In the first phase, we perform SFT on our curated multilingual dataset, enabling the model to learn the desired instruction-following and terminology control behavior. In the second phase, we apply GRPO to further align the model’s outputs with task-specific quality objectives, enhancing adherence to the provided terminology constraints while keeping the translation quality. For translation quality, we define a BLEU-based reward : R_{\text{BLEU}}(y,y^{\ast})=\frac{\text{BLEU}(y,y^{\ast})}{100} where \text{BLEU}(y,y^{\ast}) is the sentence-level BLEU score computed using SacreBLEU ( Post 2018 ) . The score is normalized to the range [0,1] to scale the reward. For terminology adherence, we design a constraint-following reward that parses the user instruction for