Prompting LLMs: Length Control for Isometric Machine Translation

Dávid Javorský, Ondřej Bojar, François Yvon · 2025

Read Prompting LLMs: Length Control for Isometric Machine Translation by Dávid Javorský, Ondřej Bojar, François Yvon (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04855.txt.

Introduction

Dávid Javorský, Ondřej Bojar, and François Yvon probe in 2025 how to keep a translation the same length as its source, a need in captions and dubbed speech. They used eight open language models on three language pairs from English. They varied the wording of prompts and the count and choice of worked examples.

Prompt wording that matched the examples proved key to length control. Models only wrote shorter text when shown extreme cases, and equal-length examples often led them to ignore the limit. More examples helped quality a bit, with gains fading past five. Weighing several outputs at once gave the best mix of length and quality.

Catalog record

CREATORDávid Javorský, Ondřej Bojar, François Yvon
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-04855

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Prompting LLMs: Length Control for Isometric Machine Translation Dávid Javorský, Ondřej Bojar, François Yvon Abstract In this study, we explore the effectiveness of isometric machine translation across multiple language pairs (En → \to De, En → \to Fr, and En → \to Es) under the conditions of the IWSLT Isometric Shared Task 2022. Using eight open-source large language models (LLMs) of varying sizes, we investigate how different prompting strategies, varying numbers of few-shot examples, and demonstration selection influence translation quality and length control. We discover that the phrasing of instructions, when aligned with the properties of the provided demonstrations, plays a crucial role in controlling the output length. Our experiments show that LLMs tend to produce shorter translations only when presented with extreme examples, while isometric demonstrations often lead to the models disregarding length constraints. While few-shot prompting generally enhances translation quality, further improvements are marginal across 5, 10, and 20-shot settings. Finally, considering multiple outputs allows to notably improve overall tradeoff between the length and quality, yielding state-of-the-art performance for some language pairs. 1 Introduction Accurate and concise translations are increasingly needed in media applications such as subtitling Matusov et al. 2019 ; Karakanta et al. 2020 and dubbing Federico et al. 2020 ; Lakew et al. 2021 ; Tam et al. 2022 ; Lakew et al. 2022 ; Rao et al. 2023 , where length constraints are critical. Dubbing, in particular, requires translations to stay within ±10% of the source character-level length for seamless audio alignment Lakew et al. 2022 , a constraint known as isometric machine translation . The 2022 Isometric MT Shared Task Anastasopoulos et al. 2022 found that most participating systems used lead tokens for length control, with some incorporating reranking or adjusted positional embeddings. Recent work also explored reinforcement learning for isometric English-Hindi MT Mhaskar et al. 2024 and examined length constraints in multiple language pairs Bhavsar et al. 2022 . Controlling translation length remains challenging compared to other constrained MT tasks, such as politeness Sennrich et al. 2016 or diversity Shu et al. 2019 control. Previous approaches in encoder-decoder MT used length tokens Lakew et al. 2019 , positional embeddings Takase and Okazaki 2019 ; Buet and Yvon 2021 , restricted search spaces Niehues 2020 , auxiliary length prediction tasks Yang et al. 2020 , and explicit compression methods Li et al. 2020 . Figure 1: Overview of our experiment with prompts asking for different length constraints for the desired translation, complemented with few-shot examples demonstrating the given constraint (match) or not (no-match). Strong enough control to reach isometric translation needs matching instructions and preferably Tiny or Short demonstrations. The construction of demonstration sets is described in Section 3 and the prompt content is presented in Table 6 in Section B.2 . En-De En-Fr En-Es Setup LR LC \uparrow Count LR LC \uparrow Count LR LC \uparrow Count Dev Both 1.14 \pm 0.3 38.2 1415 1.14 \pm 0.3 36.4 1412 1.08 \pm 0.3 50.5 1316 Test Development 1.15 \pm 0.2 37.5 200 1.16 \pm 0.2 34.0 200 1.03 \pm 0.2 58.0 200 Final Evaluation 1.03 \pm 0.2 65.5 200 1.09 \pm 0.5 72.5 200 0.98 \pm 0.2 64.0 200 Table 1: The average target-to-source sample length ratio and its standard deviation (LR), length compliance (LC), i.e. the percentage of target-side sentences within a ±10% range of the source character count, and the number of samples for two setups ( Development and Final Evaluation ) and for the testset (the MuST-C tst-COMMON and blind test sets) and the devset (MuST-C). The devset is used for selecting examples for few-shot prompting. En-De Pool type Count Min Max avg \pm std Random 1415 0.43 5.80 1.14 \pm 0.27 Isometric 537 0.90 1.10 1.02 \pm 0.05 Same 50 0.99 1.00 1.00 \pm 0.00 Short 343 0.43 1.00 0.90 \pm 0.11 Tiny 50 0.43 0.81 0.68 \pm 0.11 Table 2: Statistics of pools for En \to De: The number of samples, minimum and maximum target/source length ratio, and its average and standard deviation. With the rise of large language models (LLMs) Radford et al. 2019 , there has been a shift toward prompting Vilar et al. 2023 ; Zhang et al. 2023a ; Bawden and Yvon 2023 and fine-tuning Zhang et al. 2023b ; Moslem et al. 2023 for MT. Prompting strategies notably affect performance, especially in few-shot settings Vilar et al. 2023 . Studies found that randomly selected examples often improve results Zhang et al. 2023a ; Bawden and Yvon 2023 , though performance gains plateau beyond five examples Chowdhery et al. 2023 ; Vilar et al. 2023 . While models like Bloom tend to overgenerate in zero-shot settings Bawden and Yvon 2023 , fine-tuning methods such as QLoRA Zhang et al. 2023b have shown superior performance over few-shot learning. Real-time adaptive MT has also demonstrated strong results, with models like ChatGPT rivaling traditional MT systems Moslem et al. 2023 ; Hendy et al. 2023 . The use of LLMs for MT has led to the exploration of various prompt templates, with simple structures like ‘ [src]: [input] \n [tgt]: ’ proving effective Zhang et al. 2023a ; Briakou et al. 2023 ; Zeng et al. 2022 . The impact of example selection has also been examined, confirming that beyond five-shot settings, improvements become marginal Garcia et al. 2023 ; Zhang et al. 2023a ; Chowdhery et al. 2023 ; Vilar et al. 2023 . Given these insights, we explore the application of LLMs to isometric MT, focusing on length control strategies. We analyze four prompting approaches: (1) uncontrolled translation, (2) isometric translation (±10% length variation), (3) same-length translation, and (4) shorter translation, each paired with corresponding demonstration sets. Experiments are conducted on eight open-weight models (Llama 3, Gemma 2, Qwen 2 of two sizes each, and Mistral and Mixtral) across 0, 5, 10, and 20-shot settings for En \to De, En \to Fr, and En \to Es, following the 2022 Isometric Shared Task setup ( Anastasopoulos et al. 2022 ) . Our results show that few-shot demonstrations affect translation outputs, but precise length control requires well-aligned instructions reflecting example properties, as summarized in Figure 1 . Additionally, we show that generating multiple outputs with different example sets substantially improves length control, matching competitive isometric MT systems and offering high potential for synthetic data creation in training encoder-decoder models. We publicly release all collected data for potential future analyses. 1 1 1 https://github.com/J4VORSKY/Isometric-MT 2 Experimental Setup Development First, we conduct experiments with multiple settings (varying prompt type, the type of pools of demonstrations, and shot count in few-shot learning) to identify the best-performing configuration for length control. We refer to this as the Development setup and use the following data: • Demonstration set : We use the MuST-C dev-set for selecting few-shot examples. We choose the devset over the trainset to reserve the latter for potential future fine-tuning. • Testset : We use the first 200 examples from the MuST-C tst-COMMON, matching the number of examples in the evaluation blindset of the 2022 Isometric Shared Task. Final Evaluation We then use the best-performing setting from the Development and evaluate it on the Isometric Shared Task test set: • Demonstration set : We use the same demontration set

Read the complete text