Beyond “To whom it may concern”: Tailoring Machine Translation to Audience and Intent
Raphael Merx, Ekaterina Vylomova, Trevor Cohn · 2026
Read Beyond “To whom it may concern”: Tailoring Machine Translation to Audience and Intent by Raphael Merx, Ekaterina Vylomova, Trevor Cohn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03259.txt.
Introduction
Merx, Vylomova, and Cohn show that translation should fit its audience and purpose. The same source text can need unlike translations for unlike readers, tones, and intents. Machine translation still treats the task as a fixed map from source to target. Language models let users state purpose outright, but that skill had not been tested at scale.
Their study spans 50 languages, five model sizes, and eight text domains. Clear instructions raised adaptedness a lot, most on informal domains such as chat and social media. Gains grew with model size and with resource level of the language. Instructions beat matched few-shot examples and paragraph context. Standard metrics often fined good adapted translations, so they miss this quality. When no curated instructions exist, models can write their own from document context. That closed up to 80 percent of the gap to curated ones. The authors call for purpose-aware metrics.
Catalog record
| CREATOR | Raphael Merx, Ekaterina Vylomova, Trevor Cohn |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.03259
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond “To whom it may concern”: Tailoring Machine Translation to Audience and Intent Raphael Merx, Ekaterina Vylomova, Trevor Cohn Abstract Translation quality depends on purpose: the same source text demands different translations depending on audience, tone, and communicative intent. Yet MT models and metrics treat translation as a fixed mapping from source to target. LLMs enable users to explicitly specify purpose alongside source text, yet this capability has not been evaluated at scale. We introduce a systematic evaluation of purpose-driven MT across 50 languages, 5 model sizes and 8 text domains. We find that (1) explicit instructions substantially improve translation adaptedness, with larger gains on informal domains (conversation, social media), for larger model sizes and for higher-resource languages; (2) instructions outperform semantically-matched few-shot examples and paragraph-level context; (3) traditional MT metrics fail to capture adaptation quality, often penalizing adapted translations; (4) when curated instructions are unavailable, models can self-generate them from surrounding document context, closing up to 80% of the adaptedness gap to curated instructions. Our results establish that purpose-adapted MT is a viable and measurable capability of LLMs, while highlighting the need for purpose-aware metrics. 1 Introduction Translation theory has long argued that quality is relative to purpose. Skopos theory Nord 1994 holds that a translation’s adequacy should be measured against its intended function, not against a universal standard of correctness. For example, the English phrase “Excuse me” can be translated in myriad different ways depending on intent and audience. Yet traditional MT systems, and the metrics that evaluate them, treat translation as a function from source to target, ignoring purpose. This reduction is a methodological choice. Decades of parallel-corpus training and single-reference evaluation (optimizing for scores against one “correct” reference translation) systematically excluded pragmatic factors like target audience and communicative intent Ma et al. 2025 . Carpuat et al. 2025 call for a paradigm shift: a translation should be faithful to its intended purpose, not merely to the source. LLMs offer a potential solution, as they let users specify the intended context (audience, formality level, domain) alongside the source text. But does this actually work? For which languages, model sizes, and domains? And what is the most effective way to convey purpose: explicit instructions, in-context examples, or surrounding document context? Prior work on context-aware MT is limited in scope, examining either implicit context through document-level translation Wang et al. 2023 , or explicit translation specifications on a single language pair and a single domain Yamada 2023 ; He 2024 ; Sharkas 2025 . NLP research on contextual adaptation through LLMs focuses on narrow phenomena like formality or pronoun consistency Jiang et al. 2023 ; Choudhary et al. 2025 . The broader question of whether LLMs can systematically leverage user-provided specifications to produce purpose-adapted translations remains open. We address the following research questions: RQ1 To what extent can LLMs adapt translations to instructions, and how does this vary by model size, language resource level, and domain? RQ2 How does instruction-based adaptation compare to few-shot in-context examples, and to document context? RQ3 When curated instructions are unavailable, can models generate effective ones from surrounding document context? We evaluate on two translation benchmarks, across 50+ languages, 5 model sizes (Gemma-3-4B/12B/27B, Gemma-4-31B, Qwen3.5-27B), and 8 domains. We find that: 1. Instructions improve translation adaptedness, with gains scaling with model size (§ 4.1 ), and particularly high for informal domains (§ 4.3 ); 2. Instructions outperform semantically-matched few-shot examples and paragraph-level translation (§ 4.4 ); 3. Models can self-generate effective instructions from surrounding document context, closing up to 80% of the adaptedness gap to curated gold instructions (§ 5 ). Together, these findings show that LLMs can operationalize purpose-aware translation, and offer concrete mechanisms for adapting outputs to user intent, a step toward a more user-centered paradigm in MT research. French (Metadata: conversation; informal register; chat with a friend (both female) in a messenger; sarcasm) Source: “I am glad that you agree” Instruction: Translate as a casual, sarcastic message between female friends in a messenger chat. W/o: Je suis content(e) que vous soyez d’accord. \rightarrow formal vous, relatively flat sentence that feels stiff W/: Ah, super que tu sois d’accord, finalement. \rightarrow casual tu, more natural and lighter tone Javanese (Metadata: narration; informal register; Inspired by the fairy tale Timun Mas) Source: “Once upon a time, there lived a husband and wife in a village.” Instruction: Translate as a traditional fairy tale narration. Use simple, evocative language suitable for children. W/o: Dukuné, ana lanang lan wadon sing urip ing desa. \rightarrow matter-of-fact phrasing with no storytelling flavor W/: Ing jaman biyen, ana sawijining pasangan bojo sing urip ing desa. \rightarrow fairy-tale opener, evocative and child-friendly Table 1: Example translations from the 27B model showing register adaptation through instructions. W/o = without instruction; W/ = with instruction. Figure 1: Full pipeline with Indonesian example: (a) instruction generation from source metadata; (b) translation (without/with instructions); (c) translation rating with an LLM-judge, which always sees the instructions. For additional examples, see Table 1 ; for the full prompts used at each stage, see Appendix E . 2 Related Work Specification-guided MT. A small body of work explores incorporating translation specifications (such as purpose, audience, or translation briefs) into LLM prompts, finding that specification-prompted translations receive higher human ratings Yamada 2023 ; He 2024 ; Sharkas 2025 ; Kayano and Sugawara 2025 . However, these evaluations are limited to single language pairs, small test sets (typically 3-5 passages), and often qualitative in nature. For example, He 2024 , working with English-to-Chinese translation of a science article through GPT-4, find that a brief explanation of the translation purpose and context helps deliver a more appropriate translation. Our work scales specification-guided MT evaluation to 50+ languages, 8 domains, 5 model sizes. Contextual adaptation in LLM-based MT. Recent work investigates whether surrounding document context helps LLMs make discourse-level decisions such as formality consistency, pronoun agreement, and lexical cohesion Choudhary et al. 2025 . A tension exists between in-context examples and explicit specifications, where LLMs can ignore explicit instructions when given few-shot examples Zhu et al. 2024 . More broadly, the question of whether LLMs translate better from explicit instructions, or implicit patterns found in few-shot examples, remains open Wu et al. 2025 . Limitations of reference-based metrics. Reference-based metrics like COMET Guerreiro et al. 2024 and MetricX Juraska et al. 2024 primarily evaluate semantic adequacy, which can make them fail to capture cultural adaptation Yuan et al. 2026 . They also depend on a single reference, which can be one valid translation among many Zouhar et al. 2024 . In the latest WMT meta-evaluation, Lavie et al. 2025 find that on harder test sets, ChrF Popović 2015 and LLM-as-judge correlate more strongly with human ratings than COMET or MetricX, calling into question the recent reliance on neural MT metrics. Further, LLM-based evaluation can integrate contextual information that these metrics do not account for Sun et al. 2025 , making it a more versatile way to evaluate translation. Figure 2: Annotation interface in Label Studio. Annotators see the instruction, source sentence, and model translation (left). They mark error spans and give a translation rating (ESA methodology), plus an adaptedness score (right). 3 Methodology 3.1 Task We study translation conditioned on both