A Dataset for Probing Translationese Preferences in English-to-Swedish Translation
2026
Read A Dataset for Probing Translationese Preferences in English-to-Swedish Translation (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08450.txt.
Introduction
Kunz and colleagues build the first openly available English-to-Swedish dataset pairing translationese sentences with idiomatic alternatives. Each pair is tagged with the problems in the original translation. The set is designed to probe what language models prefer when translating. Tests on smaller Swedish and multilingual models find they often favor the translationese phrasing. Dropping the English source sentence makes models pick the human alternative more often, showing that seeing the source pushes models toward literal wording, though even without it they still often prefer translationese.
Catalog record
| YEAR | 2026 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54836 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.08450
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Dataset for Probing Translationese Preferences in English-to-Swedish Translation Abstract Translations often carry traces of the source language, a phenomenon known as translationese . We introduce the first freely available English-to-Swedish dataset contrasting translationese sentences with idiomatic alternatives, designed to probe intrinsic preferences of language models. It includes error tags and descriptions of the problems in the original translations. In experiments evaluating smaller Swedish and multilingual LLMs with our dataset, we find that they often favor the translationese phrasing. Human alternatives are chosen more often when the English source sentence is omitted, indicating that exposure to the source biases models toward literal translations, although even without context models often prefer the translationese variant. Our dataset and findings provide a resource and benchmark for developing models that produce more natural, idiomatic output in non-English languages. Keywords: Translationese, Idiomaticity, Machine Translation, Evaluation, Minimal Pair Probes \setexampleoptions fs=, fspreamble=, subnumberformat=a., addsubnumbersep=-4pt, A Dataset for Probing Translationese Preferences in English-to-Swedish Translation Jenny Kunz Anja Jarochenko Marcel Bollmann Department of Computer and Information Science Linköping University [email protected] [email protected] [email protected] Abstract content 1. Introduction Translations have long been known to carry traces of the source language and differ in style and features from texts originally written in the target language; a phenomenon commonly called translationese ( 9 ) . Research shows that translationese is particularly strong in machine translation, leading to simplified and less varied language marked by reduced lexical and morphological richness ( 27 ) . Such output is often easily distinguishable from original text ( 15 ; 18 ) . Recent work suggests that LLMs produce less literal translations ( 24 ) . However, even though their outputs show increased lexical diversity compared to specialized machine translation systems, they can still be reliably distinguished from human-written text ( 14 ) . For many languages, it is common to use translated datasets especially for LLM evaluation ( 22 ; 4 ) and instruction tuning ( 17 ; 6 ; 12 ) , as there is often no practical alternative. In addition, for low-resource languages, but to a lesser extent even for high-resource languages such as English, large portions of modern web-crawled corpora that constitute the training data of LLMs are translations ( 25 ) . It is therefore important to analyze this problem and assess its extent, in order to move towards models that produce natural, idiomatic output in non-English languages. 16 investigate conventionalized idiom knowledge and translationese in LLM outputs for Swedish. Analyzing sentence pairs from a book on translationese ( 13 ) , which contains translations with translationese phrasing alongside idiomatic alternatives suggested by the author, they examine whether the model assigns higher probability to the translationese version or to the idiomatic alternative. Building on this work, we construct a dataset for probing translationese preferences. Our dataset is similar in its setup, but addresses several limitations of the previous dataset: it is fully open under a permissive license, includes the English source sentence with preceding context to also study preferences in a translation context, and provides annotations tagging the type of translationese, enabling more fine-grained analysis of model behavior. To our knowledge, this is the first freely available dataset explicitly contrasting translationese with idiomatic alternatives for Swedish. We provide a detailed description of the dataset, including both qualitative and quantitative analyses of translationese examples in English-to-Swedish translations produced by a smaller specialized machine translation system. We also compare these to translations generated by a state-of-the-art LLM, highlighting that while it does not fully resolve any specific problem, it is much better at generating more idiomatic words and phrasings. In experiments probing the intrinsic preferences of smaller multilingual LLMs with our dataset, findings show a strong bias toward the translationese phrasing. Notably, models favor the human alternative more often when omitting translation context in the prompt, indicating that exposure to the English source sentence biases them toward the more literal, translationese wording. However, increasing the context window helps the model become less biased towards the translationese variant. Release We release our dataset on HuggingFace under https://huggingface.co/datasets/liu-nlp/translationese-opensubtitles . A full version including all annotations with further explanations, including the code to reproduce our experiments, is available on GitHub: https://github.com/jekunz/translationese . 2. Background Translationese Translated texts often preserve features of the source language while avoiding target-language constructions that diverge more strongly from the source language. This is just one aspect of a phenomenon known in both human and machine translation as translationese ( 9 ) . Translationese is not generally a sign of poor translation quality ( 10 ) but the resulting texts have properties that are different from idiomatic language. As 3 shows, translated texts tend to be more explicit than their originals, are lexically and syntactically simpler, and follow a more conventional style. 15 show that classifiers can easily distinguish between original and translated text, and even determine the source language of the translated text. 14 show that this still holds true with LLMs, as distinguishing human from LLM-translated text works almost perfectly. 27 show that machine translation systems tend to simplify language, losing lexical and morphological variety, but that Transformer-based models preserve more diversity than earlier systems. Similarly, 14 find that LLMs generate more varied text than specialized machine translation systems, yet their outputs remain clearly distinguishable from human-written text. 5 compare human and machine translationese, finding that while human translations have similar properties across modalities (written versus spoken), machine translations exhibit fundamentally different patterns. 18 let annotators mark unnatural parts of translations, distinguishing between rigid sentence structures and literal word choices. They find that LLM outputs still contain much translationese, and trace this problem to translationese in the training data. They propose a polishing step where the model revises its own translation to reduce this effect, while prompts for natural style do not consistently help. Other studies have examined idiom translation as a common case of overly literal translation. 8 show that systems often translate idioms word by word, leading to semantic errors. 7 also find that models tend to treat idioms compositionally, i.e., as literal phrases. When they recognize idioms as non-compositional units, interactions between the idiom’s parts and the surrounding context decrease. We annotate “idioms” as one category of error in our dataset to enable further research into this phenomenon. English-to-Swedish translationese 9 gives the classic example that English often uses an adjective together with a generic noun (e.g. “an [ADJ] thing”), whereas Swedish prefers to use a pronominal construction (“something [ADJ]”) instead. In translations from English, such noun constructions therefore appear more frequently than in original Swedish texts—e.g., “a silly thing happened” may be literally translated as en fånig sak hände instead of the more idiomatic något fånigt hände (“something silly happened”). 13 documents other typical features of translation-influenced Swedish: Participial forms are less common than in English, e.g., Ta en bild på dig själv tittandes in i kameran (“Take a picture of yourself looking into the camera”) versus the more idiomatic när du tittar mot kameran (“…when you look at the camera”). Also, noun phrases in predicative expressions often use the article when it should be omitted, e.g., Jag är en översättare \rightarrow Jag är översättare (“I am a translator” \rightarrow “I am translator”). 2 analyzes adjective usage