You Are What You Train: Effects of Data Composition on Training Context-aware Machine Translation Models

Paweł Mąka, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis · 2025

Read You Are What You Train: Effects of Data Composition on Training Context-aware Machine Translation Models by Paweł Mąka, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-14031.txt.

Description

Paweł Mąka and three co-authors test the you are what you train idea for context-aware translation in a 2025 paper. Good translation needs surrounding context to resolve pronouns and keep coherence, and a leading suspicion blames sparse context-rich examples in standard training data for models' weak context use. Nobody had verified that suspicion systematically.

The team built training sets with controlled proportions of contextually relevant examples, in single-language-pair and multilingual settings. Data sparsity tracked performance strongly, confirming it as a key bottleneck. But gains on one contextual phenomenon did not transfer to others, and cross-lingual transfer was no stronger within language families than across them. Two training strategies they propose squeeze more from the available data, adding several points of accuracy on a context test suite.

Catalog record

CREATORPaweł Mąka, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-14031

Browse the full library · About the Tower of Babel