ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT

Mikołaj Pokrywka, Wojciech Kusa, Mieszko Rutkowski, Mikołaj Koszowski · 2025

Read ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT by Mikołaj Pokrywka, Wojciech Kusa, Mieszko Rutkowski, Mikołaj Koszowski (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04929.txt.

Introduction

Mikołaj Pokrywka and three co-authors built in 2025 a data set for translating online shop text between Czech and Polish with context. Product text is full of short, ambiguous lines, and plain translators guess badly. Their set pairs over eleven thousand product sentences with images and catalog data.

The team tested ways to feed that context to translators. A vision-language model gained from seeing the product image, and text models gained from category paths and image descriptions. Context, the study shows, is not decoration in shop translation; it is the difference between a guess and the right word.

Catalog record

CREATORMikołaj Pokrywka, Wojciech Kusa, Mieszko Rutkowski, Mikołaj Koszowski
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-04929

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT Mikołaj Pokrywka, Wojciech Kusa, Mieszko Rutkowski, Mikołaj Koszowski Abstract Neural Machine Translation (NMT) has improved translation by using Transformer-based models, but it still struggles with word ambiguity and context. This problem is especially important in domain-specific applications, which often have problems with unclear sentences or poor data quality. Our research explores how adding information to models can improve translations in the context of e-commerce data. To this end we create ConECT -- a new Czech-to-Polish e-commerce product translation dataset coupled with images and product metadata consisting of 11,400 sentence pairs. We then investigate and compare different methods that are applicable to context-aware translation. We test a vision-language model (VLM), finding that visual context aids translation quality. Additionally, we explore the incorporation of contextual information into text-to-text models, such as the product’s category path or image descriptions. The results of our study demonstrate that the incorporation of contextual information leads to an improvement in the quality of machine translation. We make the new dataset publicly available. 1 1 1 https://huggingface.co/datasets/allegro/ConECT 1 Introduction Neural Machine Translation (NMT) has significantly advanced the field of machine translation by leveraging Transformer-based models Bahdanau et al. 2015 ; Vaswani et al. 2017b . These models have been critical in enhancing translation quality, particularly by incorporating mechanisms such as cross-attention to achieve better semantic understanding. However, despite these improvements, sentence-level translation in NMT often struggles with issues such as contextual disambiguation Rios Gonzales et al. 2017 . For example, the word “pen” can refer to a writing instrument or an enclosure for animals, depending on the context. These ambiguities present a significant challenge in achieving accurate translations based solely on the context of single sentences. Figure 1: We evaluate three methods for contextualisation in e-commerce MT: (1) combining images with text in VLM, (2) appending category path context in NMT, and (3) a cascade approach consisting of a vision Q&A and a text-to-text NMT. In an attempt to address these limitations, Multimodal Machine Translation (MMT) has emerged as a promising paradigm Specia et al. 2016 ; Shen et al. 2024 . MMT integrates visual information alongside textual data to provide additional context, thereby enhancing translation quality. Studies have shown that leveraging visual cues can significantly improve the disambiguation of lexical items and contribute to more accurate translations Yao and Wan 2020 ; Wang and Xiong 2021 . For instance, visual context can help resolve ambiguities in sentences where the textual information alone is insufficient Liu et al. 2021 . However, high-quality machine translation training data with aligned images is available only for a fraction of the parallel corpora. In this work, we explore the effectiveness of integrating context information into a domain-specific translation task. We create a new testset for Czech-to-Polish translation in e-commerce, and test three approaches for context-aware MT (Figure 1 ). We fine-tune and evaluate a VLM, as well as traditional NMT models for which we integrate contextual information as special instruction tokens. Our findings demonstrate that visual context can enhance translation quality in domain-specific scenarios, serving as a valuable additional feature. Our contributions are as follows: • We release a new multimodal dataset for e-commerce MT for the under-researched cs-pl language pair; • We fine-tune and evaluate a VLM on the domain-specific MT task; • We propose a method for encoding category paths and image descriptions in small NMT models. We start by describing related work ( section 2 ), then introduce the ConECT dataset ( section 3 ), explain our experimental setup ( section 4 ), and finally discuss the results ( section 5 ). 2 Related work In this section we discuss multimodal and e-commerce product-oriented MT. 2.1 Multimodal Machine Translation Multimodal Machine Translation (MMT), which integrates visual context into machine translation, has garnered significant attention in recent years Elliott et al. 2016 ; Shen et al. 2024 . Research has shown that visuals can effectively bridge linguistic gaps between languages Chen et al. 2019 ; Sigurdsson et al. 2020 ; Su et al. 2019 . Early efforts by Calixto and Liu 2017 incorporated global image features into the encoder or decoder of NMT models. Subsequent studies have explored the use of more granular image contexts, such as spatial image regions with attention mechanisms Calixto et al. 2017a ; Caglayan et al. 2016 . Yang et al. 2020 introduced a method for joint training of source-to-target and target-to-source models to promote visual agreement. Yin et al. 2020 built a multi-modal graph linking image objects and source words, leveraging an external visual grounding model for alignment. 2.2 Benchmarks and Shared Tasks in MMT Between 2016 and 2018, WMT organised a shared task on MMT Specia et al. 2016 ; Elliott et al. 2017 ; Barrault et al. 2018 . The organisers primarily used the Multi30K dataset Elliott et al. 2016 and metrics like BLEU, METEOR, and TER. In the first edition of the shared task, Specia et al. 2016 found that neural MMT models initially underperformed compared to text-only SMT models. However, Elliott et al. 2017 and Barrault et al. 2018 expanded languages and test datasets, noting performance improvements in MMT models, especially when incorporating external resources. Elliott 2018 introduced an adversarial evaluation method to assess whether multimodal translation systems effectively utilize visual context. This method evaluates the performance difference of a system when provided with either a congruent or an incongruent image as additional context. Futeral et al. 2023 introduced the CoMMuTE (Contrastive Multilingual Multimodal Translation Evaluation) dataset to evaluate multimodal machine translation systems with a focus on resolving ambiguity using images. Their approach, which includes neural adapters and guided self-attention, showed significant improvement over text-only models, particularly in English–French, English–German, and English–Czech translations. Futeral et al. 2024 extended the setup with ZeroMMT , a technique for zero-shot multimodal machine translation that does not rely on fully supervised data. This method, which uses visually conditioned masked language modelling and Kullback-Leibler divergence training, demonstrated near state-of-the-art performance and was extended to Arabic, Russian, and Chinese. 2.3 E-commerce product-oriented MT An e-commerce product-oriented machine translation task uses product images and product metadata as inputs. Calixto et al. 2017c pioneered this task with a bilingual product description dataset, evaluating models such as phrase-based statistical MT (PBSMT), text-only MT, and MMT. Their results highlighted PBSMT’s superior performance, with MMT models enhancing translation when re-ranking PBSMT outputs. Calixto et al. 2017b highlight the potential impact of multi-modal NMT in the context of e-commerce product listings. With only a limited amount of multimodal and multilingual training data available, both text-only and multi-modal NMT models failed to outperform a productive SMT system. Song et al. 2021 introduced a large-scale dataset along with a unified pre-training and fine-tuning framework, proposing pre-training tasks for aligning bilingual texts and product images. These tasks include masked word reconstruction with bilingual and image context, semantic matching between text and image, and masking of source words conveying product attributes. This framework contributed to more robust translation models. 2.4 Visual information integration in LLMs The field of NLP has evolved with the introduction of large language models (LLMs) Ouyang et al. 2022 ; Achiam et al. 2023 ; Touvron et al. 2023 . Since traditional text-only

Read the complete text