A comparison of data filtering techniques for English-Polish LLM-based machine translation in the biomedical domain

Jorge del Pozo Lérida, Kamil Kojs, János Máté, Mikołaj Antoni Barański, Christian Hardmeier · 2025

Read A comparison of data filtering techniques for English-Polish LLM-based machine translation in the biomedical domain by Jorge del Pozo Lérida, Kamil Kojs, János Máté, Mikołaj Antoni Barański, Christian Hardmeier (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16533.txt.

Description

Jorge del Pozo Lérida and four co-authors match ways to clean the paired text sets used to train machine translation, for English and Polish medical text. Their paper is from two thousand twenty-five.

Big text sets scraped from the web hold weak pairs and repeats, and they cost a lot to train on. The team filtered a known medical set with three known tools, made sets of varied sizes, and used them to fine-tune a translation model. They scored the output with a standard mark and with bilingual readers. Two of the filters cut the set a lot while keeping quality high. The results guide teams that build translation for niche fields.

Catalog record

CREATORJorge del Pozo Lérida, Kamil Kojs, János Máté, Mikołaj Antoni Barański, Christian Hardmeier
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-16533

Browse the full library · About the Tower of Babel