A comparison of data filtering techniques for English-Polish LLM-based machine translation in the biomedical domain
Jorge del Pozo Lérida, Kamil Kojs, János Máté, Mikołaj Antoni Barański, Christian Hardmeier · 2025
Read A comparison of data filtering techniques for English-Polish LLM-based machine translation in the biomedical domain by Jorge del Pozo Lérida, Kamil Kojs, János Máté, Mikołaj Antoni Barański, Christian Hardmeier (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16533.txt.
Description
Jorge del Pozo Lérida and four co-authors match ways to clean the paired text sets used to train machine translation, for English and Polish medical text. Their paper is from two thousand twenty-five.
Big text sets scraped from the web hold weak pairs and repeats, and they cost a lot to train on. The team filtered a known medical set with three known tools, made sets of varied sizes, and used them to fine-tune a translation model. They scored the output with a standard mark and with bilingual readers. Two of the filters cut the set a lot while keeping quality high. The results guide teams that build translation for niche fields.
Catalog record
| CREATOR | Jorge del Pozo Lérida, Kamil Kojs, János Máté, Mikołaj Antoni Barański, Christian Hardmeier |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-16533