Proverbs Run in Pairs: Evaluating Proverb Translation Capability of Large Language Model
Minghan Wang, Viet-Thanh Pham, Farhad Moghimifar, Thuy-Trang Vu · 2025
Read Proverbs Run in Pairs: Evaluating Proverb Translation Capability of Large Language Model by Minghan Wang, Viet-Thanh Pham, Farhad Moghimifar, Thuy-Trang Vu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-11953.txt.
Description
Minghan Wang, Viet-Thanh Pham, Farhad Moghimifar, and Thuy-Trang Vu test how well machines translate proverbs, the most culture-soaked lines a language owns. Their 2025 paper builds a dataset of standalone proverbs and proverbs inside conversation, across four language pairs. Standard translation research mostly ignores such fixed, figurative sayings.
Models did well between languages with similar cultural backgrounds and stumbled further apart. Large language models generally beat classic neural translation systems at the task. The sharper finding concerns grading: BLEU, CHRF++, and COMET all proved unreliable judges of proverb quality, since a literal rendering can score well while losing the saying's point. The authors call for evaluation metrics that understand culture, not just word overlap. Proverbs, they show, expose what fluent translation still misses.
Catalog record
| CREATOR | Minghan Wang, Viet-Thanh Pham, Farhad Moghimifar, Thuy-Trang Vu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-11953