M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluation
Zhaopeng Feng, Jiayuan Su, Jiamei Zheng, Jiahan Ren, Yan Zhang, Jian Wu, Hongwei Wang, Zuozhu Liu · 2024
Read M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluation by Zhaopeng Feng, Jiayuan Su, Jiamei Zheng, Jiahan Ren, Yan Zhang, Jian Wu, Hongwei Wang, Zuozhu Liu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-20127.txt.
Description
Zhaopeng Feng and a team of co-authors let language models debate each other to grade machine translations. Their framework appeared in twenty twenty-four. Single-model judges have lagged behind trained automatic metrics, despite sounding human.
The system, M-MAD, splits the standard human checklist into separate dimensions first. Specialist agents then argue each dimension among themselves, pooling their reasoning. A final synthesis merges the dimension scores into one verdict. Across test sets, the debate beat every earlier language-model judge. It even rivaled the best reference-based metrics while running on a modest model. Ablations credit the design, not luck. Code and data are public for other teams to stress-test.
Catalog record
| CREATOR | Zhaopeng Feng, Jiayuan Su, Jiamei Zheng, Jiahan Ren, Yan Zhang, Jian Wu, Hongwei Wang, Zuozhu Liu |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-20127