Neural Models and Language Model Prompting for the Multidimensional Evaluation of Open-Ended Conversations
Michelle Elizabeth, Alicja Kasicka, Natalia Krawczyk, Magalie Ochs, Gwénolé Lecorvé, Justyna Gromada, Lina M. Rojas-Barahona · 2025
Read Neural Models and Language Model Prompting for the Multidimensional Evaluation of Open-Ended Conversations by Michelle Elizabeth, Alicja Kasicka, Natalia Krawczyk, Magalie Ochs, Gwénolé Lecorvé, Justyna Gromada, Lina M. Rojas-Barahona (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-00841.txt.
Description
Michelle Elizabeth leads a team of seven authors on scoring open-ended conversations. Generative dialogue systems are growing fast in number. Judging their quality has become a crucial challenge. The work targets a Dialogue System Technology Challenge track. Models must give dialogue-level scores per dimension. All models used must stay under thirteen billion parameters.
The team pursues two strategies in parallel. One prompts language models to act as evaluators. The other trains encoder-based classification and regression models. Prompting achieves only modest agreement with human judgments. It still ranks second on the test set, behind only the baseline. The smaller trained models agree well on some validation dimensions. Their scores drop on the test set, though. Its score ranges differ sharply for some dimensions.
Catalog record
| CREATOR | Michelle Elizabeth, Alicja Kasicka, Natalia Krawczyk, Magalie Ochs, Gwénolé Lecorvé, Justyna Gromada, Lina M. Rojas-Barahona |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-00841