ProxAnn : Use-Oriented Evaluations of Topic Models and Document Clustering
Alexander Hoyle, Lorena Calvo-Bartolomé, Jordan Boyd-Graber, Philip Resnik · 2025
Read ProxAnn : Use-Oriented Evaluations of Topic Models and Document Clustering by Alexander Hoyle, Lorena Calvo-Bartolomé, Jordan Boyd-Graber, Philip Resnik (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-00828.txt.
Description
Alexander Hoyle and three co-authors build a fairer way to judge topic models. Their 2025 paper presents ProxAnn, an evaluation method grounded in real use. Automated scores for topic models track human taste poorly. Expert labels match better but cannot scale. The team wanted a test that mirrors how practitioners actually use these models.
In their protocol, a judge reads a topic's texts, names the category, then applies that name to new texts. Crowd workers ran the protocol on many models across two datasets. The team then checked whether language models could stand in for the humans. The best stand-ins matched a human judge closely enough to serve. Software, a web tool, and the data are shared freely.
Catalog record
| CREATOR | Alexander Hoyle, Lorena Calvo-Bartolomé, Jordan Boyd-Graber, Philip Resnik |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-00828