Rich Insights from Cheap Signals: Efficient Evaluations via Tensor Factorization
Felipe Maia Polo, Aida Nematzadeh, Virginia Aglietti, Adam Fisch, Isabela Albuquerque · 2026
Read Rich Insights from Cheap Signals: Efficient Evaluations via Tensor Factorization by Felipe Maia Polo, Aida Nematzadeh, Virginia Aglietti, Adam Fisch, Isabela Albuquerque (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-02029.txt.
Description
Felipe Maia Polo, Aida Nematzadeh, Virginia Aglietti, and colleagues extract fine-grained model evaluations from cheap automated scores. Their 2026 work moves past leaderboard averages that hide how a model does on each prompt type. Human gold labels are too costly at that scale, while raw automated ratings drift from human taste.
Their statistical model, based on tensor factorization, merges cheap autorater data with a small human set. Autorater scores pre-train latent representations of prompts and of the models being judged. A small calibration set then aligns those representations to human preferences. The method proves robust to autorater quality and predicts per-prompt human views better than standard baselines. Tight confidence intervals come along, and granular leaderboards can be built from autorater scores alone.
Catalog record
| CREATOR | Felipe Maia Polo, Aida Nematzadeh, Virginia Aglietti, Adam Fisch, Isabela Albuquerque |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71263 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.02029