Rich Insights from Cheap Signals: Efficient Evaluations via Tensor Factorization

Felipe Maia Polo, Aida Nematzadeh, Virginia Aglietti, Adam Fisch, Isabela Albuquerque · 2026

Read Rich Insights from Cheap Signals: Efficient Evaluations via Tensor Factorization by Felipe Maia Polo, Aida Nematzadeh, Virginia Aglietti, Adam Fisch, Isabela Albuquerque (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-02029.txt.

Description

Felipe Maia Polo, Aida Nematzadeh, Virginia Aglietti, and colleagues extract fine-grained model evaluations from cheap automated scores. Their 2026 work moves past leaderboard averages that hide how a model does on each prompt type. Human gold labels are too costly at that scale, while raw automated ratings drift from human taste.

Their statistical model, based on tensor factorization, merges cheap autorater data with a small human set. Autorater scores pre-train latent representations of prompts and of the models being judged. A small calibration set then aligns those representations to human preferences. The method proves robust to autorater quality and predicts per-prompt human views better than standard baselines. Tight confidence intervals come along, and granular leaderboards can be built from autorater scores alone.

Catalog record

CREATORFelipe Maia Polo, Aida Nematzadeh, Virginia Aglietti, Adam Fisch, Isabela Albuquerque
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71263
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.02029

Browse the full library · About the Tower of Babel