SkillRater: Untangling Capabilities in Multimodal Data
Naveen Sahi, Jeremy Dohmann, Armen Aghajanyan, Akshat Shrivastava · 2026
Read SkillRater: Untangling Capabilities in Multimodal Data by Naveen Sahi, Jeremy Dohmann, Armen Aghajanyan, Akshat Shrivastava (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11615.txt.
Description
Training data are usually scored with one number for quality, but a model that must learn several skills at once cannot be served well by a single score. Naveen Sahi and three co-authors argue quality is better treated as multidimensional, with one dimension per capability. Their framework, SkillRater, trains a separate rater for each skill by meta-learning on its own validation goal, then combines the raters with a progressive rule.
At each training stage a sample survives if any rater ranks it above a bar that tightens over time, keeping variety early and focusing on high-value samples late. Tested on vision-language models across visual understanding, text reading, and science reasoning, the method improves a two-billion-parameter model over unfiltered training on all three skills. The rater signals prove nearly independent, confirming that the split captures genuinely separate dimensions of quality.
Catalog record
| CREATOR | Naveen Sahi, Jeremy Dohmann, Armen Aghajanyan, Akshat Shrivastava |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-11615