SkillRater: Untangling Capabilities in Multimodal Data

Naveen Sahi, Jeremy Dohmann, Armen Aghajanyan, Akshat Shrivastava · 2026

Read SkillRater: Untangling Capabilities in Multimodal Data by Naveen Sahi, Jeremy Dohmann, Armen Aghajanyan, Akshat Shrivastava (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11615.txt.

Description

Training data are usually scored with one number for quality, but a model that must learn several skills at once cannot be served well by a single score. Naveen Sahi and three co-authors argue quality is better treated as multidimensional, with one dimension per capability. Their framework, SkillRater, trains a separate rater for each skill by meta-learning on its own validation goal, then combines the raters with a progressive rule.

At each training stage a sample survives if any rater ranks it above a bar that tightens over time, keeping variety early and focusing on high-value samples late. Tested on vision-language models across visual understanding, text reading, and science reasoning, the method improves a two-billion-parameter model over unfiltered training on all three skills. The rater signals prove nearly independent, confirming that the split captures genuinely separate dimensions of quality.

Catalog record

CREATORNaveen Sahi, Jeremy Dohmann, Armen Aghajanyan, Akshat Shrivastava
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-11615

Browse the full library · About the Tower of Babel