Cream of the Crop: Harvesting Rich, Scalable and Transferable Multi-Modal Data for Instruction Fine-Tuning

Mengyao Lyu, Yan Li, Huasong Zhong, Wenhao Yang, Hui Chen, Jungong Han, Guiguang Ding, Zhenheng Yang · 2025

Read Cream of the Crop: Harvesting Rich, Scalable and Transferable Multi-Modal Data for Instruction Fine-Tuning by Mengyao Lyu, Yan Li, Huasong Zhong, Wenhao Yang, Hui Chen, Jungong Han, Guiguang Ding, Zhenheng Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13383.txt.

Description

Lyu and colleagues present in twenty twenty-five a simple way to pick the best training examples for teaching vision-language models, even from pools of millions. Picking data usually needs costly deep features or narrow quality scores, and often fails to beat random choice. Their method profiles each example by the interaction style it shows and by scores across many separate skills, making a rich but explicit card.

Choosing examples then becomes a coverage puzzle over the skill-style grid, with a proven approximation guarantee. Across two model families, two giant pools, and many test suites, the picks won consistently. Using under a third of one pool kept nearly all of full-data performance. The profiles also transferred to new pools and models for free.

Catalog record

CREATORMengyao Lyu, Yan Li, Huasong Zhong, Wenhao Yang, Hui Chen, Jungong Han, Guiguang Ding, Zhenheng Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-13383

Browse the full library · About the Tower of Babel