Laser : Stratified Selective Sampling for Instruction Tuning with Dedicated Scoring Strategy

Paramita Mirza *, Lucas Weber *, Fabian Küch · 2025

Read Laser : Stratified Selective Sampling for Instruction Tuning with Dedicated Scoring Strategy by Paramita Mirza *, Lucas Weber *, Fabian Küch (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22157.txt.

Description

Paramita Mirza, Lucas Weber, and Fabian Küch show that a small, well-chosen slice of training data can tune a language model as well as the whole pile. Recent work found that post-training sets shrink well without hurting quality, but picking the slice is often costly or narrow. Their 2025 paper builds a selection pipeline meant to be both cheap and general.

The pipeline first sorts examples into task-based bins, so the final mix can be steered for a general-purpose model. Specialist scorer models then grade quality, and a light, sturdy method grades how hard each example is. A clustering step over embeddings guards variety, improving on earlier diversity methods. The authors argue the combined strategy gives strong fine-tuning with little overhead. Their approach lets smaller teams afford careful data curation instead of brute-force training on everything at hand.

Catalog record

CREATORParamita Mirza *, Lucas Weber *, Fabian Küch
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-22157

Browse the full library · About the Tower of Babel