Laser : Stratified Selective Sampling for Instruction Tuning with Dedicated Scoring Strategy
Paramita Mirza *, Lucas Weber *, Fabian Küch · 2025
Read Laser : Stratified Selective Sampling for Instruction Tuning with Dedicated Scoring Strategy by Paramita Mirza *, Lucas Weber *, Fabian Küch (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22157.txt.
Description
Paramita Mirza, Lucas Weber, and Fabian Küch show that a small, well-chosen slice of training data can tune a language model as well as the whole pile. Recent work found that post-training sets shrink well without hurting quality, but picking the slice is often costly or narrow. Their 2025 paper builds a selection pipeline meant to be both cheap and general.
The pipeline first sorts examples into task-based bins, so the final mix can be steered for a general-purpose model. Specialist scorer models then grade quality, and a light, sturdy method grades how hard each example is. A clustering step over embeddings guards variety, improving on earlier diversity methods. The authors argue the combined strategy gives strong fine-tuning with little overhead. Their approach lets smaller teams afford careful data curation instead of brute-force training on everything at hand.
Catalog record
| CREATOR | Paramita Mirza *, Lucas Weber *, Fabian Küch |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-22157