Skill-Aware Data Selection and Fine-Tuning for Data-Efficient Reasoning Distillation

Lechen Zhang Yunxiang Zhang Wei Hu Lu Wang · 2026

Read Skill-Aware Data Selection and Fine-Tuning for Data-Efficient Reasoning Distillation by Lechen Zhang Yunxiang Zhang Wei Hu Lu Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-10109.txt.

Description

Lechen Zhang, Yunxiang Zhang, Wei Hu, and Lu Wang cut the data cost of teaching small models to reason. Large models can solve hard tasks step by step. Small models learn that skill from the large ones. But that teaching tends to need vast data sets. The cost in time and funds is high.

The team picks data by skill. They ask which skills a task needs. They then pick the cases that teach those skills best. With less data, fine tune runs can still yield strong skill. The work aims to make the teaching cheap and sharp at once.

Catalog record

CREATORLechen Zhang Yunxiang Zhang Wei Hu Lu Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-10109

Browse the full library · About the Tower of Babel