MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection

Haowen Wang, Yaxin Du, Jian Yang, Jiajun Wu, Shukai Liu, Yuxuan Zhang, Pingjie Wang, Siheng Chen, Tuney Zheng, Ming Zhou, Xianglong Liu, Bryan Dai, [0.5em] Beihang University, IQuest Research · 2026

Read MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection by Haowen Wang, Yaxin Du, Jian Yang, Jiajun Wu, Shukai Liu, Yuxuan Zhang, Pingjie Wang, Siheng Chen, Tuney Zheng, Ming Zhou, Xianglong Liu, Bryan Dai, [0.5em] Beihang University, IQuest Research (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30288.txt.

Description

Haowen Wang and 11 co-authors propose MIRA, a filter for mid-training data selection that discovers rubrics per source. Mid-training uses large curated mixtures to strengthen capabilities before final training. Effective selection requires scalability and source-adaptive criteria. MIRA makes rubric construction part of selection: it discovers what to evaluate per group, then distills those judgments into student scorers for filtering the full corpus. On code mid-training with 21 sources and 5 groups, MIRA beats baselines across nine code benchmarks and equals the full run on only half the tokens.

Catalog record

CREATORHaowen Wang, Yaxin Du, Jian Yang, Jiajun Wu, Shukai Liu, Yuxuan Zhang, Pingjie Wang, Siheng Chen, Tuney Zheng, Ming Zhou, Xianglong Liu, Bryan Dai, [0.5em] Beihang University, IQuest Research
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65404
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30288

Browse the full library · About the Tower of Babel