MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection
Haowen Wang, Yaxin Du, Jian Yang, Jiajun Wu, Shukai Liu, Yuxuan Zhang, Pingjie Wang, Siheng Chen, Tuney Zheng, Ming Zhou, Xianglong Liu, Bryan Dai, [0.5em] Beihang University, IQuest Research · 2026
Read MIRA: Mid-training Rubric Anchoring for Source-Aware Data Selection by Haowen Wang, Yaxin Du, Jian Yang, Jiajun Wu, Shukai Liu, Yuxuan Zhang, Pingjie Wang, Siheng Chen, Tuney Zheng, Ming Zhou, Xianglong Liu, Bryan Dai, [0.5em] Beihang University, IQuest Research (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30288.txt.
Description
Haowen Wang and 11 co-authors propose MIRA, a filter for mid-training data selection that discovers rubrics per source. Mid-training uses large curated mixtures to strengthen capabilities before final training. Effective selection requires scalability and source-adaptive criteria. MIRA makes rubric construction part of selection: it discovers what to evaluate per group, then distills those judgments into student scorers for filtering the full corpus. On code mid-training with 21 sources and 5 groups, MIRA beats baselines across nine code benchmarks and equals the full run on only half the tokens.
Catalog record
| CREATOR | Haowen Wang, Yaxin Du, Jian Yang, Jiajun Wu, Shukai Liu, Yuxuan Zhang, Pingjie Wang, Siheng Chen, Tuney Zheng, Ming Zhou, Xianglong Liu, Bryan Dai, [0.5em] Beihang University, IQuest Research |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65404 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.30288