Not All Preferences Are Created Equal: Stability-Aware and Gradient-Efficient Alignment for Reasoning Models

Hui Wu, Hengyi Cai, Jinman Zhao, Xinran Chen, Ziheng Li, Zhejun Zhao, Shuaiqiang Wang, Yuchen Li, Dawei Yin · 2026

Read Not All Preferences Are Created Equal: Stability-Aware and Gradient-Efficient Alignment for Reasoning Models by Hui Wu, Hengyi Cai, Jinman Zhao, Xinran Chen, Ziheng Li, Zhejun Zhao, Shuaiqiang Wang, Yuchen Li, Dawei Yin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01207.txt.

Description

Hui Wu and co-authors study which like-pairs deserve slopes in offline tuning. Old plans slope on every picked-versus-spurned pair alike, wasting work. A pair's worth hinges on news value and steadiness, drifts as the model shifts, and can turn noisy where slopes meet sharp curves. The best teaching comes from steady sure slips, where the model errs with trust yet curves stay flat. Their SAGE plan pools pairs by hardness and picks by signal-to-curve scores, beating full-data baselines with smoother trains.

Catalog record

CREATORHui Wu, Hengyi Cai, Jinman Zhao, Xinran Chen, Ziheng Li, Zhejun Zhao, Shuaiqiang Wang, Yuchen Li, Dawei Yin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40537
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.01207

Browse the full library · About the Tower of Babel