Not All Negative Samples Are Equal: LLMs Learn Better from Plausible Reasoning

Zixiang Di, Jinyi Han, Shuo Zhang, Ying Liao, Zhi Li, Xiaofeng Ji, Yongqi Wang, Zheming Yang, Ming Gao, Bingdong Li †, Jie Wang † · 2026

Read Not All Negative Samples Are Equal: LLMs Learn Better from Plausible Reasoning by Zixiang Di, Jinyi Han, Shuo Zhang, Ying Liao, Zhi Li, Xiaofeng Ji, Yongqi Wang, Zheming Yang, Ming Gao, Bingdong Li †, Jie Wang † (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-03516.txt.

Description

A wrong answer can still teach a model something, but not all wrong answers teach equally well. Zixiang Di and 10 co-authors argue in a 2026 paper that plausible mistakes are the valuable ones. Their method, Plausible Negative Samples, synthesizes incorrect responses with a good solution's format and structure. The answers still land on the wrong result.

A dedicated generator is trained with reverse reinforcement learning and a composite reward covering format, inverted accuracy, and reasoning quality. The samples look nearly indistinguishable from correct solutions. Used as plug-and-play data for preference optimization, they lift three backbone models across seven mathematics benchmarks. They beat other negative-sample methods and improve on reinforcement-trained models by about two points on average.

Catalog record

CREATORZixiang Di, Jinyi Han, Shuo Zhang, Ying Liao, Zhi Li, Xiaofeng Ji, Yongqi Wang, Zheming Yang, Ming Gao, Bingdong Li †, Jie Wang †
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-03516

Browse the full library · About the Tower of Babel