TUR-DPO: Topology- and Uncertainty-Aware Direct Preference Optimization
Abdulhady Abas, Fatemeh Daneshfar, Seyedali Mirjalili, Mourad Oussalah · 2026
Read TUR-DPO: Topology- and Uncertainty-Aware Direct Preference Optimization by Abdulhady Abas, Fatemeh Daneshfar, Seyedali Mirjalili, Mourad Oussalah (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-00224.txt.
Description
Abdulhady Abas Abdullah and 3 co-authors refine Direct Preference Optimization, a popular method for aligning language models with human choices. The standard method treats each preference as a flat win or loss. It also suffers when preferences are noisy, or when a fragile chain of thought produced the losing answer.
Their variant, TUR-DPO, also rewards how an answer is derived, not only what it says. It draws out light reasoning maps and mixes faithfulness, utility, and map quality into one uncertainty signal. That signal weights the training objective, with no reinforcement learning needed. On open models across math, factual questions, summaries, and dialogue, TUR-DPO beat the standard method on win rates, faithfulness, and calibration, and matched a heavier rival on reasoning tasks.
Catalog record
| CREATOR | Abdulhady Abas, Fatemeh Daneshfar, Seyedali Mirjalili, Mourad Oussalah |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.00224