TUR-DPO: Topology- and Uncertainty-Aware Direct Preference Optimization

Abdulhady Abas, Fatemeh Daneshfar, Seyedali Mirjalili, Mourad Oussalah · 2026

Read TUR-DPO: Topology- and Uncertainty-Aware Direct Preference Optimization by Abdulhady Abas, Fatemeh Daneshfar, Seyedali Mirjalili, Mourad Oussalah (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-00224.txt.

Description

Abdulhady Abas Abdullah and 3 co-authors refine Direct Preference Optimization, a popular method for aligning language models with human choices. The standard method treats each preference as a flat win or loss. It also suffers when preferences are noisy, or when a fragile chain of thought produced the losing answer.

Their variant, TUR-DPO, also rewards how an answer is derived, not only what it says. It draws out light reasoning maps and mixes faithfulness, utility, and map quality into one uncertainty signal. That signal weights the training objective, with no reinforcement learning needed. On open models across math, factual questions, summaries, and dialogue, TUR-DPO beat the standard method on win rates, faithfulness, and calibration, and matched a heavier rival on reasoning tasks.

Catalog record

CREATORAbdulhady Abas, Fatemeh Daneshfar, Seyedali Mirjalili, Mourad Oussalah
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.00224

Browse the full library · About the Tower of Babel