Inducing Robustness in a 2-dimensional Direct Preference Optimisation Paradigm
Sarvesh Shashidhar, Ritik, Nachiketa Patil, Suraj Racha, Ganesh Ramakrishnan · 2025
Read Inducing Robustness in a 2-dimensional Direct Preference Optimisation Paradigm by Sarvesh Shashidhar, Ritik, Nachiketa Patil, Suraj Racha, Ganesh Ramakrishnan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01706.txt.
Description
Sarvesh Shashidhar and co-authors probe a training method for large language models in a 2025 paper. Direct preference methods teach a model from ranked answers, but they score each whole answer at once. Real readers judge parts of an answer, since even a good reply can hold weak segments. A two dimensional scoring version was built to fix that gap. The team tests how it holds up when scores carry noise.
Their study compares win rates for the standard method and the two dimensional one on open data sets. Both work well on clean scores yet falter when labels or scores are noisy. The authors add a fix that makes the segment scores resist noise, backed by theory and lab tests. They also lay out other noise patterns that could appear in real scoring work.
Catalog record
| CREATOR | Sarvesh Shashidhar, Ritik, Nachiketa Patil, Suraj Racha, Ganesh Ramakrishnan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-01706