Trust, Don’t Trust, or Flip: Robust Preference-Based Reinforcement Learning with Multi-Expert Feedback
Seyed Amir Hosseini, Maryam Abdolali, Amirhosein Tavakkoli, Fardin Ayar, Ehsan Javanmardi, Manabu Tsukada, Mahdi Javanmardi · 2026
Read Trust, Don’t Trust, or Flip: Robust Preference-Based Reinforcement Learning with Multi-Expert Feedback by Seyed Amir Hosseini, Maryam Abdolali, Amirhosein Tavakkoli, Fardin Ayar, Ehsan Javanmardi, Manabu Tsukada, Mahdi Javanmardi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18751.txt.
Description
Seyed Amir Hosseini and six colleagues tackle a flaw in a way to teach agents from side-by-side picks. In that path, a judge says which of two runs was best, and the agent learns from the pick. Real judges are a mixed lot. Some are sharp, some are sloppy, and some pick the worst run on aim. Past tools treat all picks the same, or try to drop the bad judges out.
Both paths fail when a judge is sly: their picks look like help but push the agent wrong. The team build a tool that can trust, doubt, or flip each judge's picks as it learns. A flipped pick from a sly judge turns from harm to help. Such care could make pick-based training safe in the wild, where judges can't all be vetted.
Catalog record
| CREATOR | Seyed Amir Hosseini, Maryam Abdolali, Amirhosein Tavakkoli, Fardin Ayar, Ehsan Javanmardi, Manabu Tsukada, Mahdi Javanmardi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-18751