Trust, Don’t Trust, or Flip: Robust Preference-Based Reinforcement Learning with Multi-Expert Feedback

Seyed Amir Hosseini, Maryam Abdolali, Amirhosein Tavakkoli, Fardin Ayar, Ehsan Javanmardi, Manabu Tsukada, Mahdi Javanmardi · 2026

Read Trust, Don’t Trust, or Flip: Robust Preference-Based Reinforcement Learning with Multi-Expert Feedback by Seyed Amir Hosseini, Maryam Abdolali, Amirhosein Tavakkoli, Fardin Ayar, Ehsan Javanmardi, Manabu Tsukada, Mahdi Javanmardi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18751.txt.

Description

Seyed Amir Hosseini and six colleagues tackle a flaw in a way to teach agents from side-by-side picks. In that path, a judge says which of two runs was best, and the agent learns from the pick. Real judges are a mixed lot. Some are sharp, some are sloppy, and some pick the worst run on aim. Past tools treat all picks the same, or try to drop the bad judges out.

Both paths fail when a judge is sly: their picks look like help but push the agent wrong. The team build a tool that can trust, doubt, or flip each judge's picks as it learns. A flipped pick from a sly judge turns from harm to help. Such care could make pick-based training safe in the wild, where judges can't all be vetted.

Catalog record

CREATORSeyed Amir Hosseini, Maryam Abdolali, Amirhosein Tavakkoli, Fardin Ayar, Ehsan Javanmardi, Manabu Tsukada, Mahdi Javanmardi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-18751

Browse the full library · About the Tower of Babel