Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences

Zhuoran Jin, Hongbang Yuan, Kejian Zhu, Jiachun Li, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao · 2025

Read Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences by Zhuoran Jin, Hongbang Yuan, Kejian Zhu, Jiachun Li, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-23451.txt.

Description

Zhuoran Jin and seven co-authors propose Omni-Reward, a generalist reward model that works across many modalities with free-form preferences. Reward models align AI behavior with human preferences, yet most cover only text and images, leaving video and audio weakly supported. Training on fixed binary preference pairs also misses the diversity of personal preferences.

Omni-Reward is built to accept free-form preference descriptions and to score content across modalities in one model. A generalist reward model could steer assistants that see, hear, and watch, aligning them to richer human judgments than a single thumbs-up signal.

Catalog record

CREATORZhuoran Jin, Hongbang Yuan, Kejian Zhu, Jiachun Li, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE86 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-23451

Browse the full library · About the Tower of Babel