Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences
Zhuoran Jin, Hongbang Yuan, Kejian Zhu, Jiachun Li, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao · 2025
Read Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences by Zhuoran Jin, Hongbang Yuan, Kejian Zhu, Jiachun Li, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-23451.txt.
Description
Zhuoran Jin and seven co-authors propose Omni-Reward, a generalist reward model that works across many modalities with free-form preferences. Reward models align AI behavior with human preferences, yet most cover only text and images, leaving video and audio weakly supported. Training on fixed binary preference pairs also misses the diversity of personal preferences.
Omni-Reward is built to accept free-form preference descriptions and to score content across modalities in one model. A generalist reward model could steer assistants that see, hear, and watch, aligning them to richer human judgments than a single thumbs-up signal.
Catalog record
| CREATOR | Zhuoran Jin, Hongbang Yuan, Kejian Zhu, Jiachun Li, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 86 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-23451