Test-Time Reasoning Through Visual Human Preferences with VLMs and Soft Rewards

Alexander Gambashidze, Konstantin Sobolev, Andrey Kuznetsov, Ivan Oseledets · 2025

Read Test-Time Reasoning Through Visual Human Preferences with VLMs and Soft Rewards by Alexander Gambashidze, Konstantin Sobolev, Andrey Kuznetsov, Ivan Oseledets (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-19948.txt.

Description

Alexander Gambashidze and co-authors teach sight-and-word models to judge which image a person would like, in a 2025 paper. The ask is whether such a model can think its way to a human taste call at the time of the test.

The team trained the models with reward-style learning on two sets of human picks. The trained models called the liked image right about two times in three, on a par with tools built just for that task. Their edge is that they show their work, in plain steps a person can read. Light reward schemes then let the models rank any images, of any shape or load. The authors say the skill can sharpen tools that turn words into images, with less hand-marked data.

Catalog record

CREATORAlexander Gambashidze, Konstantin Sobolev, Andrey Kuznetsov, Ivan Oseledets
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE13 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-19948

Browse the full library · About the Tower of Babel