Enhancing Vision-Language Model Training with Reinforcement Learning in Synthetic Worlds for Real-World Success
George Bredis, Stanislav Dereka, Viacheslav Sinii · 2025
Read Enhancing Vision-Language Model Training with Reinforcement Learning in Synthetic Worlds for Real-World Success by George Bredis, Stanislav Dereka, Viacheslav Sinii (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-04280.txt.
Description
George Bredis, Stanislav Dereka, Viacheslav Sinii, and co-authors train vision-language models in cheap virtual worlds so the skills carry to real tasks. Models that see and read still struggle to turn what they see into a sound chain of actions. Prior training in simulators seldom proved that the skill would transfer.
Their method updates action words with a standard policy method while learning value only at the level of whole steps. Training in one simple simulator at a time led to gains on game control, spatial planning, and web tasks with real images. General image understanding stayed intact, which supports training in synthetic worlds.
Catalog record
| CREATOR | George Bredis, Stanislav Dereka, Viacheslav Sinii |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-04280