Learning Self-Correction in Vision–Language Models via Rollout Augmentation
Yi Ding, Ziliang Qiu, Bolian Li, Ruqi Zhang · 2026
Read Learning Self-Correction in Vision–Language Models via Rollout Augmentation by Yi Ding, Ziliang Qiu, Bolian Li, Ruqi Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08503.txt.
Description
Vision-language models rarely learn to correct themselves, because genuine self-correction shows up too seldom to teach from. Yi Ding and three co-authors propose Octopus, a rollout-augmentation method that makes many self-fix examples by mixing old rollouts in new ways. Their arXiv paper targets the sparse-signal problem in reinforcement learning.
Reuse boosts sample efficiency while balanced supervision steadies optimization. A response-masking trick separates self-correction from direct reasoning so the two skills do not fight each other. The resulting Octopus-8B model leads open-source vision-language models across seven benchmarks, beating the best prior baseline by a point while using only 0.72 times the training time per step. Readers get a controllable self-correction recipe.
Catalog record
| CREATOR | Yi Ding, Ziliang Qiu, Bolian Li, Ruqi Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-08503