Learning Self-Correction in Vision–Language Models via Rollout Augmentation

Yi Ding, Ziliang Qiu, Bolian Li, Ruqi Zhang · 2026

Read Learning Self-Correction in Vision–Language Models via Rollout Augmentation by Yi Ding, Ziliang Qiu, Bolian Li, Ruqi Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08503.txt.

Description

Vision-language models rarely learn to correct themselves, because genuine self-correction shows up too seldom to teach from. Yi Ding and three co-authors propose Octopus, a rollout-augmentation method that makes many self-fix examples by mixing old rollouts in new ways. Their arXiv paper targets the sparse-signal problem in reinforcement learning.

Reuse boosts sample efficiency while balanced supervision steadies optimization. A response-masking trick separates self-correction from direct reasoning so the two skills do not fight each other. The resulting Octopus-8B model leads open-source vision-language models across seven benchmarks, beating the best prior baseline by a point while using only 0.72 times the training time per step. Readers get a controllable self-correction recipe.

Catalog record

CREATORYi Ding, Ziliang Qiu, Bolian Li, Ruqi Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-08503

Browse the full library · About the Tower of Babel