Balanced Thinking: Improving Chain of Thought Training in Vision Language Models

Shaked Perek, Ben WieselAvihu DekelNimrod ShabtayEli SchwartzIBM Research [email protected] · 2026

Read Balanced Thinking: Improving Chain of Thought Training in Vision Language Models by Shaked Perek, Ben WieselAvihu DekelNimrod ShabtayEli SchwartzIBM Research [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-18656.txt.

Description

Perek and co-authors fix a training imbalance in vision-language model reasoning. Long think traces dwarf short answer spans in standard fine-tuning loss. Their SCALe method weights reasoning and answer parts separately. A cosine schedule shifts focus from thinking to answers as training proceeds. Loss stays independent of segment length. Across benchmarks, SCALe beats plain fine-tuning and matches a full fine-tune plus RL pipeline. It needs about one seventh of the training time. Paired with GRPO, it gives the best scores overall.

Catalog record

CREATORShaked Perek, Ben WieselAvihu DekelNimrod ShabtayEli SchwartzIBM Research [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41905
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.18656

Browse the full library · About the Tower of Babel