VLAs are Confined yet Capable of Generalizing to Novel Instructions

Quanyi Li · 2025

Read VLAs are Confined yet Capable of Generalizing to Novel Instructions by Quanyi Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-03500.txt.

Description

Quanyi Li tests whether vision-language-action robot models can recombine skills from separate tasks when given new instructions, in a 2025 paper. Such models often master each demonstrated task yet fail when asked to merge two of them, like placing an item somewhere it was only ever carried past. The fix works inside the model at run time, blending the hidden text states of two base tasks.

On a new benchmark of twenty recombined tasks, leading models scored under fifteen percent success, while the blended version of one model reached eighty-three percent. The study also shows the models tend to tie object names to memorized places, and that their internal prompts, though unreadable to humans, can steer or even hijack behavior.

Catalog record

CREATORQuanyi Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-03500

Browse the full library · About the Tower of Babel