Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigation

Pingrui Zhang, Yifei Su, Pengyuan Wu, Dong An, Li Zhang, Zhigang Wang, Dong Wang, Yan Ding, Bin Zhao, Xuelong Li · 2025

Read Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigation by Pingrui Zhang, Yifei Su, Pengyuan Wu, Dong An, Li Zhang, Zhigang Wang, Dong Wang, Yan Ding, Bin Zhao, Xuelong Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20897.txt.

Description

Pingrui Zhang and nine co-authors present a 2025 paper on guiding a robot by spoken directions, using imagined descriptions instead of imagined pictures. Navigation agents must follow instructions while seeing only part of a scene. Earlier systems imagined future views as images, which costs heavy computing and adds clutter. Their Adaptive Text Dreamer instead imagines key scene features in words, built on a large language model with two cooperating branches.

One branch handles logical integration, the other imaginative prediction, echoing the brain's two hemispheres. Only a small adapter in each branch is fine-tuned, waking the model's domain knowledge cheaply. A cross-interaction step polishes the imagined text and feeds it to a navigation expert. On the standard R2R benchmark the system reached state-of-the-art results with fewer parameters. The authors release their code publicly.

Catalog record

CREATORPingrui Zhang, Yifei Su, Pengyuan Wu, Dong An, Li Zhang, Zhigang Wang, Dong Wang, Yan Ding, Bin Zhao, Xuelong Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-20897

Browse the full library · About the Tower of Babel