LaGEA : La nguage G uided E mbodied A gents for Robotic Manipulation

Abdul Monaf Chowdhury, Akm Moshiur Rahman Mazumder, Rabeya Akter, Safaeid Hossain Arib · 2025

Read LaGEA : La nguage G uided E mbodied A gents for Robotic Manipulation by Abdul Monaf Chowdhury, Akm Moshiur Rahman Mazumder, Rabeya Akter, Safaeid Hossain Arib (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23155.txt.

Introduction

Language feedback could teach robot arms from their own failed tries, if it is timed and grounded. Abdul Monaf Chowdhury and 3 co-authors built LaGEA on that idea. A vision-language model reflects on each attempt under a fixed schema. The reflection names what went wrong in concise words. Decisive moments in the trajectory get located, and feedback is tied to visual state.

Progress and feedback agreement become bounded step rewards. A failure-aware coefficient fades the guidance as skill grows. On Meta-World and Fetch tasks, success beat top prior methods. Gains were largest on fetch tasks, and learning converged faster.

Catalog record

CREATORAbdul Monaf Chowdhury, Akm Moshiur Rahman Mazumder, Rabeya Akter, Safaeid Hossain Arib
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-23155

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LaGEA : La nguage G uided E mbodied A gents for Robotic Manipulation Abdul Monaf Chowdhury, Akm Moshiur Rahman Mazumder, Rabeya Akter, Safaeid Hossain Arib Abstract Robotic manipulation benefits from foundation models that describe goals, but today’s agents still lack a principled way to learn from their own mistakes. We ask whether natural language can serve as feedback, an error-reasoning signal that helps embodied agents diagnose what went wrong and correct course. We introduce LaGEA ( La nguage G uided E mbodied A gents), a framework that turns episodic, schema-constrained reflections from a vision language model (VLM) into temporally grounded guidance for reinforcement learning. LaGEA summarizes each attempt in concise language, localizes the decisive moments in the trajectory, aligns feedback with visual state in a shared representation, and converts goal progress and feedback agreement into bounded, step-wise shaping rewards whose influence is modulated by an adaptive, failure-aware coefficient. This design yields dense signals early when exploration needs direction and gracefully recedes as competence grows. On the Meta-World MT10 embodied manipulation benchmark, LaGEA improves average success over the state-of-the-art (SOTA) methods by 9.0% on random goals and 5.3% on fixed goals, while converging faster. These results support our hypothesis: language, when structured and grounded in time, is an effective mechanism for teaching robots to self-reflect on mistakes and make better choices. Code will be released soon. 1 Introduction Multimodal foundation models have reshaped sequential decision-making ( Yang et al. 2023 ) , from language-grounded affordance reasoning ( Ahn et al. 2022 ) to vision–language–action transfer, robots now display compelling zero-shot behaviour and semantic competence ( Driess et al. 2023 ; Kim et al. 2024 ; Brohan et al. 2024 ) . Yet converting such priors into reliable learning signals still hinges on reward design, which remains a bottleneck across tasks and scenes. To reduce engineering overhead, a pragmatic trend is to treat VLMs as zero-shot reward models ( Rocamonde et al. 2023 ) , scoring progress from natural-language goals and visual observations ( Baumli et al. 2023 ) . Yet these scores usually summarize overall outcomes rather than provide step-wise credit, can fluctuate with viewpoint and context, and inherit biases and inconsistency ( Wang et al. 2022 ; Li et al. 2024 ) . Densifying VLM-derived rewards into per-step signals helps but does not remove hallucination or noise-induced drift. Simply adding these signals can destabilize training or encourage reward hacking. Contrastive objectives like FuRL ( Fu et al. 2024 ) reduce reward misalignment, but on long-horizon, sparse-reward tasks, early misalignment can compound, misdirecting exploration. This highlights the need for structured, temporally grounded guidance that reduces noise and helps the agent recognize and learn from its own failures. Agents need to recognize what went wrong, when it happened, and why it matters for the next decision. General-purpose VLMs, while capable at instruction-following, are not calibrated for this role, as they can hallucinate or rationalize errors under small distribution shifts ( Lin et al. 2021 ) . Prior self-reflection paradigms ( Shinn et al. 2023 ) show that textual self-critique can improve decision making, but these demonstrations largely live in text-only environments such as ALFWorld ( Shridhar et al. 2020 ) , where observation, action, and feedback share a symbolic interface. Learning from failure is a fundamental aspect of reasoning; therefore, we ask a critical question: How can embodied policies derive reliable, temporally localized failure attributions directly from visual trajectories of the stochastic robotic environments where explorations are expensive? Learning from mistakes requires detecting failures and causal understanding. For this purpose, we present our framework LaGEA , which addresses this by using VLMs to generate episodic natural-language reflections on a robot’s behavior, summarizing what was attempted, which constraints were violated, and providing actionable rationales. As smaller VLMs can hallucinate or drift in free-form text ( Guan et al. 2024 ; Chen et al. 2024 ) , feedback is structured and aligned with goal and instruction texts, making LAGEA transferable across agents, viewpoints, and environments while maintaining stability. With these structured reflections in hand, we turn feedback into a signal the agent can actually use at each step rather than as a single episode score. LaGEA maps the feedback into the agent’s visual representation and attaches a local progress signal to each transition. We adopt potential-based reward shaping, adding only the change in this signal from successive states, which avoids over-rewarding static states ( Wiewiora 2003 ) . The potential itself blends two agreements: how well the current state matches the instruction-defined goal, and how well the transition aligns with the VLM’s diagnosis around the key frames, so progress is rewarded precisely where the diagnosis says it matters. To keep learning stable, we dynamically modulate its scale against the environment task reward and feed the overall reward to the critic of our online RL algorithm ( Haarnoja et al. 2018 ) . We evaluate LaGEA on diverse robotic manipulation tasks ( Yu et al. 2020 ) . LaGEA transforms VLM critique into localized, action-grounded shaping, obtains faster convergence and higher success rates over strong off-policy baselines. Our core contributions are: • We present LaGEA , an embodied VLM-RL framework that generates causal episodic feedback which are localized in time to turn failures into guidance and improve recovery after near misses. • We demonstrate that LaGEA can convert episodic, natural language self-reflection into a dense reward shaping signal through feedback alignment and feedback-VLM delta reward potential that can solve complex, sparse reward robot manipulation tasks. • We provide extensive experimental analysis of LaGEA on state-of-the-art (SOTA) robotic manipulation benchmarks ( Yu et al. 2020 ) and present insights into LaGEA ’s learning procedure via thorough ablation studies. 2 Related Work VLMs for RL. Foundation models ( Wiggins & Tejani 2022 ) have proven broadly useful across downstream applications ( Khandelwal et al. 2022 ; Chowdhury et al. 2025 ) , motivating their incorporation into reinforcement learning pipelines. Early work showed that language models can act as reward generators in purely textual settings ( Kwon et al. 2023 ) , but extending this idea to visuomotor control is nontrivial because reward specification is often ambiguous or brittle. A natural remedy is to leverage visual reasoning to infer progress toward a goal directly from observations ( Adeniji et al. 2023 ) . One approach ( Wang et al. 2024 ) queries a VLM to compare state images and judge improvement along a task trajectory; another aligns trajectory frames with language descriptions or demonstration captions and uses the resulting similarities as dense rewards ( Fu et al. 2024 ) . However, empirical studies indicate that such contrastive alignment introduces noise, and its reliability depends strongly on how the task is specified in language ( Nam et al. 2023 ) . Natural Language in Embodied AI. With VLM architectures pushing this multimodal interface forward ( Liu et al. 2023 ; Karamcheti et al. 2024 ) , a growing body of work integrates visual and linguistic inputs directly into large language models to drive embodied behavior, spanning navigation ( Majumdar et al. 2020 ) , manipulation ( Lynch & Sermanet 2020b ) , and mixed settings ( Suglia et al. 2021 ) . Beyond end-to-end conditioning, many systems focus on

Read the complete text