PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraft

Yuchen Guo, Junli Gong, Weicheng Wang, Hongmin Cai, Yiu-ming Cheung, Weifeng Su · 2026

Read PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraft by Yuchen Guo, Junli Gong, Weicheng Wang, Hongmin Cai, Yiu-ming Cheung, Weifeng Su (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27762.txt.

Introduction

Yuchen Guo and colleagues present PEAM, a parametric embodied agent memory for Minecraft that stores past skills as model weights. A slow model handles open-ended thought while a fast module runs stored skills, and one small add-on per skill type keeps old skills from being lost. Treating failure as a training signal, failure-and-fix pairs teach the agent both what succeeds and how fixed actions differ from failed ones.

Catalog record

CREATORYuchen Guo, Junli Gong, Weicheng Wang, Hongmin Cai, Yiu-ming Cheung, Weifeng Su
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44635
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.27762

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraft Yuchen Guo, Junli Gong, Weicheng Wang, Hongmin Cai, Yiu-ming Cheung, Weifeng Su Abstract We present PEAM , a P arametric E mbodied A gent M emory framework in Minecraft that transforms agent memory from inference-time retrieval into parameter-resident skills internalized through experience. PEAM pairs a slow deliberative LLM for open-ended reasoning with a fast parametric module for reflexive execution of consolidated skills. The fast module is a multimodal Mixture-of-Experts LoRA architecture with per-category physically isolated adapters, enabling parameter-level continual learning without catastrophic forgetting. We treat failure as a first-class training signal: failure–correction trajectory pairs are internalized through a joint behavioral-cloning and contrastive objective, so the agent learns not only what succeeds but also how corrected actions differ from failed ones. To govern consolidation, PEAM introduces a parameterization-worthiness score for deciding which experience should be internalized, and a scale-free self-triggered consolidation mechanism for deciding when to internalize without task-specific hand-tuned thresholds, making the agent self-evolving as the trigger transfers across task distributions without re-tuning. Experiments in Minecraft show that PEAM improves long-horizon task performance, mitigates forgetting on previously consolidated skills, and improves parametric-versus-retrieval efficiency over retrieval-based embodied agents and parametric memory variants. Figure 1: PEAM turns episodic agent memory into parametric embodied skills. A slow deliberative LLM explores, verifies, and produces corrected trajectories, which are staged in episodic memory. PEAM then decides what to internalize using parameterization-worthiness (PV), when to consolidate using self-triggered consolidation (STC), and how to update parameters through isolated MoE-LoRA adapters trained on success and failure–correction pairs. 1 Introduction Current LLM-based embodied agents typically rely on memory that is non-parametric: past trajectories, reflections, and skills are stored externally and re-injected at inference time, while the agent’s parametric policy ( e.g. , the parameters of the backbone model or trainable policy module) remains unchanged across tasks 9 ; 30 ; 4 . Minecraft is a suitable environment for evaluating embodied agent performance, which requires players to explore vast, procedurally generated 3D terrains and unlock a tech tree using gathered resources 24 . But for current agents, after repeated attempts at the same craft chain, the policy may remain unchanged even if an external skill library has grown 12 ; 25 . This design has practical costs as deployment continues. Every recall consumes context budget because past trajectories must be re-injected into the prompt; retrieval and prompt construction add latency to each decision cycle; and experience that remains external must be reintroduced whenever the agent needs to use it. We view this as a missing consolidation pathway rather than a failure of retrieval-augmented memory itself: external memory supports recall, but it does not by itself specify how selected experience becomes part of the agent’s parametric competence. A long-standing view in cognitive neuroscience holds that durable memory arises from two complementary systems: a fast, sparse episodic store that encodes new experience and a slow, distributed parametric store that integrates stable structure over time 18 . These systems are coupled by offline consolidation, classically associated with sleep, in which episodic traces are replayed and gradually written into distributed representations 18 ; 11 . Related cultivate-then-consolidate patterns also appear in recent LLM-scale systems, such as DeepSeek-V4, which cultivates domain-specific experts through independent training and then consolidates them into a unified model via distillation 3 . Across these settings, durable competence separates the acquisition of new experience from its integration into long-term parameters. Existing embodied-agent memory systems instantiate the acquisition side through skill libraries, reflection logs, and retrieval-augmented contexts 23 ; 24 ; 25 ; 12 ; 31 . PEAM addresses the consolidation side for embodied agents by deciding which accumulated traces should become parametric competence, and when. Rather than replaying traces into a shared substrate or distilling specialists into a single model, PEAM consolidates experience into per-category adapters, using parameter isolation to reduce cross-category forgetting. PEAM operationalizes this principle as a two-tier embodied agent. A slow deliberative LLM handles open-ended reasoning, curriculum proposal, code synthesis, and outcome verification. An external episodic store stages successful and corrected trajectories, while a fast parametric module executes consolidated skills through a multimodal Mixture-of-Experts LoRA architecture 20 ; 5 . The tiers communicate through a consolidation pipeline that decides which episodic traces should be internalized into parametric adapters and when consolidation should occur. PEAM makes three design choices. First, failure is treated as a training signal: rather than converting failed trajectories only into textual guidance for later prompts, PEAM trains on failure-correction trajectory pairs through a joint behavioral-cloning and contrastive objective 19 . Second, consolidation occurs into per-category isolated adapters, so internalizing a craft skill does not update the parameters used for a combat skill; forgetting resistance is supported by architecture rather than only by regularization 10 ; 21 ; 17 ; 16 . Third, PEAM formalizes the questions of what and when to consolidate: a parameterization-worthiness score ranks candidate experience along cost, stability, redundancy, and interference dimensions, and a self-triggered consolidation mechanism decides when to internalize based on the agent’s failure statistics rather than a task-specific hand-tuned schedule. Together, these mechanisms provide a pathway by which selected experience can move from external recall into the agent’s trainable parameters. We instantiate PEAM in Minecraft, where long-horizon embodied tasks exercise skill reuse, correction, and consolidation, and evaluate against retrieval-based embodied agents and parametric memory variants on task success, forgetting, inference efficiency, and cross-distribution stability of the consolidation trigger 24 . In addition to the main comparison, our experiments report methodology findings relevant to agent evaluation: forward-pass preference margins can fail to predict generate-path deployability, quantized on-device agent serving introduces deployment-specific failure modes, and trajectory re-slicing can provide a controlled substitute for cross-distribution trigger evaluation. The remainder of the paper details the method, experiments, and limitations. 2 Related Work Retrieval-based memory in embodied agents. Perception 8 , decision-making 22 , and memory 4 are integral components of world modeling for embodied agent 14 ; 6 . A dominant design in LLM agents treats memory as a non-parametric store: past trajectories, reflections, and skills are written externally and retrieved into the context at inference time 4 ; 9 ; 30 ( e.g. , Retrieval-Augmented Generation (RAG) 7 ). ReAct established the reasoning-acting interface adopted by many later agents 28 , while Reflexion stores failure feedback as natural-language reflections for subsequent attempts 23 . In embodied domains, recent systems extend this pattern with structured spatial, semantic, and multimodal memories: Embodied-RAG builds hierarchical non-parametric memory for embodied retrieval and generation 27 , while open-world Minecraft agents such as VOYAGER 24 , JARVIS-1 25 , Optimus-1 12 , and GITM 31 maintain external skill, trajectory, or collaboration memories for long-horizon behavior. PEAM differs from this family architecturally: retrieved memory remains in prompt space, whereas PEAM consolidates selected experience into parameters. We adopt VOYAGER’s Minecraft execution framework ( e.g. , its Mineflayer-based bot interface and code-as-action pipeline) as a shared testbed, holding the action interface fixed while changing the memory architecture. PEAM also differs from Reflexion in how it uses failure: Reflexion converts failures into textual guidance for future prompts, whereas PEAM trains on failure-correction pairs directly, making corrected behavior available through the parametric policy rather than through retrieval. Parametric memory and continual

Read the complete text