MrCoM: A Meta-Regularized World-Model Generalizing Across Multi-Scenarios
Xuantang Xiong, Ni Mu, Runpeng Xie, Senhao Yang, Yaqing Wang, Lexiang Wang, Yao Luan, Siyuan Li, Shuang Xu, Yiqin Yang, Bo Xu · 2025
Read MrCoM: A Meta-Regularized World-Model Generalizing Across Multi-Scenarios by Xuantang Xiong, Ni Mu, Runpeng Xie, Senhao Yang, Yaqing Wang, Lexiang Wang, Yao Luan, Siyuan Li, Shuang Xu, Yiqin Yang, Bo Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-06252.txt.
Introduction
Xuantang Xiong and co-authors build a world model that can generalize across many scenarios instead of just one task. Model-based reinforcement learning gains efficiency by predicting how the world responds, yet most world models are trained for a single setting. The authors note that scenarios inside one simulation engine share deep structural traits worth exploiting.
Their model, MrCoM, splits the latent state space into parts based on how the dynamics behave. Meta-state regularization pulls out a shared description of scenario information, while meta-value regularization keeps world-model training aligned with policy goals across settings. The authors bound the generalization error in theory and report clear wins over prior methods in varied scenarios.
Catalog record
| CREATOR | Xuantang Xiong, Ni Mu, Runpeng Xie, Senhao Yang, Yaqing Wang, Lexiang Wang, Yao Luan, Siyuan Li, Shuang Xu, Yiqin Yang, Bo Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-06252
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MrCoM: A Meta-Regularized World-Model Generalizing Across Multi-Scenarios Xuantang Xiong, Ni Mu, Runpeng Xie, Senhao Yang, Yaqing Wang, Lexiang Wang, Yao Luan, Siyuan Li, Shuang Xu, Yiqin Yang, Bo Xu Abstract Model-based reinforcement learning (MBRL) is a crucial approach to enhance the generalization capabilities and improve the sample efficiency of RL algorithms. However, current MBRL methods focus primarily on building world models for single tasks and rarely address generalization across different scenarios. Building on the insight that dynamics within the same simulation engine share inherent properties, we attempt to construct a unified world model capable of generalizing across different scenarios, named Meta-Regularized Contextual World-Model (MrCoM). This method first decomposes the latent state space into various components based on the dynamic characteristics, thereby enhancing the accuracy of world-model prediction. Further, MrCoM adopts meta-state regularization to extract unified representation of scenario-relevant information, and meta-value regularization to align world-model optimization with policy learning across diverse scenario objectives. We theoretically analyze the generalization error upper bound of MrCoM in multi-scenario settings. We systematically evaluate our algorithm’s generalization ability across diverse scenarios, demonstrating significantly better performance than previous state-of-the-art methods. 1 Introduction Reinforcement learning, a key algorithm for solving decision-making problems, has achieved significant success in domains such as games ( 20 ; 3 ) , robotic control ( 2 ) , autonomous driving ( 11 ) , and even large language model tuning ( 21 ) . One crucial requirement for broader RL applications is to improve the sample efficiency, and recent advances in model-based approaches have emerged as a promising solution by learning a world-model. Most model-based RL methods mainly focus on single-task settings, where a dedicated world model is trained for each individual task. Recent studies have demonstrated the robustness of world model hyperparameters across diverse scenarios, that is, maintaining identical training configurations to train world models under varying conditions. However, they seldom investigate cross-scenario generalization of the world models themselves. This critical gap prevents the deployment of a unified world model across multiple environments, which could substantially reduce training costs. Therefore, this naturally leads to the following question: How to learn a unified world-model capable of generalizing across multi-scenarios? In this work, we aim to provide an effective solution to construct the unified world-model. First, we decompose the latent state space into various components based on the dynamic characteristics, thereby enhancing the accuracy of world-model prediction. Further, we propose the meta-state regularization mechanism to extract unified representation of scenario-relevant information, and meta-value regularization to align world-model optimization with policy learning across diverse scenario objectives. We name our method as Meta-Regularized Contextual World-Model (MrCoM). We conduct the theoretical analysis for the model’s generalization capability and derive the upper bound of the generalization error of our method. For the experiments, we select the Mujoco-based MDC ( 23 ; 24 ) , a simulation engine for robotic actions, as the benchmark. We construct the meta-scenarios set by changing the scenario dynamics and objective (e.g., robot’s limb size and length). Through extensive experimental results, we demonstrate the advantage of our method over current state-of-the-art approaches. In summary, our contributions are threefold: 1. We propose MrCoM to learn a unified world-model based on the meta-state and meta-value regularization mechanism. 2. We conduct the theoretical analysis for MrCoM, deriving the generalization error upper bound. 3. We conduct comprehensive experiments demonstrating that, compared to other baselines, the world model learned by MrCoM can generalize across diverse scenarios. Figure 1: Framework of MrCoM, which merges multi-scenario data into a unified world-model. The meta-state regularization extracts scenario-relevant information, and meta-value regularization aligns world-model optimization with policy learning. 2 Problem Formulation 2.1 POMDP A Partially Observable Markov Decision Process (POMDP) is an extension of a Markov Decision Process (MDP) used to model decision-making problems under uncertainty and incomplete information. It can be represented as a tuple \mathcal{M}=\{\mathcal{S},\mathcal{A},r,T,{\rho}_{0},\gamma,\mathcal{O},\Omega\} . Here, \mathcal{S}\in\mathbb{R}^{n} denotes the state space, \mathcal{A}\in\mathbb{R}^{m} represents the action space, r:\mathcal{S}\times\mathcal{A}\times\mathcal{S}\rightarrow\mathbb{R} is the reward function, T:\mathcal{S}\times\mathcal{A}\rightarrow\mathcal{S} defines the dynamics of state transitions, {\rho}_{0}\in\mathcal{S} represents the initial state distribution, \gamma\in[0,1) is the discount factor, \mathcal{O}\in\mathbb{R}^{k} denotes the observation space, and \Omega:S\rightarrow\mathcal{O} is observation function representing the mapping from states to observations. The policy of an agent, denoted as \pi_{\phi}:\mathcal{O}\rightarrow\mathcal{A} , is parameterized by \phi . The discounted infinite horizon return is defined as G_{\pi}=\sum^{\infty}_{t=0}\gamma^{t}r(s_{t},a_{t}) , where t represents the time step and a denotes the action taken by the agent. The objective is to find the optimal policy parameter \phi^{*} that maximizes the cumulative discounted reward: \displaystyle\phi^{*}=\arg\max_{\phi}\mathbb{E}\left[\sum^{\infty}_{t=0}\gamma^{t}r(s_{t},\pi_{\phi}(o_{t}))\right] (1) 2.2 Meta-POMDP In this work, we consider finding a policy that maximizes expected return in a Meta-POMDP, defined as \mathcal{M}=\{\mathcal{M}^{i}\}_{i=1}^{N}=\{\mathcal{S}^{i},\mathcal{A}^{i},r^{i},T^{i},{\rho}_{0}^{i},\gamma,\mathcal{O}^{i},\Omega^{i}\}_{i=1}^{N} , with a set of scenario indices 1,\cdots,N . Each scenario i is randomly sampled from the meta-scenario set \mathcal{T}^{i}\sim p(\mathcal{T}) , where \mathcal{T}^{i} corresponds to a particular POMDP tuple \mathcal{M}^{i} . In this work, we assume that each scenario i presents a different state space \mathcal{S}^{i} , action space \mathcal{A}^{i} , reward function r^{i} , dynamics T^{i} and observation space \mathcal{O}^{i} . We formulate the policy optimization problem as finding a policy that maximizes expected return over all the scenarios: \displaystyle\phi^{*}=\arg\max_{\phi}\mathbb{E}_{\mathcal{T}^{i}\sim p(\mathcal{T})}\left[\sum^{\infty}_{t=0}\gamma^{t}r^{i}(s^{i}_{t},\pi_{\phi}(o^{i}_{t}))\right] (2) Comparison with other setting: Unlike prior works, we aim to learn a unified world-model effective across varying state spaces \mathcal{S}^{i} , action spaces \mathcal{A}^{i} , dynamics T^{i} , reward functions r^{i} , and observation functions \mathcal{O}^{i} . For example, CaDM ( 13 ) seeks to construct a world-model capable of generalizing across diverse dynamics T^{i} , which is represented as \{\mathcal{M}^{i}\}_{i=1}^{N}=\{\mathcal{S},\mathcal{A},r,T^{i},{\rho}_{0},\gamma,\mathcal{O},\Omega\}_{i=1}^{N} . Dreamerv3 ( 8 ) is designed to construct a world-model for individual tasks, where \mathcal{M}=\{\mathcal{S},\mathcal{A},r,T,{\rho}_{0},\gamma,\mathcal{O},\Omega\} . MAMBA ( 17 ) seeks to construct a world model effective across diverse scenarios with varying dynamics T^{i} and reward functions r^{i} , which is denoted as \{\mathcal{M}^{i}\}_{i=1}^{N}=\{\mathcal{S},\mathcal{A},r^{i},T^{i},{\rho}_{0},\gamma,\mathcal{O},\Omega\}_{i=1}^{N} . 2.3 Model-based RL Model-Based Reinforcement Learning (MBRL) constructs an approximate dynamics model \hat{T}_{\theta}(s^{\prime}_{t}\mid s_{t},a_{t}) , where the model parameters \theta are optimized based on observed data \mathcal{D} through maximizing the likelihood: \displaystyle\theta^{*}=\arg\max_{\theta}\sum_{(s,a,s^{\prime})\sim\mathcal{D}}\log\hat{T}_{\theta}(s^{\prime}_{t}\mid s_{t},a_{t}) (3) In the MBRL framework, policy learning can take advantage of real data and simulated data generated by the approximated dynamics model \hat{T}_{\theta}(s^{\prime}_{t}\mid s_{t},a_{t}) , which enhances sample efficiency and reduces dependency on real environment interactions. Figure 2: Model architecture of algorithms. Compared to other methods, MrCoM implements a refined partitioning of the latent state space. In this Figure, \tilde{s} denotes the latent state, d the deterministic latent state, u the stochastic latent state, and h the auxiliary state. In MrCoM, the latent state \tilde{s}_{t} is composed of the concatenation of u_{t} , d_{t} , and h_{t} . 3 Method In this section, we propose a novel method to learn a unified world-model capable of generalizing across multi-scenarios, named M eta- R egularized C ontextual W o rld- M odel (MrCoM). We first present the model architecture, which decomposes the latent state space based on dynamic characteristics, enhancing prediction accuracy of the world model. Subsequently, we separately introduce meta-state regularization and meta-value regularization mechanisms. These two mechanisms extract scenario-relevant information and align model-policy optimization, achieving cross-scenario generalization of the world model. The framework of MrCoM is shown in Figure 1 . To optimize the world-model across multiple scenario, we inject contextual information into state prediction with the transformer architecture. During inference initialization,