ReactiveGWM: Steering NPC in Reactive Game World Models
https://inv-wzq.github.io/ReactiveGWM/, Zeqing Wang, Danze Chen, Zhaohu Xing, Zizhao Tong, Yinhan Zhang, Xingyi Yang, Yeying Jin · 2026
Read ReactiveGWM: Steering NPC in Reactive Game World Models by https://inv-wzq.github.io/ReactiveGWM/, Zeqing Wang, Danze Chen, Zhaohu Xing, Zizhao Tong, Yinhan Zhang, Xingyi Yang, Yeying Jin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15256.txt.
Introduction
Zeqing Wang and six colleagues present ReactiveGWM, a game world model with flexible control roles for players and non-player characters. Existing models bind roles to fixed characters, failing multi-character games where different characters take external control while NPCs react. ReactiveGWM assigns control modes at initialization and simulates controlled players alongside reactive NPCs. Spatial role binding grounds character handles to regions in the first frame, and unified agency conditioning merges player actions and NPC rules into per-character control tokens.
Catalog record
| CREATOR | https://inv-wzq.github.io/ReactiveGWM/, Zeqing Wang, Danze Chen, Zhaohu Xing, Zizhao Tong, Yinhan Zhang, Xingyi Yang, Yeying Jin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49168 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.15256
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ReactiveGWM: Steering NPC in Reactive Game World Models https://inv-wzq.github.io/ReactiveGWM/, Zeqing Wang, Danze Chen, Zhaohu Xing, Zizhao Tong, Yinhan Zhang, Xingyi Yang, Yeying Jin Abstract Current game world models simulate environments from a subjective, player-centric perspective. However, by treating the Non-Player Character (NPC) merely as background pixels, these models cannot capture interactions between the player and NPC. In that sense, they act as passive video renderers rather than real simulation engines, lacking the physical understanding needed to model action-induced NPC reactivities. We introduce ReactiveGWM , a reactive game world model that synthesizes dynamic interactions between the player and NPC. Instead of entangling all interaction dynamics, ReactiveGWM explicitly decouples player controls from NPC behaviors. Player actions are injected into the diffusion backbone via a lightweight additive bias, while high-level NPC responses (e.g., Offense, Control, Defense) are grounded through cross-attention modules. Crucially, these modules learn a game-agnostic representation of interactive logic. This enables zero-shot strategy transfer : our learned modules can be plugged directly into off-the-shelf, unannotated world models of different games. This instantly unlocks steerable NPC interactions without any domain-specific retraining. Evaluated on two Street Fighter games, ReactiveGWM maintains fine-grain player controllability while achieving robust, prompt-aligned NPC strategy adherence, paving the way for scalable, strategy-rich interaction with the NPC. $\clubsuit$ $\clubsuit$ footnotetext: This work was completed during a research internship at Tencent, supervised by Yeying Jin. $\diamondsuit$ $\diamondsuit$ footnotetext: Project leader. Figure 1: Visualization of the steerable NPC executing distinct strategies in Street Fighter Alpha 3 ( SF3 ) game. The NPC is denoted by ▲ triangle. 1 Introduction Recent advancements in world models 3 ; 12 have established a new paradigm for simulating complex environments. By capturing the underlying dynamics from vast amounts of offline gameplay videos, this paradigm naturally extends to the development of game world models 33 ; 29 , unlocking unprecedented possibilities for interactive game generation. However, most existing game world models fail to explicitly model the Non-Player Character (NPC) . Instead, they simulate environments from a player-centric perspective 5 ; 33 ; 17 . These models use player-centric prompts to generate non-player elements as part of the interactive background. Such a design implicitly assumes a deterministic relation between the player and the background. As a result, NPCs are often reduced to background pixels rather than modeled as dynamic and autonomous agents since their behaviors are tightly tied to fixed action sequences specified in the prompt. This makes most existing game world models closer to passive video renderers than to real game simulation engines. In actual games, NPCs follow high-level strategies to achieve dynamic and autonomous engagement. Ignoring this aspect limits gameplay to a largely solitary experience and prevents meaningful competitive interaction between the player and NPCs. To overcome this limitation, we introduce a novel reactive game world model, termed ReactiveGWM . ReactiveGWM is explicitly designed to synthesize dynamic interactions between a player and an autonomous NPC. To achieve this, we construct novel datasets that decouple NPC autonomy from player control. In these datasets, in addition to gameplay videos and player action labels, each sample includes a structured prompt that only guides the NPC. Instead of entangling all interaction dynamics in standard player-centric prompts, our prompts specify the NPC with explicit strategic guidance and both active and passive behaviors, enabling autonomous strategy execution. Given these structured, strategy-aligned datasets, we train ReactiveGWM to encode player control and NPC autonomy without entangling roles. Specifically, player actions are injected into the video diffusion backbone via a lightweight additive bias. Concurrently, we ground high-level NPC strategies in the cross-attention modules. Training on these data allows the cross-attention modules to learn a player-agnostic representation of interaction logic. Crucially, to enforce strategic autonomy rather than player-centric guidance, these modules are driven entirely by pure NPC behavioral (e.g., Offense, Control, Defense). This disentangles the NPC’s tactical intent from shallow descriptive prompts. Meanwhile, the game-specific physical and visual dynamics are still modeled by the original self-attention and feed-forward layers. By separating NPC behavioral logic from these dynamics, the learned behavior modules also form a game-agnostic representation . This representation can be transferred across different games in a plug-and-play manner. Evaluated on two distinct Street Fighter games, our experiments show that ReactiveGWM maintains fine-grained player controllability while enabling autonomous, strategy-aligned NPC behavior. The results demonstrate realistic and dynamic interactions between the player and the NPC. More importantly, ReactiveGWM exhibits strong zero-shot strategy transferability . The learned NPC autonomy modules can be directly plugged into off-the-shelf vanilla world models of different games without additional annotation. This enables steerable NPC interactions without domain-specific strategy retraining, while preserving the native dynamics of the target game. In summary, our contributions are summarized as follows: (1) We propose ReactiveGWM , breaking the limitations of player-centric modeling by simultaneously supporting fine-grained player control and strategy-driven NPC autonomy. (2) We construct new strategy-aligned datasets to explicitly distinguish tactical intent of the NPC from pixel-level rendering. With decoupled injection for player and NPC control, ReactiveGWM achieves both fine-grained player controllability and strategy-aligned NPC behavior. (3) We demonstrate that our specialized modules learn a game-agnostic interactive logic. These modules can be seamlessly transferred to off-the-shelf, unannotated target games, paving the way for highly scalable, strategy-rich game generation. 2 Related works 2.1 Controllable video generation With the rapid advancement of video diffusion models 35 ; 20 ; 25 ; 24 ; 4 ; 44 ; 22 ; 37 , visual content generation has achieved unprecedented fidelity. While detailed text prompts enable customized generation, they inherently lack fine-grained control, frequently resulting in spatiotemporal ambiguities. To achieve rigorous spatial and temporal alignment, controllable video generation frameworks incorporate auxiliary conditions, such as motion priors and trajectory inputs 36 ; 8 ; 45 ; 41 ; 48 , camera trajectories 39 ; 43 ; 16 , and structural guidance for consistent character animation 30 ; 10 ; 19 ; 18 ; 42 ; 49 . Beyond such localized controllability, a more ambitious line of work seeks to actively simulate causal physical mechanics, with the generative paradigm naturally evolving toward world models. By predicting future states and environmental transitions conditioned on current observations and external interventions 5 ; 23 ; 3 , world models equip agents with a predictive “mental model” of the physical world. This capability is foundational for downstream decision-making, facilitating strategic planning and “learning in imagination” 12 ; 11 ; 13 ; 15 . Consequently, it has been shown to enable sample-efficient policy optimization in reinforcement learning and robotics 14 ; 28 ; 40 , mitigating the cost of exhaustive interactions with the actual environment. 2.2 Game world models Game world models aim to construct simulations of game environments, predicting future visual frames conditioned on player inputs. Pioneering works like GameNGen 34 demonstrated that diffusion models can serve as real-time neural engines for DOOM, while DIAMOND 1 established that the visual fidelity of diffusion world models significantly impacts downstream policy learning. Subsequent efforts, including Matrix-Game 2.0/3.0 17 ; 29 , LingBot-World 33 , GameFactory 46 , and Oasis 9 , have pushed the boundaries toward streaming, long-horizon, and open-domain generation. However, the conditioning vocabulary in the majority of these models remains uniformly restricted to the primary player’s action stream. Consequently, the Non-Player Character (NPC) are fundamentally absorbed into the