IMWM: Intuition Models Complement World Models for Latent Planning

Baoqi Gao, Ruize Han, Miao Wang, Song Wang, [4pt] Beihang University Shenzhen University of Advanced Technology · 2026

Read IMWM: Intuition Models Complement World Models for Latent Planning by Baoqi Gao, Ruize Han, Miao Wang, Song Wang, [4pt] Beihang University Shenzhen University of Advanced Technology (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01626.txt.

Introduction

Gao and co-authors pair an intuition model with a world model for planning. Planning with a learned latent world model suits control from raw pixels. A strong world model alone is not enough, they show. Even a perfect world model leaves a finite-budget planner failing some tasks. The bottleneck can lie in search, not model accuracy. IMWM adds an intuition model trained from demonstrations. It learns to recognize promising actions for the planner. Three light components join the two models in the system.

Retrieval Initialization seeds proposals from a retrieved demonstration. Hybrid Cost mixes the intuition score with rollout cost. A Reliability Gate adjusts trust in intuition per setting. Four pixel-based tasks show higher mean success than world-model-only planning. Gains reach 28 point 5 points on OGBench-Cube.

Catalog record

CREATORBaoqi Gao, Ruize Han, Miao Wang, Song Wang, [4pt] Beihang University Shenzhen University of Advanced Technology
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE109 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.01626

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

IMWM: Intuition Models Complement World Models for Latent Planning Baoqi Gao, Ruize Han, Miao Wang, Song Wang, [4pt] Beihang University Shenzhen University of Advanced Technology Abstract Planning with a learned latent world model is a promising route to control from raw pixels, but a strong world model alone is not enough. We show this experimentally: even with a perfect world model (operationalized by replacing the learned forward predictor with an idealized rollout of the true environment dynamics), a finite-budget sample-based planner still fails on some tasks, indicating that the bottleneck can lie in search rather than in world-model accuracy. Motivated by this gap, we propose IMWM ( Intuition Model + World Model ), which pairs the world model with an intuition model trained from demonstrations to recognize promising actions. The two models collaborate through three lightweight components: (i) Retrieval Initialization, which initializes the planner’s action proposal from a retrieved demonstration; (ii) Hybrid Cost, which combines the intuition score with the world-model rollout cost; and (iii) a Reliability Gate, which adjusts how much the planner trusts intuition in each setting. Across four pixel-based goal-reaching tasks (Two-Room, Reacher, Push-T, and OGBench-Cube), IMWM has higher mean success than the world-model-only planner on all four, with the largest gains on Two-Room ( 99.2 % 99.2\% , + 11.5 +11.5 percentage points) and OGBench-Cube ( 94.7 % 94.7\% , + 28.5 +28.5 percentage points). 1 Introduction Planning with a learned latent world model is a competitive route to control from raw pixels: an encoder maps each image to a compact latent state, a learned forward predictor rolls that state under candidate actions, and a sample-based optimizer such as the Cross-Entropy Method ( 34 ) or its modern variants ( 31 ; 18 ) ranks action sequences under a latent cost (e.g. terminal latent-MSE to a goal embedding). The lineage runs from visual MPC ( 8 ) through the PlaNet/Dreamer family ( 14 ; 13 ; 15 ; 16 ) and PETS ( 5 ) to modern latent MPC such as TD-MPC ( 18 ) . The default way to improve such a planner is to improve the world model. This paper studies when that instinct is insufficient. Planning has two ingredients: a model that predicts outcomes, and a search that proposes which actions to try. We show experimentally that the search can be the binding constraint. We operationalize a perfect world model by replacing the learned forward predictor with an idealized rollout of the true environment dynamics, while keeping the terminal latent-MSE cost and the original CEM budget fixed. Even so, on these 12-cell diagnostic grids, this oracle-dynamics planner still trails IMWM by 17.9 pp on OGBench-Cube and 13.7 pp on Two-Room, and its failures are search failures: 127/139 failed OGBench-Cube episodes and 86/87 failed Two-Room episodes contain zero goal-reaching candidates in the CEM population at both logged replans. The bottleneck is where the finite queries land, not how the world model predicts. We also prove this theoretically: under finite CEM queries against any black-box latent cost, the planner’s success probability obeys a proposal-volume bound that is independent of how the predictor was trained (Theorem A.1 , Appendix A ). Figure 1: A world model alone is not enough. Left (World Model only): the planner rolls out candidate actions accurately, but under a finite search budget it cannot find a goal-reaching action in the vast action space. Right (IMWM): pairing the same frozen world model with a frozen intuition model concentrates the search on the promising actions and reaches the goal. Both regimes use the same frozen world model and the same CEM budget; only the proposal, cost, and gating differ. If the search is what fails, it needs better guidance about which actions are worth trying (Figure 1 ). We take loose inspiration from how biological decision-making works: not as a single forward simulator, but as several functional motifs whose influence is regulated rather than fixed. These motifs include arbitration between control systems ( 6 ; 21 ) , memory-driven retrieval and predictive maps ( 24 ; 30 ; 39 ) , and hierarchical action chunking ( 7 ) . We make no claim about biological mechanism; we use these motifs only as a design vocabulary, with established machine-learning analogues in episodic control ( 3 ; 32 ) , decision-time retrieval and imitation ( 12 ; 33 ) , behavior and trajectory priors ( 29 ; 40 ; 19 ) , and value-aware model learning ( 10 ; 20 ) . § 3 positions IMWM against this work, and Appendix F gives the full ledger. We propose IMWM ( Intuition Model + World Model ), in which an intuition model complements rather than replaces the world model. The intuition model D_{\psi_{\mathrm{inv}}} is a contrastive scalar score over (start latent, goal latent, action chunk) tuples, trained on demonstration windows with an InfoNCE-style objective ( 42 ) to recognize promising actions. Its encoder and scorer feed three lightweight collaboration components. (i) Retrieval Initialization centers the CEM proposal on the action chunk retrieved by nearest-neighbor lookup in the intuition encoder’s latent space, biasing queries toward demonstration-supported regions. (ii) A Hybrid Cost combines the intuition score with the world-model rollout cost. (iii) A Reliability Gate adjusts, per setting, how much the planner trusts intuition versus falling back to the world model alone. The algorithmic family and all trained artifacts are fixed across experiments, under frozen thresholds; the only per-setting decision is which of three pre-specified recipes the gate selects, so improvements cannot come from per-task tuning (§ 4 ). Contributions. 1. Diagnosis of finite-CEM latent-WM planning. We show experimentally , with an oracle-dynamics ablation that replaces the learned predictor with idealized true-dynamics rollout under the original CEM budget, that an idealized world model still fails on OGBench-Cube and Two-Room because the search, not the predictor, is the binding constraint (§ 2.1 ); we also prove a matching predictor-independent proposal-volume bound (Theorem A.1 , Appendix A ). 2. The IMWM method. We design and train an intuition model that recognizes promising actions, and integrate it with the world model through three components: Retrieval Initialization, a Hybrid Cost, and a Reliability Gate (§ 4 ). 3. Empirical gains on four pixel-based goal-reaching tasks. Over a 48-cell fresh-seed grid (12 cells \times 4 tasks), IMWM achieves higher mean success than the world-model-only planner on all four tasks: Two-Room 99.2\% ( +11.5 pp, 12/0/0 W/T/L), OGBench-Cube 94.7\% ( +28.5 pp, 12/0/0 ), Push-T 92.7\% ( +2.8 pp, 8/1/3 ), and Reacher 83.8\% (a near-tie, +0.7 pp, 7/1/4 ), where the gate routes to the forward-only fallback (§ 5.2 ). Roadmap. § 2 establishes the finite-query diagnosis; § 3 positions IMWM against related work; § 4 defines IMWM; § 5 reports main results, routing, and ablations; and § 6 and § 7 discuss limitations and conclude. 2 Diagnosis: where latent world-model planning fails The introduction framed planning as two ingredients (a model that predicts and a search that proposes) and claimed they are separate failure surfaces. Here we ask the sharp version: assuming the world model is correct, can a finite-budget planner still fail? It can, and we show this experimentally . We swap the learned forward predictor for the true environment dynamics

Read the complete text