Identifiable Token Correspondence for World Models
Youngin Kim, Ray Sun, Inho Kim, Bumsoo Park, Hyun Oh Song · 2026
Read Identifiable Token Correspondence for World Models by Youngin Kim, Ray Sun, Inho Kim, Bumsoo Park, Hyun Oh Song (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16457.txt.
Introduction
Kim and four others present Identifiable Token Correspondence, a reading step for models treating next-frame prediction as an assignment problem. Current methods treat next-frame output as token generation, missing token persistence, which causes mismatch in rollouts. Under ITC, next-frame tokens come from copying an earlier token or making a new one. ITC leaves architecture and training unchanged and fits bases. Tests show state-of-the-art scores on four benchmarks, with 72.5 percent return on Craftax-classic.
Catalog record
| CREATOR | Youngin Kim, Ray Sun, Inho Kim, Bumsoo Park, Hyun Oh Song |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62483 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.16457
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Identifiable Token Correspondence for World Models Youngin Kim, Ray Sun, Inho Kim, Bumsoo Park, Hyun Oh Song Abstract Token-based transformer world models have shown strong performance in visual reinforcement learning, but often suffer from temporal inconsistency in long-horizon rollouts, including object duplication, disappearance, and transmutation. A key reason is that most existing approaches treat next-frame prediction purely as a token generation problem, without considering the persistence of tokens across time. We introduce Identifiable Token Correspondence (ITC), a decoding step for token-based transformer world models that formulates next-frame prediction as a structured assignment problem with latent token correspondence variables: each next-frame token is explained either by copying a token from the previous frame or by generating a new one. ITC leaves the transformer architecture and training procedure unchanged and can be added on top of existing backbones. Our experiments show state-of-the-art performance on 4 challenging benchmarks. The proposed method achieves a return of 72.5% and a score of 35.6% on the Craftax-classic benchmark, significantly surpassing the previous best of 67.4% and 27.9%. We release our source code on https://github.com/snu-mllab/Identifiable-Token-Correspondence . † † affiliationnotice: Equal contribution 1 Introduction Figure 1 : Sequential frames in visual environments like Craftax-classic and Atari contain the same underlying entities. Reinforcement learning (RL) provides a framework for training agents to interact with their environment through reward signals ( 34 ) . To avoid heavy reliance on costly environment interactions, model-based RL learns a predictive model of the environment dynamics, enabling the agent to simulate future trajectories called “imaginations” ( 13 ; 23 ) . Recently, token-based transformer world models have emerged as powerful approaches ( 23 ; 11 ) . These models treat sequences of past states and actions as token streams and predict the next state token-by-token. We focus on this class of world models throughout the paper. However, despite recent advances, such models often exhibit temporal inconsistency in long-horizon rollouts, including object duplication, disappearance, and transmutation into different objects. These errors compound over time and significantly limit the usefulness of long imagined trajectories for policy training. A central reason for this failure is that most existing world models treat next-frame prediction purely as a token generation problem ( 23 ; 11 ) . In realistic environments, however, many tokens in successive frames correspond to the same underlying entities that persist and move over time (see Figure 1 ). Predicting the next frame therefore requires determining not only what token should appear at each position, but also where that token comes from. When correspondence across time is not modeled explicitly, these two questions are conflated, forcing the model to relearn persistent structure at every step and making identity preservation fragile. Figure 2 : Our proposed world model enhances next state prediction by solving an optimal transport problem with previous state tokens ( s_{t} , blue) and the transformer’s output for candidate next-state tokens ( \tilde{s}_{t+1} , green) to generate the final next-state tokens ( \hat{s}_{t+1} ). Optimal transport defines an affinity matrix from the s_{t} and \tilde{s}_{t+1} tokens to the positions for \hat{s}_{t+1} . A solver takes the affinity matrix and produces a transport plan, assigning a token from s_{t} or \tilde{s}_{t+1} to each final next-state token in \hat{s}_{t+1} . This approach enables effective reuse of relevant past tokens. To address this problem, we propose Identifiable Token Correspondence (ITC), a decoding step that augments existing token-based transformer world models. ITC is not a new architecture or training recipe: it leaves the transformer and its training loss unchanged, and inserts an optimal transport solver between the transformer’s next-token predictions and the final next-state tokens. The solver introduces latent variables that assign each next-frame token either to a copied token from the previous frame or to a token sampled from the transformer’s predictions. This enables partial reuse of previous tokens, reducing hallucinations and improving object persistence over time, while preserving the upstream world model and its training procedure. We evaluate ITC on the Craftax-classic, Craftax, MinAtar, and Atari 100K benchmarks. Craftax-classic is a challenging 2D open-world game featuring long-horizon tasks and dynamic enemies ( 22 ) . ITC achieves a return of 72.5% and a score of 35.6%, setting a new state-of-the-art and outperforming the previous best results of 67.4% and 27.9%, respectively ( 11 ) . Craftax is a harder environment based on Craftax-classic, in which ITC also exceeds baselines ( 22 ) . MinAtar is a suite of 4 Atari games with simplified representations, which tests generality across different game dynamics ( 39 ) . ITC surpasses the previous state of the art for model-based RL in all 4 games ( 11 ) . Atari 100K is a suite of 26 Atari games with diverse visual structure. ITC surpasses the previous state-of-the-art token-based world model ( 8 ) across the 26 games. 2 Preliminaries 2.1 Model-based Reinforcement Learning Reinforcement learning considers a Partially Observable Markov Decision Process (POMDP), characterized by (\mathbb{S},\mathbb{A},\Omega,T,O,R,\gamma) , where \mathbb{S} is a set of states, \mathbb{A} is a set of discrete actions, \Omega is a set of observations, T gives the transition probabilities between states T(s^{\prime}\mid s,a) , O gives the observation probabilities O(o\mid s) , and R is a reward function R(s,a) ( 34 ) . The goal is to find a policy \pi which chooses actions for each state that maximizes the expected discounted return \mathbb{E}_{\pi}\left[\sum_{t\geq 0}\gamma^{t}r_{t}\right] , where \gamma is a discount factor. A world model takes an input of previous state s_{t} and action a_{t} , then returns a predicted output of next state \hat{s}_{t+1} , reward r_{t} , and done signal d_{t} , similar to the real environment. The agent collects real environment trajectories during training by interacting with the environment using the policy \pi . Then the world model trains on the trajectories saved in the replay buffer. Over the course of training, the agent is trained on both the trajectories collected from the real environment and generated trajectories from the world model, called imaginations . 2.2 RoPE Rotary Position Embedding (RoPE) is a positional encoding method that injects positional information into a transformer’s attention mechanism by applying rotations to query and key vectors ( 33 ) . These rotations cause the attention operation to naturally encode relative offsets between tokens. Concretely, each input token embedding is partitioned into pairs of coordinates, with each pair forming a 2D subspace where a rotation is applied according to the token’s 1D position index. Owing to its simplicity and scalability, RoPE has become the standard positional encoding in modern transformer architectures. However, RoPE uses a single-dimensional position index, which is unable to distinguish between temporal differences (i.e., tokens from different time steps) and spatial differences (i.e., tokens from different positions within the same frame). To incorporate both spatial and temporal information into the model, 3D positional encoding for multi-dimensional information has been developed ( 37 ; 38 ) . Each token’s embedding is divided into three sub-vectors corresponding to its temporal, vertical, and horizontal coordinates. RoPE is then applied independently along each axis, enabling the attention mechanism to capture localized relational structure across both space and time. This formulation allows the model to generalize over local interactions (e.g., neighboring pixels or frames), regardless of absolute location. It preserves adjacency