Abstract

A single action-conditioned latent predictive architecture can in principle be trained on the structured state of a driving scene, a robot workspace, or a financial order book. The ingredients for doing so within any one of these domains already exist and are individually validated: masked-latent prediction [1, 2], action-conditioned latent world models [3, 8, 9], discrete action tokenization [4, 5, 6], and joint-embedding prediction on voxelized state [14, 15, 17]. What is not established, and what TERRA addresses, is the transfer question: when does a representation or predictor learned in one structured-state domain carry over to a structurally analogous but otherwise unrelated domain, and by how much. We give this question a formal treatment. We model each domain as a controlled Markov process on a graded latent grid, factor any instantiation into thin domain adapters and a shared domain-invariant core, and identify a cross-domain correspondence with an approximate Markov decision process homomorphism [21, 23, 24] whose quality is measured by a lax bisimulation discrepancy [24, 27] and, at the level of whole domains lacking a shared coordinate system, by a Gromov-Wasserstein distance between their action-conditioned transition operators [29]. Under a Lipschitz predictor we derive a predictive transfer bound that separates source-model error from structural mismatch, grows geometrically in the prediction horizon, and is certified from below by the Gromov-Wasserstein distance; we then connect latent error to decision regret through the Lipschitz value property of bisimulation metrics [21, 22, 25]. The resulting Structured-State Transfer Hypothesis is stated as a falsifiable claim with a preregistered experimental program, centered on a transfer test from driving scenes to order books, including the conditions under which the claim is refuted. We present no empirical results: this is a research proposal whose contribution is to convert a widely repeated intuition into testable theory. · 2026

Read Abstract by A single action-conditioned latent predictive architecture can in principle be trained on the structured state of a driving scene, a robot workspace, or a financial order book. The ingredients for doing so within any one of these domains already exist and are individually validated: masked-latent prediction [1, 2], action-conditioned latent world models [3, 8, 9], discrete action tokenization [4, 5, 6], and joint-embedding prediction on voxelized state [14, 15, 17]. What is not established, and what TERRA addresses, is the transfer question: when does a representation or predictor learned in one structured-state domain carry over to a structurally analogous but otherwise unrelated domain, and by how much. We give this question a formal treatment. We model each domain as a controlled Markov process on a graded latent grid, factor any instantiation into thin domain adapters and a shared domain-invariant core, and identify a cross-domain correspondence with an approximate Markov decision process homomorphism [21, 23, 24] whose quality is measured by a lax bisimulation discrepancy [24, 27] and, at the level of whole domains lacking a shared coordinate system, by a Gromov-Wasserstein distance between their action-conditioned transition operators [29]. Under a Lipschitz predictor we derive a predictive transfer bound that separates source-model error from structural mismatch, grows geometrically in the prediction horizon, and is certified from below by the Gromov-Wasserstein distance; we then connect latent error to decision regret through the Lipschitz value property of bisimulation metrics [21, 22, 25]. The resulting Structured-State Transfer Hypothesis is stated as a falsifiable claim with a preregistered experimental program, centered on a transfer test from driving scenes to order books, including the conditions under which the claim is refuted. We present no empirical results: this is a research proposal whose contribution is to convert a widely repeated intuition into testable theory. (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01520.txt.

Introduction

Shokri asks when learned models transfer across unlike domains. One latent predictive design could serve many settings. A driving scene, a robot workspace, and an order book all qualify. The parts for one domain exist and are validated. What is not known is when transfer works, and by how much. TERRA gives that question a formal treatment. Each domain is a controlled Markov process on a latent grid.

Any setup splits into thin adapters and a shared core. The core is meant to stay invariant across domains. A cross-domain map is scored by bisimulation discrepancy. Gromov-Wasserstein distance serves domains with no shared coordinates. A transfer bound separates source error from structural mismatch. The claim is stated in falsifiable form.

Catalog record

CREATORA single action-conditioned latent predictive architecture can in principle be trained on the structured state of a driving scene, a robot workspace, or a financial order book. The ingredients for doing so within any one of these domains already exist and are individually validated: masked-latent prediction [1, 2], action-conditioned latent world models [3, 8, 9], discrete action tokenization [4, 5, 6], and joint-embedding prediction on voxelized state [14, 15, 17]. What is not established, and what TERRA addresses, is the transfer question: when does a representation or predictor learned in one structured-state domain carry over to a structurally analogous but otherwise unrelated domain, and by how much. We give this question a formal treatment. We model each domain as a controlled Markov process on a graded latent grid, factor any instantiation into thin domain adapters and a shared domain-invariant core, and identify a cross-domain correspondence with an approximate Markov decision process homomorphism [21, 23, 24] whose quality is measured by a lax bisimulation discrepancy [24, 27] and, at the level of whole domains lacking a shared coordinate system, by a Gromov-Wasserstein distance between their action-conditioned transition operators [29]. Under a Lipschitz predictor we derive a predictive transfer bound that separates source-model error from structural mismatch, grows geometrically in the prediction horizon, and is certified from below by the Gromov-Wasserstein distance; we then connect latent error to decision regret through the Lipschitz value property of bisimulation metrics [21, 22, 25]. The resulting Structured-State Transfer Hypothesis is stated as a falsifiable claim with a preregistered experimental program, centered on a transfer test from driving scenes to order books, including the conditions under which the claim is refuted. We present no empirical results: this is a research proposal whose contribution is to convert a widely repeated intuition into testable theory.
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.01520

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Abstract A single action-conditioned latent predictive architecture can in principle be trained on the structured state of a driving scene, a robot workspace, or a financial order book. The ingredients for doing so within any one of these domains already exist and are individually validated: masked-latent prediction [1, 2], action-conditioned latent world models [3, 8, 9], discrete action tokenization [4, 5, 6], and joint-embedding prediction on voxelized state [14, 15, 17]. What is not established, and what TERRA addresses, is the transfer question: when does a representation or predictor learned in one structured-state domain carry over to a structurally analogous but otherwise unrelated domain, and by how much. We give this question a formal treatment. We model each domain as a controlled Markov process on a graded latent grid, factor any instantiation into thin domain adapters and a shared domain-invariant core, and identify a cross-domain correspondence with an approximate Markov decision process homomorphism [21, 23, 24] whose quality is measured by a lax bisimulation discrepancy [24, 27] and, at the level of whole domains lacking a shared coordinate system, by a Gromov-Wasserstein distance between their action-conditioned transition operators [29]. Under a Lipschitz predictor we derive a predictive transfer bound that separates source-model error from structural mismatch, grows geometrically in the prediction horizon, and is certified from below by the Gromov-Wasserstein distance; we then connect latent error to decision regret through the Lipschitz value property of bisimulation metrics [21, 22, 25]. The resulting Structured-State Transfer Hypothesis is stated as a falsifiable claim with a preregistered experimental program, centered on a transfer test from driving scenes to order books, including the conditions under which the claim is refuted. We present no empirical results: this is a research proposal whose contribution is to convert a widely repeated intuition into testable theory. Abstract TERRA: Task-Embedded Reasoning and Representation Architecture for Cross-Domain Applications Shayan Shokri 1 1 1 [email protected] Humanpath Labs Inc. 1 Introduction The claim that a predictive model of structured state in one domain captures something reusable about decision-making in another is repeated frequently and tested almost never. Action-conditioned latent predictors now work within individual domains. V-JEPA 2-AC [3] plans on a physical robot from rollouts in representation space; joint-embedding prediction yields strong features on point clouds [14] and on LiDAR occupancy grids [17]. Each of these is a within-domain result. The promise that the same architecture, or even the same learned representation, carries from a physical scene to a nonphysical structured system such as a market microstructure or a power grid is asserted widely, yet it is supported by neither a theory of when it should hold nor an experiment establishing whether it does. TERRA takes the position that this transfer question is the open problem worth solving, and that it is answerable with existing mathematical tools rather than left as an intuition. Our contributions are the following. 1. A formal model of a structured-state decision domain as a controlled Markov process on a graded latent grid (Section 3), and a factorization of any instantiation into domain adapters and a shared domain-invariant core (Section 4). 2. A theory of cross-domain correspondence (Section 5) that identifies the correspondence between two domains with an approximate Markov decision process homomorphism, measures its quality by a lax bisimulation discrepancy and a Gromov-Wasserstein distance between transition operators, and derives a predictive transfer bound separating source-model error from structural mismatch. 3. A reduction from latent prediction error to decision regret using the Lipschitz value property of bisimulation metrics, so that the bound speaks to control quality and not only to representation fidelity. 4. The Structured-State Transfer Hypothesis, stated as a falsifiable empirical claim, and a preregistered experimental program with explicit refutation conditions (Section 8). We use the architecture of an existing structured-state predictor, retained under the TERRA name, as a concrete substrate that realizes the interface (Section 6). The substrate is a recombination of validated components and is presented as engineering. The theory of transfer built on top of it is the contribution. 2 Related Work and Positioning We position TERRA against two bodies of work: the predictive-learning lineage it uses as a substrate, and the abstraction-theory lineage on which the transfer analysis rests. Predictive-learning substrate. I-JEPA [1] and V-JEPA [2] established masked prediction in representation space following the program of LeCun [11]. V-JEPA 2-AC [3] adds action conditioning and plans on a real robot. DINO-WM [9], PLDM [10], Dreamer [8], and the original world model of Ha and Schmidhuber [7] form the action-conditioned world-modeling line; RT-2 [4], OpenVLA [5], and \pi_{0} [6] supply discrete action tokenization; BYOL [12] and masked autoencoding [13] supply the anti-collapse and masking machinery; Point-JEPA [14], 3D-JEPA [15], GeoMAE [16], and the LiDAR occupancy world model [17] bring all of this to voxelized state, the substrate domain TERRA itself uses. None of these components is claimed as novel here. Abstraction theory. Markov decision process homomorphisms [23] and bisimulation metrics [21, 22] formalize when two states, or two processes, are behaviorally equivalent, and they come with value-preservation guarantees. The lax bisimulation metric [24] extends this to abstract over actions and is exactly the relation realized by a homomorphism. DeepMDP [25] and deep bisimulation for control [26] learn latent spaces in which embedding distance upper-bounds bisimulation distance. Continuous homomorphisms with policy-gradient guarantees [27] and a generalized bisimulation metric between pairs of distinct processes [28] extend the theory to the cross-process setting that transfer requires. We use these tools, together with the Gromov-Wasserstein distance between metric-measure spaces [29], to make the cross-domain correspondence quantitative. What remains open. None of the predictive-learning work tests transfer across the boundary from physical to nonphysical domains, and none of the abstraction-theory work has been instantiated for high-dimensional action-conditioned latent predictors over structured grids. TERRA joins the two. 3 Structured-State Decision Domains Definition 1 (Structured-state domain) . A structured-state decision domain is a tuple \mathcal{D}=(\mathcal{S},\mathcal{A},T,r) where the state space \mathcal{S} is a Polish space admitting a graded grid representation V\in\mathbb{R}^{d_{1}\times\cdots\times d_{k}\times c} ; the action space \mathcal{A} admits a discrete tokenization \tau:\mathcal{A}\to\mathcal{V}^{*} over a finite vocabulary \mathcal{V} ; T:\mathcal{S}\times\mathcal{A}\to\mathcal{P}(\mathcal{S}) is a transition kernel; and r:\mathcal{S}\times\mathcal{A}\to\mathbb{R} is a bounded task signal. A driving scene is a voxel grid over physical space; an order book is a grid over price, time, and side; a risk portfolio is a grid over assets and factors. We call \mathcal{D} admissible when all four elements are present and T is learnable from logged trajectories. Admissibility is a property of the domain and not of any model. 4 The Domain-Invariant Interface The interface is what makes transfer measurable rather than rhetorical. We factor any instantiation into two parts. The domain adapters, which are small and domain-specific, comprise an encoder front-end e_{\mathcal{D}}:\mathcal{S}\to\mathcal{Z} mapping the grid into a canonical latent space (\mathcal{Z},d_{\mathcal{Z}}) , together with the tokenizer \tau_{\mathcal{D}} . The shared core, which is large and domain-invariant, comprises a context-encoder trunk f_{\theta} , an exponential-moving-average target encoder f_{\bar{\theta}} , and an action-conditioned latent predictor g_{\phi}:\mathcal{Z}\times\mathcal{V}^{*}\to\mathcal{Z} that operates only on canonical latents and action tokens. Because the core is identical across domains by construction, any transfer that is observed when the core trained on one domain is reused on another is attributable to shared structure rather than to architecture reuse. This

Read the complete text