A 3D Isovist World Model: Revealing a City’s Unseen Geometry and Its Emergent Cross-City Signature
Xuhui Lin, Stephen Law, Nanjiang Chen, Kunyao Li, Tao Yang · 2026
Read A 3D Isovist World Model: Revealing a City’s Unseen Geometry and Its Emergent Cross-City Signature by Xuhui Lin, Stephen Law, Nanjiang Chen, Kunyao Li, Tao Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03609.txt.
Introduction
Lin and co-authors build a world model for city navigation based on open space, not looks. Embodied agents need to know where they can go. Most world models predict appearance instead. Occupancy grids flatten the city to a ground plane and lose height structure. A 3D isovist captures what is missing. It maps distance to the nearest surface in every direction from one spot.
Their model predicts the next isovist from past isovists and a movement action. A depth residual keeps building edges sharp. Scheduled sampling with self-rollout keeps corrupted context on the geometry manifold. A persistent latent map links paths for consistency. A surprise emerges from training on Manhattan and Paris. City identity can be decoded linearly from the model's temporal latents. The signature thus lives in learned motion dynamics, not in looks. The dataset and pipeline are open for robotics and urban research.
Catalog record
| CREATOR | Xuhui Lin, Stephen Law, Nanjiang Chen, Kunyao Li, Tao Yang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.03609
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A 3D Isovist World Model: Revealing a City’s Unseen Geometry and Its Emergent Cross-City Signature Xuhui Lin, Stephen Law, Nanjiang Chen, Kunyao Li, Tao Yang Abstract Embodied agents that navigate cities rely on world models that predict how their surroundings will change as they move. But for navigation, what matters is not what the buildings look like; it is where the agent can go. Most world models nonetheless predict appearance, learning how a scene looks rather than the space an agent can move through. Those that do target geometry, such as bird’s-eye-view occupancy grids, flatten the three-dimensional environment onto a ground plane, discarding the above-ground and multi-level structure that shapes real navigation. What is missing is a predictive target that captures the navigable geometry an agent actually traverses, without photometric entanglement and without collapsing the third dimension. Our key idea is to model the open volume between buildings, the negative space , encoded as a 3D isovist : a spherical visibility-depth map recording the distance to the nearest surface in every direction. We introduce an embodied world model that predicts the next isovist from a short history of past isovists and a movement action. The prediction is formulated as a depth residual so the decoder inherits sharp building edges, trained with self-rollout scheduled sampling to keep corrupted context on the geometry manifold, and equipped with a persistent latent bird’s-eye-view spatial map for cross-path consistency. Our central finding is emergent and unexpected: a single city-blind model trained on Manhattan and Paris develops a cross-city spatial signature , with city identity linearly decodable from its temporal latents far above single-frame baselines, so the signature lives in the learned dynamics rather than in appearance. The representation is lightweight, interpretable, and reproducible, offering a geometric substrate for spatial reasoning in embodied AI, robotics, and urban analysis, released with an open dataset and pipeline. Keywords: Isovist, World Model, Embodied Navigation, Negative Space, Urban Morphology. 1 Introduction Consider an embodied agent moving along a city street. At every step, what it can perceive is not the buildings themselves but the space between them: the open, traversable volume its body occupies, shaped by surrounding facades, corners, and passages. This visibility volume — the set of all points visible from a given location — is called an isovist ( 4 ) . Formalized in architectural theory ( 18 ) and spatial cognition ( 28 ; 32 ) , isovists have long served to analyse how humans experience and navigate urban space. In three dimensions, an isovist is compactly encoded as a spherical depth map D\in\mathbb{R}^{H\times W} indexed by polar angle \theta and azimuth \varphi , where each cell records the distance to the nearest surface in that direction. Each step of motion produces a new isovist; the sequence of isovists along a path is therefore a natural state representation for a navigating agent — and, we argue, the natural perceptual substrate for an embodied world model of the city. Despite this, isovists have never, to our knowledge, been used as the predictive state of a world model. We argue this is a significant gap, and that closing it changes what the model is forced to represent. From the perspective of a robot or autonomous agent navigating a city, the central question is not “what do the buildings look like?” but “what space is available to me, and how will it change if I move?” The isovist answers this directly. It is perception-aligned , encoding exactly the geometry a range sensor measures; it is action-conditioned by construction, since each movement produces a new visibility volume; and it is reconstructive , since isovists accumulated from multiple viewpoints collectively reveal the surrounding building surfaces, with negative space giving rise to positive-space geometry. We refer to this framing as world modelling in negative space (Figure 1 ). This representational choice is the first thing that distinguishes our work, and it is worth stating precisely what it buys an embodied agent. Appearance-first world models predict future RGB frames ( 1 ; 30 ; 31 ) and so must carry photometric variation, such as lighting, shadow, and surface colour, that is irrelevant to where an agent can move; geometry is entangled with appearance, and geometric consistency becomes a secondary concern. Bird’s-eye-view occupancy models ( 36 ) are geometry-pure but collapse the full three-dimensional structure of the environment onto a flat ground plane, discarding the vertical and above-ground geometry that shapes real navigation, such as overpasses, stacked facades, and multi-level plazas and frontage. The spherical isovist sits between these: it is metric geometry with no photometric channel to entangle, and because it records depth in every direction around the agent rather than a flattened footprint, it preserves the above-ground and multi-level structure a ground-plane grid loses. In this sense the isovist reveals what is otherwise hidden. It exposes the navigable geometry of the space itself, independent of how that space happens to be lit or coloured, which is precisely the structure that supports how embodied agents move through cities, on the ground and above it. Implicit scene representations such as NeRF ( 22 ) and 3D Gaussian Splatting ( 19 ) reconstruct seen scenes at high fidelity but are scene-specific and not designed for action-conditioned generation across novel trajectories. Our model instead starts from geometry directly: given a sequence of past isovists and a movement action, predict the next isovist, replacing abstract latent features and photometric input with an interpretable, sensor-proximal target. A world model that merely predicts the next frame, however, has no memory of where it has been. When two independently sampled paths cross the same intersection, a frame-only model can predict mutually contradictory geometry at the shared location, because nothing ties the two predictions together. To address this, we augment the predictor with a persistent latent BEV spatial map : an explicit grid of latent features over world coordinates that the model reads from at its current position and writes to after each observation. Because the map is keyed by absolute world position rather than by trajectory, two paths crossing a cell read and write the same memory, providing a mechanism for cross-path geometric consistency. This explicit, writable map is the structural contrast with the closest prior work, Gornet and Thomson’s emergent implicit cognitive map for localization in a small, fixed environment ( 12 ) ; ours is explicit, persistent, and writable, intended as a generation substrate over real cities (Section 2 ). Training an autoregressive isovist predictor exposes a familiar failure mode: the teacher-forcing gap. A model trained only on ground-truth context meets its own imperfect predictions at inference, and errors compound over a rollout ( 5 ) . The natural fix, corrupting the context during training, has a subtlety specific to our representation: Gaussian-blur corruption of a depth map produces something that is not a valid isovist, because it smooths across the sharp depth discontinuities at building edges, a pattern that never occurs in real geometry. We instead use self-rollout scheduled sampling : a no-grad forward pass yields the model’s own prediction, and the last context frame is replaced by a convex blend of ground truth and that prediction on a linear curriculum. Because the prediction is itself a valid