TerraDiT- Ω \mathrm{\Omega} : Unified Spatial Control for Satellite Image Synthesis with Any Geospatial Primitive
Brian Wei, Srikumar Sastry *, Daniel Cher *, Eric Xing, Nathan Jacobs · 2026
Read TerraDiT- Ω \mathrm{\Omega} : Unified Spatial Control for Satellite Image Synthesis with Any Geospatial Primitive by Brian Wei, Srikumar Sastry *, Daniel Cher *, Eric Xing, Nathan Jacobs (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31029.txt.
Introduction
Wei and four colleagues present TerraDiT-Omega, a model that creates satellite images under direct spatial control. Older systems force map data into dense rasters or sparse prompts. Dense rasters cost heavy labeling effort. Sparse prompts lose the exact shapes stored in vector data. TerraDiT-Omega instead takes native geospatial primitives: the polygons, polylines, boxes, and points that maps already use.
A method called Geometry-Aware Local Attention feeds these shapes into the model's attention layers. In tests, the system beat both dense-control and sparse-control baselines in every input format. It can also make synthetic training data with a single model. That extra data improved land-cover segmentation, object detection, road graph extraction, and scene classification. The authors release code, data, and model weights for others to build on.
Catalog record
| CREATOR | Brian Wei, Srikumar Sastry *, Daniel Cher *, Eric Xing, Nathan Jacobs |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.31029
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TerraDiT- Ω \mathrm{\Omega} : Unified Spatial Control for Satellite Image Synthesis with Any Geospatial Primitive Brian Wei, Srikumar Sastry *, Daniel Cher *, Eric Xing, Nathan Jacobs Abstract Generative models have achieved remarkable progress, yet applying them to satellite imagery remains challenging. Unlike natural imagery, satellite scenes are structured by spatially complex and semantically distinct geometries. Prior work addresses this complexity by adapting natural image frameworks using dense rasters or sparse prompts, trading off annotation cost and fidelity while breaking compatibility with vector primitives commonly used to represent geographic information. We introduce TerraDiT- Ω \mathrm{\Omega} , a unified spatial control framework that generates satellite imagery directly from any native geospatial primitive. By jointly leveraging precise annotations (polygons, polylines) and coarser ones (bounding boxes, points), the model supports controllable layouts across varying annotation budgets, broadening applicability to design tasks such as urban planning while remaining naturally compatible with end-to-end GeoAI workflows. To effectively leverage these primitives during generation, we propose Geometry-Aware Local Attention, a conditioning mechanism that injects explicit geometric cues into the attention space. Across all conditioning formats, our approach consistently outperforms both dense-control and sparse-control baselines. Furthermore, this flexibility enables controllable synthetic data augmentation using a single generative model, improving downstream performance on land-cover segmentation, object detection, road graph extraction, and scene classification. Code, data, and weights are available at https://github.com/mvrl/TerraDiT . Figure 1 : TerraDiT- \mathrm{\Omega} enables precise synthesis using polygons , polylines , bounding boxes , points , along with text and geolocation conditioning. 1 Introduction Controllable generative modeling has advanced rapidly in recent years. Beyond text-to-image synthesis, modern frameworks allow generation to be guided by spatial signals such as depth maps, edge maps, segmentation layouts, or bounding boxes [ 35 , 22 , 66 , 28 , 56 ] . Architectures for spatial grounding, including ControlNet [ 66 ] and instance-grounded approaches [ 56 , 28 ] , enable precise spatial control in natural images featuring few, dominant objects. However, these models struggle in the remote sensing (RS) domain, where scenes consist of dense, interconnected arrangements of small elements [ 44 , 53 ] . To bridge this gap, prior work in geospatial generation adapts existing controllable architectures using geographic data sources such as OpenStreetMap, which provide rich vector annotations describing roads, buildings, land use regions, and other geospatial features [ 24 ] . Such annotations are converted into vision-compatible formats like pixel maps, bounding boxes, or point prompts to guide synthesis [ 8 , 49 , 3 , 48 , 43 , 42 ] . The resulting conditional models benefit Earth observation applications such as disaster response, environmental monitoring, and urban planning [ 14 , 18 , 34 , 37 ] . While these adaptations make existing architectures applicable to satellite imagery, they introduce two fundamental limitations. First, converting native geospatial primitive formats degrades their precise geometric properties [ 44 ] . For example, single-channel pixel maps ambiguously flatten complex overlaps, rendering a layered overpass indistinguishable from a standard intersection. Vector primitives explicitly preserve this structural continuation, alongside other multi-instance overlaps like land-use zones intersecting buildings. While multi-channel pixel maps can resolve overlaps, they create severe computational bottlenecks in dense satellite scenes with significantly more instances than natural imagery. Second, format conversion misaligns with practical GeoAI workflows. Dense semantic maps require prohibitive annotation efforts: on average, annotating a single OpenEarthMap segmentation tile takes 2.5 hours [ 61 ] . Conversely, while sparse signals like point prompts scale cheaply, they underspecify crucial geometric layouts. Locking into a single supervision granularity forces practitioners to deploy separate architectures for different tasks, making large-scale deployment impractical. A truly robust end-to-end framework should support varying annotation budgets—from cheap point prompts to precise polygons—without requiring format conversion. To address these limitations, we propose TerraDiT- \mathrm{\Omega} , a unified framework that directly consumes native geospatial primitives (polygons, polylines, bounding boxes, and points) to guide satellite image synthesis (see Fig. 1 ). Directly operating on primitives introduces new technical challenges: geospatial features exhibit diverse geometric complexity and spatial correlations with surrounding instances [ 44 ] . While prior grounding frameworks encode spatial controls through generic learnable tokens [ 28 , 56 ] , this implicit spatial learning struggles in satellite scenes containing many small, overlapping instances. Furthermore, generic tokens fail to account for the varying degrees of structural detail across different input formats. To address this, we propose a conditioning mechanism that injects explicit geometric cues tailored to the complexity of each input primitive. By directly modeling geospatial primitives, which serve as the foundational basis of remote sensing annotations, our framework is also capable of versatile synthetic data generation. Rather than requiring separate, task-specific generative pipelines, our single unified framework inherently supports a wide variety of downstream tasks. Our contributions are summarized as follows: • We introduce TerraDiT- \mathrm{\Omega} , a unified generative framework that directly operates on any geospatial primitive (polygons, polylines, boxes, and points), enabling state-of-the-art controllable synthesis across conditioning formats. • We propose Geometry-Aware Local Attention , a conditioning mechanism that explicitly preserves geometric cues in vector primitives, improving fidelity in densely structured scenes. • Through our single architecture, we demonstrate higher downstream performance across 4 RS tasks via synthetic data augmentation: land-cover segmentation, object detection, road graph extraction, and scene classification. 2 Related Works Spatially Controlled Image Generation. Spatially controlled image synthesis guides generation using layout-based conditions [ 12 , 66 , 28 , 56 , 23 , 31 , 50 , 55 , 62 , 69 , 73 , 13 , 2 , 15 ] . Make-A-Scene [ 13 ] , SpaText [ 2 ] , and ControlNet [ 66 ] offer finer-grained spatial control, conditioning on inputs such as pixel-level segmentation masks. Moving beyond rasters, recent methods explore discrete conditions: GLIGEN [ 28 ] encodes singular formats (e.g., bounding boxes), while InstanceDiffusion [ 56 ] supports mixed inputs (boxes, points, masks, scribbles). Discussion. Despite their success in natural images, these discrete approaches present three major limitations in the RS domain. First, their vision-centric formats stall GeoAI workflows; loading per-instance masks takes 6\times longer than primitives due to the high number of instances. Second, both GLIGEN and InstanceDiffusion use an identical encoding mechanism for different condition types, ignoring the heavily varying complexity in different geospatial primitives. Third, injecting generic tokens via implicit attention mechanisms imposes a representational burden in satellite imagery, which contains dense, oriented objects with complex interdependencies. In contrast, our approach directly ingests geospatial primitives, explicitly providing varying geometric priors tailored to each format. Satellite Image Synthesis. Recent advances in generative modeling for remote sensing largely extend general text-to-image frameworks to satellite imagery [ 63 , 70 , 42 , 30 ] . Unlike previous U-Net based architectures, TerraDiT- \alpha [ 42 ] employs a diffusion transformer [ 38 ] for text-to-image generation in the RS domain. To provide finer-grained control, prior works have explored incorporating metadata (e.g., time or geolocation), rasterized inputs (e.g. segmentation maps or multispectral imagery), or sparse annotations (point prompts) [ 25 , 47 , 43 , 8 , 42 ] . Notably, to lower annotation cost, TerraDiT- \Sigma [ 42 ] sparsely samples point prompts to provide structural guidance. These contextual approaches have demonstrated promising results across downstream applications such as