CityGen: Structure-Guided City-Style Synthesis for Cross-City Autonomous Driving
Zezhong Qian 1,2∗ , Zhao Yang 1,2∗ , Lu Tan 3 , Zhihao Yan 4 , Weiyi Hong 2 , Haizhuang Liu 4 , Yawei Jueluo 1 · 2026
Read CityGen: Structure-Guided City-Style Synthesis for Cross-City Autonomous Driving by Zezhong Qian 1,2∗ , Zhao Yang 1,2∗ , Lu Tan 3 , Zhihao Yan 4 , Weiyi Hong 2 , Haizhuang Liu 4 , Yawei Jueluo 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29935.txt.
Introduction
Zezhong Qian and six colleagues tackle self-driving models that falter in a new city, where looks, road shapes, and traffic shift. They offer CityTransfer-Bench, a test set split by place that checks sensing, segmentation, and planning across cities. Their CityGen tool adapts a model to a fresh city with no labeled local data, shaping scenes from HD maps plus city-level visual prompts. Tests show steady gains in cross-city toughness across tasks. The pair gives a label-light base for driving systems that must work anywhere.
Catalog record
| CREATOR | Zezhong Qian 1,2∗ , Zhao Yang 1,2∗ , Lu Tan 3 , Zhihao Yan 4 , Weiyi Hong 2 , Haizhuang Liu 4 , Yawei Jueluo 1 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28168 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.29935
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CityGen: Structure-Guided City-Style Synthesis for Cross-City Autonomous Driving Zezhong Qian 1,2∗ , Zhao Yang 1,2∗ , Lu Tan 3 , Zhihao Yan 4 , Weiyi Hong 2 , Haizhuang Liu 4 , Yawei Jueluo 1 Abstract Autonomous driving systems are commonly trained and evaluated within limited geographic regions, which hinders their scalability when deployed in new cities. However, significant domain shifts in appearance, road topology, and traffic patterns often cause severe performance degradation under cross-city deployment. Existing approaches based on domain adaptation, data augmentation, or synthetic data generation typically rely on labeled target data, city-specific annotations, or task-specific designs, limiting their scalability and effectiveness for holistic evaluation. In this paper, we introduce CityTransfer-Bench, a geographically disjoint benchmark for evaluating cross-city generalization across perception, segmentation, and planning, and propose CityGen, a diffusion-based generative framework that performs zero-label city adaptation via HD-map–conditioned synthesis guided by city-level visual prompts. Extensive experiments demonstrate that CityGen consistently improves cross-city robustness across multiple tasks, establishing a scalable and label-efficient foundation for generalizable autonomous driving. I Introduction Autonomous driving systems are typically trained and evaluated within limited geographic regions, such as specific cities [ 25 , 12 , 32 ] . However, this often leads to significant performance degradation when deployed in unseen cities, due to domain shifts in road layouts, visual appearance, traffic patterns, and environmental conditions. Achieving robust cross-city generalization is therefore a critical challenge for scalable and reliable autonomous driving systems. Despite the growing scale and diversity of autonomous driving datasets, systematic evaluation frameworks for assessing and improving model transferability across cities remain limited. Fig. 1: CityGen pipeline and evaluation protocol. Previous methods train models on City A, causing performance degradation on City B. CityGen synthesizes City-B-style images using HD-map-guided generative modeling, improving cross-city generalization in detection, segmentation, and planning. Existing methods have attempted to address cross-city generalization through domain adaptation techniques, data augmentation, and synthetic data generation. However, these methods often rely on either labeled target data or domain-specific annotations, making them less scalable and more expensive. Some methods attempt to align feature distributions between cities through adversarial training or domain-specific fine-tuning [ 6 , 5 ] , but these techniques still struggle with large-scale, real-world cross-city generalization due to inherent differences in traffic patterns, road structures, and environmental conditions. Additionally, most existing methods focus primarily on single-task performance (e.g., perception or decision-making) without a unified framework that evaluates multi-task generalization across different urban domains. Designing a benchmark for cross-city generalization is challenging due to urban differences in both low-level visual cues (such as illumination, texture, and architecture) and high-level semantics (such as road topology, lane geometry, and local driving conventions). While existing methods, such as domain adaptation and synthetic data generation [ 33 , 15 ] , attempt to address these issues, they often require extensive annotations or rely on style transfer that may not maintain semantic consistency across cities. Manual annotation for each city is prohibitively expensive, and these methods typically lack scalability across diverse urban environments. Therefore, a scalable and label-efficient framework is essential for evaluating and improving model robustness across cities [ 18 , 20 , 29 ] . To address these challenges, we introduce CityTransfer-Bench , the first benchmark specifically designed to evaluate cross-city generalization in autonomous driving. Built on the nuScenes dataset, it uses a geographically disjoint split, training on data from three Singapore regions (One-North, Queenstown, and Holland Village) and testing in the Boston Seaport area. The benchmark evaluates three key tasks—perception, segmentation, and decision-making—providing a unified protocol for assessing city-level transferability. In conjunction with this benchmark, we propose CityGen , a diffusion-based generative framework for zero-label city adaptation. CityGen uses DiT to synthesize realistic urban scenes conditioned on HD-map layouts and city-specific visual prompts [ 17 , 16 , 19 ] . This enables semantic-preserving style transfer from Singapore to Boston without additional annotations, making it a plug-and-play solution for improving cross-city robustness. Our main contributions are as follows: (1) We present CityTransfer-Bench , the first benchmark for evaluating city-level generalization across perception, segmentation, and planning tasks in autonomous driving. (2) We propose CityGen , a diffusion-based generative framework that performs zero-label city adaptation through HD-map-conditioned synthesis guided by city-level visual prompts. (3) We conduct large-scale experiments showing that the data generated by CityGen significantly improves downstream robustness on unseen cities, establishing a scalable foundation for future research in cross-domain autonomous driving. II Related work Cross-Domain Generalization in Autonomous Driving. Cross-domain generalization is a long-standing challenge in autonomous driving due to distribution shifts across environments, cities, and weather conditions. Prior work mainly addresses this issue through domain adaptation and domain generalization techniques, particularly for perception tasks such as detection and semantic segmentation [ 2 , 22 , 4 ] . Representative approaches include adversarial feature alignment, output-space adaptation, and cross-domain data mixing [ 9 , 24 , 10 , 11 ] . Several benchmarks evaluate robustness under domain shifts such as weather, illumination, and synthetic-to-real transfer [ 3 , 1 , 23 ] . However, most studies rely on labeled target-domain data, focus on single tasks, or use mixed-domain splits without strict city separation, limiting systematic evaluation of city-level generalization across autonomous driving tasks. Generative Data Synthesis for Autonomous Driving. Recent advances in generative modeling enable realistic driving scene synthesis as an alternative to conventional data augmentation. Early approaches rely on simulation or image translation, while recent methods employ diffusion or autoregressive models for high-fidelity multi-view generation [ 26 , 27 , 8 , 30 ] . Many works incorporate structural conditioning (e.g., layouts, geometry, text prompts) to improve controllability and consistency [ 14 , 7 ] . Generative world models are also explored for long-horizon prediction and planning [ 13 ] . However, most approaches emphasize realism or simulation, with limited study on cross-city generalization. In contrast, our work leverages structure-guided synthesis and evaluates it under a unified geographically disjoint cross-city benchmark. Fig. 2: CityGen framework. A vision–language model extracts scene captions from target-city multi-view images to form a text-conditioning set. CityGen synthesizes multi-view urban scenes by denoising noise with DiT blocks while injecting structural controls via conditional transfer blocks, producing coherent multi-view images aligned with target-city appearance and layout. III Method III-A Preliminary Diffusion-based Video Generation with Structural Control. Given a video \mathbf{X}=\{x^{t}\}_{t=1}^{T} , we use a spatiotemporal autoencoder to map it to latent tokens \mathbf{Z}_{0}\in\mathbb{R}^{T\times C\times H\times W} . A forward diffusion process progressively adds noise to \mathbf{Z}_{0} , producing noisy latents \mathbf{Z}_{t} , while a denoising transformer \epsilon_{\theta} is trained to predict the injected noise, conditioned on a signal \mathbf{c} : \mathcal{L}_{\mathrm{diff}}=\mathbb{E}_{\mathbf{Z}_{0},\boldsymbol{\epsilon},t}\left\|\boldsymbol{\epsilon}-\epsilon_{\theta}(\mathbf{Z}_{t},t\mid\mathbf{c})\right\|_{2}^{2}. (1) The signal \mathbf{c} is injected via cross-attention, guiding the denoising process across both spatial and temporal dimensions. To enforce structural constraints, we introduce a structural control branch in the diffusion model [ 31 ] . For the \ell -th transformer block, the backbone features \mathbf{h}^{\mathrm{main}}_{\ell} are modulated by control features \mathcal{C}_{\ell}(\mathbf{M}) derived from the structure input \mathbf{M} : \mathbf{h}^{\mathrm{final}}_{\ell}=\mathbf{h}^{\mathrm{main}}_{\ell}+\mathcal{C}_{\ell}(\mathbf{M}). (2) This structure-preserving modulation enables the generation of outputs that maintain geometric fidelity while retaining the expressiveness of the pretrained diffusion model, yielding the final sampling operator: \hat{\mathbf{X}}=\mathcal{G}(\mathbf{M},\mathbf{c}). (3) III-B Structure-Guided Geometric Control via Multi-view HD-map Projection As illustrated in Fig. 2 , CityGen tackles cross-city data synthesis by