AddressVLM: Cross-view Alignment Tuning for Image Address Localization using Large Vision-Language Models
Shixiong Xu, Chenghao Zhang, Lubin Fan, Yuan Zhou, Bin Fan, Shiming Xiang, Gaofeng Meng, Jieping Ye · 2025
Read AddressVLM: Cross-view Alignment Tuning for Image Address Localization using Large Vision-Language Models by Shixiong Xu, Chenghao Zhang, Lubin Fan, Yuan Zhou, Bin Fan, Shiming Xiang, Gaofeng Meng, Jieping Ye (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10667.txt.
Introduction
Xu and a team including Zhang, Fan, and Zhou teach image models to find a street address from a street photo in this twenty twenty-five paper. Big image-language models can name a town or land, but they often fail at the exact street spot inside a city. Street photos alone give only small local clues.
Inside, the method adds sky-view maps as wide clues and trains the model to match the two views. The team built question-and-answer sets from street photos in Pittsburgh and San Francisco to test it. Their model beat rival models by wide gaps on both sets. The work points to maps as a strong aid for fine place finding.
Catalog record
| CREATOR | Shixiong Xu, Chenghao Zhang, Lubin Fan, Yuan Zhou, Bin Fan, Shiming Xiang, Gaofeng Meng, Jieping Ye |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-10667
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AddressVLM: Cross-view Alignment Tuning for Image Address Localization using Large Vision-Language Models Shixiong Xu, Chenghao Zhang, Lubin Fan, Yuan Zhou, Bin Fan, Shiming Xiang, Gaofeng Meng, Jieping Ye Abstract Large visual language models (LVLMs) have demonstrated impressive performance in coarse-grained geo-localization at the country or city level, but they struggle with fine-grained street-level localization within urban areas. In this paper, we explore integrating city-wide address localization capabilities into LVLMs, facilitating flexible address-related question answering using street-view images. A key challenge is that the street-view visual question-and-answer (VQA) data provides only microscopic visual cues, leading to subpar performance in fine-tuned models. To tackle this issue, we incorporate perspective-invariant satellite images as macro cues and propose cross-view alignment tuning including a satellite-view and street-view image grafting mechanism, along with an automatic label generation mechanism. Then LVLM’s global understanding of street distribution is enhanced through cross-view matching. Our proposed model, named AddressVLM , consists of two-stage training protocols: cross-view alignment tuning and address localization tuning. Furthermore, we have constructed two street-view VQA datasets based on image address localization datasets from Pittsburgh and San Francisco. Qualitative and quantitative evaluations demonstrate that AddressVLM outperforms counterpart LVLMs by over 9% and 12% in average address localization accuracy on these two datasets, respectively. keywords Image address localization, Vision-language model, Cross-view alignment † † equal-contributors: These authors contributed equally to this work. † † equal-contributors: These authors contributed equally to this work. 1 Introduction Visual place recognition (VPR) aims to predict the geographic location of a given image, which can be categorized into two types: image geo-localization ( Arandjelovic et al. 2016 ; Wang et al. 2022 ; Ali-Bey et al. 2023 ) and image address localization ( Xu et al. 2024 ) . The emergence of Large Vision-Language Models (LVLMs), such as GPT-4V ( Achiam et al. 2023 ) , Qwen-VL ( Bai et al. 2023 ) , and LLaVA ( Liu et al. 2024 ) , have significantly impacted various tasks related to images and languages. As generative models capable of generating natural language, they demonstrate enhanced adaptability and flexibility in the image localization task ( Yang et al. 2023 ) . This proficiency stems from the extensive exposure to street-view and landmark images during their training phases. Recent work, GeoReasoner ( Li et al. 2024 ) , integrates a large vision-language model with human inference knowledge for street view geo-localization with reasoning, presenting significant advantages in coarse-grained localization at the country or city level. However, when it comes to address localization for specific districts ( i.e. , Downtown) or streets ( i.e. , Fifth Avenue) within a city, it may struggle to predict accurate textual address, since street-view images are more similar and difficult to distinguish and the street-level address names have not been adequately correlated with the corresponding street-view images. In contrast, the previous work AddressCLIP ( Xu et al. 2024 ) explores city-wide address localization by contrastive learning between street-view images and textual address. Nevertheless, this approach is inherently limited due to its reliance on a discriminative model that can only make distinctions among a constrained set of candidate addresses. As a result, it lacks the flexibility to provide versatile address descriptions and answer other related inquiries. Figure 1: Comparison of our AddressVLM with AddressCLIP and GeoReasoner. Our approach focuses on city-wide image address localization and flexible address questions and answers related to address using large vision-language models. To combine the advantages of previous work, in this study, we explore how to integrate street-level address localization capabilities into an LVLM. The model is expected to respond flexibly to user inquiries about address localization. We name our model AddressVLM , which is designed to handle address-related questions and provide answers accurate to the district and street level. Fig. 1 shows the comparisons of the proposed AddressVLM with AddressCLIP and GeoReasoner. Our method can answer various types of questions including generation, judgment, and multiple-choice. To realize the above goal, a reasonable approach involves fine-tuning a well-trained LVLM using street-view question-and-answer (VQA) data with LoRA adaptation ( Hu et al. 2021 ) . However, this straightforward method of address localization tuning yields suboptimal performance. The primary reason is that street-view images are sparsely collected in terms of both location and viewpoint, which inhibits the model’s ability to build a global understanding of street distribution across an entire city. Such global information is crucial for effective address localization since street-view images are densely sampled during testing. To supplement the global information in fine-tuning, we introduce perspective-invariant satellite images to establish connections between sparse street-view images. Satellite images are globally consistent and exhibit overlap, allowing for a mapping of the sparse street-view images to a global framework that facilitates inter-image correlations. Previous research in cross-view geo-localization ( Durgam et al. 2024 ) has shown the viability of correlating satellite images with street-view images. In light of this, we propose a method named cross-view alignment tuning , designed to enable LVLMs to align street-view images with street addresses on satellite images annotated with street name labels. This method integrates a global understanding of street distributions within urban environments into LVLMs. It consists of two key components: the satellite-view and street-view image grafting mechanism and the automatic alignment label generation mechanism. The former places street-view images in the upper right corner of their corresponding regional satellite images, serving as the input for cross-view alignment tuning. The latter employs an off-the-shelf LVLM to explain why the street-view image matches the address in the satellite images according to the provided address hint, thus automatically generating labels for the cross-view alignment tuning. By doing this, our full method involves two-stage training protocols: cross-view alignment tuning and address localization tuning. We introduce two city-wide street-view VQA datasets named Pitts-VQA and SF-Base-VQA, built upon the Pitts-IAL ( Torii et al. 2013 ; Xu et al. 2024 ) and SF-Base-IAL ( Berton et al. 2022 ; Xu et al. 2024 ) datasets, respectively. On Pitts-VQA, AddressVLM demonstrates an improvement of 9% compared to the baseline without cross-view alignment tuning. On SF-Base-VQA, AddressVLM achieves an improvement of 12% over the baseline. Moreover, in comparison to the state-of-the-art (SOTA) approach for image address localization using LVLMs, GeoReasoner ( Li et al. 2024 ) , our method exhibits improvements of 11% and 14% on the Pitts-VQA and SF-Base-VQA datasets, respectively. The proposed method exhibits excellent city-wide address localization capability compared to general LVLMs. We further provide qualitative results to thoroughly validate the effectiveness of the proposed cross-view alignment tuning strategy. Additional quantitative experiments show that our method can be extended to address localization in multiple cities. The contributions of this work are summarized as follows: (1) We explore integrating city-wide address localization capabilities into LVLMs to enable flexible address question and answer based on street-view images. (2) We introduce cross-view alignment tuning that integrates the global understanding of urban street distribution into LVLMs, which includes the cross-view image grafting mechanism and the automatic alignment label generation mechanism. (3) We propose AddressVLM that achieves consistent improvements over the baseline and performs superior to the SOTA method GeoReasoner and general LVLMs. 2 Related Work Visual Place Recognition. Visual place recognition aims to predict the geographic location of a given image with