Beyond 2D Matching: A Unified Single-Stage Framework for Geometry-Aware Cross-View Object Geo-Localization

Liyao Wang *, Ruipu Wu *, Haojun Xu *, Lei Shi, Linjiang Huang †, Si Liu · 2026

Read Beyond 2D Matching: A Unified Single-Stage Framework for Geometry-Aware Cross-View Object Geo-Localization by Liyao Wang *, Ruipu Wu *, Haojun Xu *, Lei Shi, Linjiang Huang †, Si Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30576.txt.

Introduction

Liyao Wang and five colleagues work on cross-view object geo-localization, finding a ground-level or drone target inside a satellite image. Older methods match appearances in images and are held back by datasets lacking geometry details, varied prompts and normal camera views. The team built CMA-Loc, a building dataset with over two hundred twenty thousand ground-satellite and drone-satellite pairs.

Their framework, GAGeo, builds on a three-dimensional foundation model that treats image order symmetrically. It fuses visual features, prompts given as points, boxes or masks, and task tokens in one pass. That pass predicts bounding boxes, segmentation masks and camera poses together. A contrastive loss, which pulls matching views together and pushes others apart, uses the satellite view as a shared anchor. This enables zero-shot ground-to-drone matching with no three-way training data, and tests beat prior methods on unseen scenes.

Catalog record

CREATORLiyao Wang *, Ruipu Wu *, Haojun Xu *, Lei Shi, Linjiang Huang †, Si Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30576

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Beyond 2D Matching: A Unified Single-Stage Framework for Geometry-Aware Cross-View Object Geo-Localization Liyao Wang *, Ruipu Wu *, Haojun Xu *, Lei Shi, Linjiang Huang †, Si Liu Abstract Cross-view object geo-localization (CVOGL) aims to locate a target object from a query view (e.g., ground or drone) within a geo-tagged reference image (e.g., satellite). Existing approaches heavily rely on 2D appearance matching and are constrained by limited datasets lacking geometric metadata, diverse prompts, and standard field-of-view imagery. To address these intertwined challenges, we first introduce CMA-Loc, a large-scale, high-fidelity building dataset comprising over 220,000 ground-satellite and drone-satellite pairs. It provides multi-modal prompts (points, boxes, masks) and camera poses to enable flexible target referring and explicit spatial modeling. Furthermore, we propose a novel single-stage Geometry-Aware Geo-localization framework (GAGeo), built upon the permutation-equivariant 3D foundation model π 3 \pi^{3} . By seamlessly integrating visual features, referring prompts, and learnable task tokens, our model adapts the inherited 3D prior to jointly predict bounding boxes, segmentation masks, and camera poses in a single forward pass. Additionally, we introduce a contrastive loss that utilizes the satellite view as a universal anchor, implicitly aligning ground and drone representations to enable zero-shot ground-to-drone localization without requiring triplet training data. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods, exhibiting exceptional generalization ability in unseen scenes and novel cross-view setups. * * footnotetext: Equal contribution. † † footnotetext: Corresponding author. Figure 1 : Overview of CMA-Loc dataset and GAGeo framework. We introduce CMA-Loc, a large-scale building dataset for advancing cross-view geo-localization, featuring 224K instance pairs and 180K images across 77.2K locations. It incorporates multi-view imagery, multi-modal prompts, and camera pose to simulate complex real-world scenarios. Leveraging CMA-Loc, we propose GAGeo, a geometry-aware cross-view object geo-localization framework that adapts a 3D Foundation Model backbone with task-specific decoders in a single-stage manner, achieving state-of-the-art performance in drone-to-satellite, ground-to-satellite, and zero-shot ground-to-drone setups. 1 Introduction In real-world applications ranging from disaster monitoring [ 12 , 39 ] to drone navigation [ 26 ] , there is a growing need to ground specific target objects onto geo-tagged reference imagery (e.g., satellite views), thereby decoupling spatial positioning from reliance on external signals. Inherently, this task requires building a spatial bridge that projects local, ego-centric observations onto a global coordinate system, seamlessly shifting the paradigm from recognizing “what” is in the scene to precisely localizing “where” it resides in the physical world. Driven by this practical need, researchers have curated various cross-view datasets, evolving from coarse-grained image-level retrieval [ 38 , 24 , 46 , 44 , 5 ] to recent object-centric datasets [ 32 , 40 ] . However, current research is severely hindered by coupled limitations in both existing datasets and methodologies. On the data side, existing resources rely exclusively on panoramic images that introduce severe geometric distortions, failing to reflect the standard field-of-view (FoV) of practical cameras. Furthermore, they are restricted to simple point-coordinate prompts, lack geometric metadata (e.g., camera positions and rotations), and suffer from limited scale (merely 12,000 pairs for CVOGL [ 32 ] ) and scene variability. Beyond these data deficiencies, prevailing research conventionally treats CVOGL as a 2D matching problem. Consequently, most existing frameworks [ 36 , 14 , 29 , 30 , 32 , 6 , 40 ] rely on appearance-based cues, failing to fully exploit the inherent 3D geometric structure. Lacking such 3D awareness, these models struggle to bridge the spatial gap, leaving them vulnerable to perceptual ambiguity and visual inconsistencies, which limit their robust generalization. To tackle these intertwined challenges, we propose a comprehensive framework comprising a high-fidelity building dataset, CMA-Loc , and a novel Geometry-Aware GEO-localization approach GAGeo . First, to bridge the data gap, CMA-Loc is constructed via two tailored pipelines. To construct ground-satellite pairs, we collect imagery from Google Street View [ 1 ] and Google Earth [ 10 ] . We then associate OpenStreetMap-derived [ 11 ] satellite footprints with SAM3-generated [ 2 ] ground masks based on geometric cues like orientation and scale. For drone-satellite pairs, we utilize a Cesium rendering pipeline to deterministically extract and pair cross-view masks via unique building IDs. Finally, to improve boundary quality across the dataset, all non-SAM-generated masks are refined using SAM3. From these refined masks, we derive bounding boxes and inner point prompts as additional referring prompts, followed by rule-based filtering to reduce low-quality or information-sparse cases. Crucially, CMA-Loc also provides accurate camera poses for each pair, unlocking the capability for explicit spatial modeling and geometric supervision. Through this rigorous construction pipeline, we ultimately curate a large-scale dataset comprising 112,063 ground-satellite and 111,704 drone-satellite instance pairs across eight global cities, resolving the data scale and scene diversity constraints of prior works. Additionally, in CMA-Loc’s test set, we incorporate a manually annotated subset, which comprises ground-drone-satellite triplets from University-1652 [ 44 ] , serving as a dedicated dataset to evaluate models’ zero-shot and cross-domain generalization. To overcome the limitations of pure 2D appearance matching, our methodology shifts the paradigm by harnessing the powerful geometric priors embedded in 3D foundation models [ 33 , 25 , 43 ] . Specifically, we adopt \pi^{3} [ 35 ] as our framework’s backbone. A primary motivation for this choice is that \pi^{3} processes input views in a permutation-equivariant manner; it establishes a shared representation space without anchoring to any initial frame’s perspective, thereby reducing viewpoint biases. Leveraging this property, we propose a unified single-stage framework capable of processing both ground-to-satellite and drone-to-satellite inputs. The model first fuses visual tokens from a frozen DINOv2 [ 27 ] with high-fidelity prompt tokens from a SAM2-pretrained encoder. Unlike previous paradigms that rely on cumbersome two-stage networks for detection and segmentation (e.g., TROGeo [ 40 ] ) or append heavy, isolated prediction heads to a backbone, our design is remarkably streamlined. We seamlessly integrate learnable tokens and prompt tokens as additional inputs for \pi^{3} , augmenting its alternating local and global attention layers with a specialized masking scheme designed for multi-task decoding. Through a single forward pass, this single-stage structure yields diverse multi-task predictions, including bounding boxes, segmentation masks, camera position and rotation. Furthermore, since collecting perfectly aligned ground-drone-satellite triplets in the real world is notoriously difficult, we introduce a contrastive learning objective optimized purely on available pairs. Conceptually akin to ImageBind [ 9 ] , which binds diverse modalities to a shared latent space using images as the central anchor, we utilize the satellite view as a universal intermediate bridge. Benefiting from the view-agnostic nature of \pi^{3} , aligning mask-pooled object representations within ground-satellite and drone-satellite pairs implicitly pulls the ground and drone representations closer together. This strategy circumvents the reliance on scarce triplet training data, unlocking the capability for zero-shot ground-to-drone geo-localization. Extensive experiments demonstrate that our framework sets a new state-of-the-art in the cross-view object geo-localization task, significantly outperforming existing approaches. Notably, in the ground-to-satellite setup, our method achieves a significant improvement of 34.38% mAcc on the object detection task and 33.36% mIoU on the object segmentation task. Meanwhile, it also exhibits robust generalization to unseen dataset and novel ground-to-drone setup, evidently outperforming previous best-performing methods. Furthermore, comprehensive ablation studies validate the effectiveness of each proposed strategy. 2 Related Works Cross-View Image Geo-Localization. Early cross-view geo-localization formulated

Read the complete text