PiLoT v2: Pixel-to-Orthogonal Map Alignment for Free-view UAV Geo-localization

Xinyi Liu, Xiaoya Cheng, Rouwan Wu, Zhaochen Wang, Shen Yan, Maojun Zhang, Yu Liu, [4pt] National University of Defense Technology {liuxinyi24, chengxy, wurouwan97, wangzhaochen, yanshen12, mjzhang, jasonyuliu}@nudt.edu.cn · 2026

Read PiLoT v2: Pixel-to-Orthogonal Map Alignment for Free-view UAV Geo-localization by Xinyi Liu, Xiaoya Cheng, Rouwan Wu, Zhaochen Wang, Shen Yan, Maojun Zhang, Yu Liu, [4pt] National University of Defense Technology {liuxinyi24, chengxy, wurouwan97, wangzhaochen, yanshen12, mjzhang, jasonyuliu}@nudt.edu.cn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31098.txt.

Introduction

Liu and six colleagues present PiLoT v2, a system that tells a drone where it is when satellite navigation is blocked or lost. The earlier PiLoT matched drone video against a single rendered view from a 3D mesh. That worked well but needed heavy meshes. Storage, map capture, and rendering costs all blocked use on small onboard computers.

PiLoT v2 matches pixels directly against flat map products instead: true digital orthophoto maps and digital surface models. Cropping a map is far cheaper than rendering a 3D scene. A cross-view network bridges the gap between flat map crops and slanted drone views. Gravity direction and a single-point laser range feed into the pose calculation as extra anchors. Tests show accuracy matching or beating the older system, at far lower storage and compute cost.

Catalog record

CREATORXinyi Liu, Xiaoya Cheng, Rouwan Wu, Zhaochen Wang, Shen Yan, Maojun Zhang, Yu Liu, [4pt] National University of Defense Technology {liuxinyi24, chengxy, wurouwan97, wangzhaochen, yanshen12, mjzhang, jasonyuliu}@nudt.edu.cn
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.31098

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PiLoT v2: Pixel-to-Orthogonal Map Alignment for Free-view UAV Geo-localization Xinyi Liu, Xiaoya Cheng, Rouwan Wu, Zhaochen Wang, Shen Yan, Maojun Zhang, Yu Liu, [4pt] National University of Defense Technology {liuxinyi24, chengxy, wurouwan97, wangzhaochen, yanshen12, mjzhang, jasonyuliu}@nudt.edu.cn Abstract Real-time, drift-free UAV geo-localization is essential for autonomous missions in GNSS-denied environments. The pioneering system, PiLoT, achieves high precision via Neural Pixel-to-3D Registration, aligning UAV video streams with a single rendered reference view from 3D meshes. However, its reliance on heavy 3D meshes incurs massive storage overheads, complex map acquisition, and significant computational rendering costs, severely hindering deployment on embedded platforms. To address these bottlenecks, we propose PiLoT v2, a lightweight yet robust evolution that shifts the paradigm to direct pixel-to-orthogonal map registration for free-view UAV geo-localization. By leveraging True Digital Orthophoto Maps (TDOMs) and Digital Surface Models (DSMs) as the reference substrate, PiLoT v2 replaces GPU-intensive 3D rendering with a highly efficient, CPU-friendly map cropping operation. To bridge the severe geometric discrepancy between these 2.5D orthogonal crops and free-view oblique UAV imagery, we train a cross-view feature registration network using a novel, large-scale geometrically annotated dataset. Furthermore, we integrate onboard sensor priors—specifically gravity direction and single-point laser range—directly into the pose optimization manifold to enhance robustness against cross-view visual degradation. Experimental results demonstrate that PiLoT v2 achieves performance comparable to, or even exceeding, its Pixel-to-3D predecessor, while offering drastically lower storage and computational costs. I Introduction Fig. 1: Revisiting of sequence localization. PiLoT v2 estimates globally aligned 6-DoF UAV poses from UAV sequences using lightweight DOM+DSM map crops. The radar chart shows that PiLoT v2 achieves a favorable balance for real-time onboard geo-localization in GNSS-denied environments, where FRC denotes failure recovery count. Unmanned Aerial Vehicle (UAV) geo-localization aims to estimate globally aligned 6-DoF poses for each frame in a UAV query sequence. Drift-free geo-localization is a fundamental prerequisite for pushing UAVs towards full autonomy in demanding real-world applications, such as digital twins, infrastructure inspection, augmented reality, and embodied AI [ 21 , 4 ] . This requirement becomes especially critical when GNSS signals are degraded or lost, where the last reliable GNSS/GPS estimate before signal loss can provide only a coarse initialization. Introducing global map priors provides an effective way to anchor UAV trajectories to absolute geographic coordinates and eliminate accumulated drift. Recent high-accuracy UAV localization systems, such as Render2Loc [ 32 ] and PiLoT [ 6 ] , formulate geo-localization as a Neural Pixel-to-3D Registration problem over geo-referenced 3D meshes. To maintain real-time performance, these methods typically align the video stream with a single rendered reference view. However, these 3D meshes demand massive storage (often >10 GB) and depend on costly oblique photogrammetry followed by time-intensive 3D reconstruction. Furthermore, online reference generation relies on computationally expensive rendering engines, placing a heavy burden on embedded GPU resources. An alternative line of work investigates lightweight orthographic maps, such as True Digital Orthophoto Maps (TDOMs) and Digital Surface Models (DSMs), as replacements for heavy 3D mesh representations [ 15 , 7 ] . These 2.5D maps are easier to acquire, straightforward to update, and significantly more compact. However, bridging the severe cross-view geometric discrepancies between pure orthographic maps and free-view oblique UAV images remains challenging. Most existing methods address this by relying on costly explicit image-to-map matching, making it difficult to support real-time sequential UAV geo-localization. To break this performance bottleneck, we propose PiLoT v2 . As visualized in Fig. 1 , PiLoT v2 revisits the dual-thread localization paradigm of its predecessor and replaces the heavy 3D mesh representation with a lightweight TDOM+DSM map. Crucially, we shift the paradigm by replacing the exorbitant 3D rendering module with a highly efficient, CPU-friendly map cropping operation. As a result, PiLoT v2 preserves the real-time dual-thread structure for UAV geo-localization while being vastly more suitable for resource-constrained onboard deployment. Using 2D orthographic crops inevitably introduces a massive cross-view gap when compared with oblique UAV query images [ 37 , 34 ] . To overcome this, we train a cross-view feature registration network using a newly constructed, large-scale synthetic dataset with precise geometric annotations. This enables the network to extract highly consistent features despite drastic viewpoint differences, demonstrating strong zero-shot generalization to real-world UAV-to-orthophoto queries. Furthermore, built-in UAV sensor priors, specifically gravity direction and single-point laser range, are readily available during flight [ 26 , 8 , 33 ] . To mitigate visual degradation and improve robustness against unstable cross-view registration, we incorporate these physical constraints directly into the Levenberg-Marquardt (LM) based pose refinement manifold. Extensive experiments show that our method matches or even outperforms SoTA 3D-mesh-based methods, while using a drastically simplified TDOM+DSM map prior. In summary, our main contributions are: • We propose PiLoT v2, a real-time UAV geo-localization pipeline that replaces costly 3D rendering with lightweight TDOM+DSM map cropping, dramatically reducing storage and computational overhead. • We train a robust feature registration network using large-scale geometrically annotated cross-view data, bridging the modality gap between orthographic map crops and oblique UAV images. • We integrate onboard gravity direction and single-point laser range priors directly into the LM-based pose optimization manifold to ensure robust cross-view registration. • We construct a comprehensive TDOM+DSM benchmark aligned with corresponding 3D oblique meshes. Evaluations demonstrate that our lightweight method matches or surpasses SoTA 3D-mesh-based systems for real-time edge deployment. II Related Work II-A UAV Geo-localization UAV geo-localization aims to estimate the 6-DoF pose of a UAV camera in a global coordinate frame. Since simultaneous localization and mapping (SLAM) [ 4 ] and visual-inertial odometry (VIO) [ 21 ] accumulate drift without global references, many methods register UAV images against geo-referenced maps. Early methods mainly rely on 2D satellite imagery [ 20 , 22 ] . These methods can provide coarse global localization, but they are usually limited to 2D or 3-DoF pose estimation under simplified orthographic assumptions. To recover full 6-DoF poses, recent works turn to simulation-driven references and geo-referenced 3D maps [ 36 , 32 , 31 , 6 ] . These methods typically initialize the pose using retrieval [ 11 , 3 ] , sensor priors, or noisy pose estimates, and then refine it through feature matching [ 25 , 18 , 38 ] , render-and-compare, or direct alignment [ 19 , 14 , 12 ] . However, they usually require dense 3D reconstruction, large map storage, and online rendering to generate viewpoint-consistent references. In contrast, our work achieves real-time sequential UAV geo-localization using lightweight TDOM/DSM crops instead of rendered 3D references. II-B Lightweight Maps Representations To reduce the cost of 3D-mesh-based localization, recent studies explore lightweight map priors, including topographic maps [ 5 ] , orthographic imagery and orthophotos [ 16 ] , LoD city models [ 39 ] , and 2.5D aerial references such as TDOM/DSM geodata [ 7 , 35 ] . LoD-Loc [ 39 ] estimates UAV pose from coarse LoD maps, avoiding heavy textured reconstruction but still relying on projected 3D geometry and coarse sensor initialization. OrthoLoC [ 7 ] further shows that TDOM/DSM can support UAV 6-DoF geo-localization, while highlighting the projection discrepancy between perspective UAV images and orthographic references. AnyVisLoc [ 35 ] further highlights the difficulty of low-altitude multi-view UAV geo-localization under 2.5D aerial and satellite references. These works demonstrate the

Read the complete text