Joint Multi-Condition Representation Modelling via Matrix Factorisation for Visual Place Recognition
2025
Read Joint Multi-Condition Representation Modelling via Matrix Factorisation for Visual Place Recognition (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17739.txt.
Introduction
Timur Ismagilov and five co-authors improve how machines recognize places from photos. Their task uses several reference photo sets taken under changing conditions. Light, weather, and viewpoint shifts can make one place look different. Deep learning helps but costs heavy training and computing power.
Their method needs no training and works with any image descriptor. It splits place data through matrix factorization, a way of factoring data into simpler parts. Matching then compares leftover errors after projection onto those parts. Recall improved by up to about eighteen percent on appearance-change tests. The team also released a new multi-viewpoint benchmark called SotonMV.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-17739
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Joint Multi-Condition Representation Modelling via Matrix Factorisation for Visual Place Recognition Thanks: *Corresponding author. Affiliation: Timur Ismagilov*, Shakaiba Majeed, Michael Milford, Tan Viet Tuyen Nguyen Thanks: Timur Ismagilov, Tan Viet Tuyen Nguyen, Sarvapali D. Ramchurn and Shoaib Ehsan are with the School of Electronics and Computer Science, University of Southampton, SO17 1BJ Southampton, U.K. [email protected], [email protected], [email protected], [email protected] Thanks: Shakaiba Majeed is with the Department of Computer Science and Engineering, Hanyang University, Seoul, South Korea. [email protected] Thanks: Michael Milford is with the QUT Centre for Robotics, School of Electrical Engineering and Robotics, Brisbane, QLD 4000, Australia. [email protected] Affiliation: Sarvapali D. Ramchurn, Shoaib Ehsan Thanks: Shoaib Ehsan is also with the School of Computer Science and Electronic Engineering, University of Essex, Colchester, CO4 3SQ, U.K. [email protected] Abstract We address multi-reference visual place recognition (VPR), where reference sets captured under varying conditions are used to improve localisation performance. While deep learning with large-scale training improves robustness, increasing data diversity and model complexity incur extensive computational cost during training and deployment. Descriptor-level fusion via voting or aggregation avoids training, but often targets multi-sensor setups or relies on heuristics with limited gains under appearance and viewpoint change. We propose a training-free, descriptor-agnostic approach that jointly models places using multiple reference descriptors via matrix decomposition into basis representations, enabling projection-based residual matching. We also introduce SotonMV, a structured benchmark for multi-viewpoint VPR. On multi-appearance data, our method improves Recall@1 by up to ≈ \approx 18% over single-reference and outperforms multi-reference baselines across appearance and viewpoint changes, with gains of ≈ \approx 5% on unstructured data, demonstrating strong generalisation while remaining lightweight. I INTRODUCTION Visual Place Recognition (VPR) is a fundamental capability for robotic systems, autonomous vehicles and Simultaneous Localisation and Mapping (SLAM) systems, enabling robust localisation and loop-closure detection to support map building and correct global drift. At its core, VPR matches live sensor observations against a database of previously visited locations, relying on topological cues without external positioning systems like GNSS. Fig. 1: Top : Each place is represented by multiple images with different conditions, whose descriptors are stacked and subspace is formed using matrix factorisation, here QR decomposition, to construct the map. Bottom : At inference, the query is projected and reconstructed using each subspace, with the smallest residual determining the match. VPR is commonly framed as an image retrieval problem, where state-of-the-art (SOTA) methods learn to extract distinctive descriptors and perform matching based on similarity. However, long term operation is impacted by perceptual aliasing from appearance variation such as weather, illumination and seasonal change, as well as viewpoint shifts and dynamic scenes, all of which can compromise matching performance. To improve performance, recent works have proposed more robust feature extractors [ 1 , 2 , 3 ] , introduced viewpoint-aware loss functions [ 4 ] , and used large-scale, diverse datasets for training [ 5 , 6 ] . While absolute accuracy has improved, broader training data increases computational cost, and larger models like MegaLoc [ 6 ] become impractical for edge deployment due to heavy backbones and large descriptor representations that dominate memory and latency. Furthermore, VPR in practise often assumes a single reference set, typically collected under favourable daylight conditions and with known camera headings, thereby constraining the range of query viewpoint at inference. Alternatively, strategies exist to improve robustness by providing multiple observations of locations at retrieval time, such as memorising past experiences [ 7 ] , voting across candidate references [ 8 ] , sequence-based matching [ 9 ] and fusion techniques that aggregate information from multiple embeddings or sensors [ 10 , 11 , 12 ] . For this work, we focus on the multi-reference setting, where places are represented by multiple observations under varying conditions. Despite their efficiency, existing methods treat these views independently or combine them with simple heuristics, leaving approaches that exploit multi-reference information largely under explored. We propose a unified mapping and matching framework that jointly incorporates multiple reference sets at match time. Using matrix decomposition, we construct basis representations that enable inference across diverse conditions without retraining, while remaining lightweight and descriptor-backbone agnostic. Our contributions are as follows: • Decomposition-based mapping: We represent each place by stacking its multi-condition descriptors and applying QR decomposition to obtain an orthonormal basis that captures shared structure across viewpoint and appearance variations, illustrated in Fig. 1 . This mapping is training-free, efficient, and supports both pre-computed and on-the-fly updates with minimal overhead. • Projection-based matching: At retrieval time, queries are projected into each place’s basis, with the residual reconstruction error serving as a similarity metric, selecting the place with the minimum residual. • SotonMV: A structured multi-view benchmark for VPR captured in Southampton, UK, with multiple images per place at regularly sampled orientations. This enables controlled evaluation of multi-view map representations and classic VPR with inference from challenging, arbitrary viewpoints. We also assess whether orientation can be inferred from descriptors alone. • Comprehensive Evaluation and Analysis: We evaluate on multi-appearance, viewpoint, and unstructured datasets, consistently improving Recall@1 over existing multi-reference methods. We analyse the effect of descriptor size and SVD rank, demonstrating that our approach retains high performance while maintaining low memory and computation. In real-world settings, VPR systems, like in SLAM and global localisation, must operate under arbitrary viewpoints, traversals, appearance conditions and sparse, uneven map coverage, while remaining computationally efficient. The ability to recall and fuse multiple relevant cues and perspectives across varied conditions into place representations provides a powerful and general foundation for robust localisation. This motivates such methods, enabling general applicability under real-world conditions while maintaining low overhead. The rest of the paper is organised as follows. Section II provides an overview of the related work. Section III outlines our proposed framework of mapping and retrieval. SotonMV is detailed in Section IV . Section V covers our experimental setup. Results are presented in Section VI . Finally, conclusions, limitations and future work are discussed in Section VII . II RELATED WORK Visual Place Recognition. Retrieval-based VPR typically converts images to distinct feature-level embeddings, which are then matched using distance metrics. Early methods relied on handcrafted local features like SIFT [ 13 ] , SURF [ 14 ] and ORB [ 15 ] , which are aggregated into global representations using techniques like VLAD [ 16 ] and bag-of-words (BoW) [ 17 ] . Such methods have since been surpassed with the emergence of deep learning, learning more distinctive and robust representations under complex appearance changes. NetVLAD [ 18 ] combines VLAD aggregation with a CNN backbone; MixVPR [ 19 ] uses pre-trained convolutional backbones and channel mixers to effectively create image embeddings; Eigenplaces [ 4 ] focuses on viewpoint invariance with lateral and frontal loss functions. Several methods consider sequences of images during matching [ 9 , 20 ] , exploiting temporal differences in short image sequences to improve confidence. GeoWarp [ 21 ] , uses a learnt pairwise image homography warping for match reranking, improving performance under little visual overlap. Despite advances in deep architectures, VPR remains vulnerable to severe appearance changes, limited viewpoint, and dynamic scenes [ 22 ] . Recent methods [ 5 , 6 ] address these challenges through network design or large-scale training strategies, but less so