RELO: Reinforcement Learning to Localize for Visual Object Tracking

Xin Chen, Chuanyu Sun, Jiao Xu, Houwen Peng, Dong Wang, Huchuan Lu, Kede Ma · 2026

Read RELO: Reinforcement Learning to Localize for Visual Object Tracking by Xin Chen, Chuanyu Sun, Jiao Xu, Houwen Peng, Dong Wang, Huchuan Lu, Kede Ma (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07379.txt.

Introduction

Xin Chen and six co-authors teach visual object trackers to localize targets through reinforcement learning instead of handcrafted heatmaps. Conventional trackers rely on spatial priors that give only surrogate supervision, poorly aligned with the overlap and success measures used to judge tracking. RELO reframes localization as a decision process over positions.

A learned policy picks locations, rewarded by frame-level overlap and sequence-level success together. A token propagation method keeps meaning consistent across frames at negligible cost. Across benchmarks, RELO beat rivals and reached a strong success score on a hard text-guided dataset without template updates, confirming reward-driven localization as a real alternative.

Catalog record

CREATORXin Chen, Chuanyu Sun, Jiao Xu, Houwen Peng, Dong Wang, Huchuan Lu, Kede Ma
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07379

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RELO: Reinforcement Learning to Localize for Visual Object Tracking Xin Chen, Chuanyu Sun, Jiao Xu, Houwen Peng, Dong Wang, Huchuan Lu, Kede Ma Abstract Conventional visual object trackers localize targets using handcrafted spatial priors, often in the form of heatmaps. Such priors provide only surrogate supervision and are poorly aligned with tracking optimization and evaluation metrics, such as intersection over union (IoU) and area under the success curve (AUC). Here, we introduce RELO, a REinforcement-learning-to-LOcalize method for visual object tracking that formulates target localization as a Markov decision process. Specifically, RELO replaces handcrafted spatial priors with a localization policy learned over spatial positions via reinforcement learning, with rewards combining frame-level IoU and sequence-level AUC. We additionally introduce layer-aligned temporal token propagation to improve semantic consistency across frames, with negligible computational overhead. Across multiple benchmarks, RELO achieves superior results, attaining 57.5 % 57.5\% AUC on LaSOT ext without template updates. This confirms that reward-driven localization provides an effective alternative to prior-driven localization for visual object tracking. https://github.com/Multimedia-Analytics-Laboratory/RELO 1 Introduction (a) Prior-driven localization learning (b) RL-based localization learning Figure 1 : Comparison of target localization learning paradigms in visual object tracking. (a) Conventional trackers learn target localization from handcrafted spatial priors. (b) RELO replaces handcrafted priors with an RL-based localization policy, optimized using task-oriented rewards. Red arrows indicate the signals used to optimize the localization module. Visual object tracking aims to estimate the state (mainly the location) of a target throughout a video given its initialization in the first frame ( 47 ) . As a fundamental problem in computer vision, it underpins a wide range of applications, including surveillance, autonomous systems, robotics, and human-computer interaction. Recent progress has been driven by discriminative tracking paradigms, especially Siamese-based methods ( 23 ; 2 ; 9 ; 44 ) and one-stream Transformer-based trackers ( 52 ; 61 ; 7 ; 26 ) , which localize the target by learning dense spatial representations over the search region. Despite their strong empirical performance, most of these trackers still rely on handcrafted spatial priors for surrogate supervision, typically in the form of center-based heatmaps that represent target likelihood (see Fig. 1 (a)). Several earlier Transformer-based approaches ( 10 ; 38 ; 6 ; 15 ) follow a similar idea in a less explicit manner, by using expectation-based corner localization. Although effective in practice, such prior-driven localization strategies have an important conceptual limitation: the supervisory signal is only indirectly related to the actual goal of tracking. Handcrafted center or corner distributions encourage the tracker to match a manually specified spatial pattern, rather than directly improving the metrics used to evaluate tracking performance, such as intersection over union (IoU) and area under the success curve (AUC). In addition, the design of these priors introduces extra assumptions that are not intrinsic to tracking. For example, some trackers adopt binary masks ( 9 ) and Gaussian-smoothed heatmaps ( 52 ) , and the resulting performance can depend on design choices that are largely heuristic. These observations motivate a different question: can a tracker learn where to localize the target directly from task-oriented feedback, without relying on handcrafted spatial priors? In this paper, we answer this question affirmatively by introducing RELO , a RE inforcement-learning-to- LO calize method for target localization in visual tracking. As shown in Fig. 1 (b), RELO formulates target localization as a Markov decision process over the spatial feature map. Each spatial location is treated as a candidate action, the regression head predicts a corresponding bounding box for every action, and a localization policy learns to select promising actions. Instead of fitting the localization module to surrogate labels, RELO optimizes the policy directly using task-oriented rewards that combine frame-level IoU and sequence-level AUC. In this way, localization behavior emerges from the tracking objective itself, rather than from manually designed supervision. RELO offers two key advantages. First, it aligns training more closely with evaluation by directly optimizing rewards that reflect tracking quality. Second, it removes the need for handcrafted spatial priors, allowing the tracker to discover effective localization behavior through reward-driven exploration. To make this reinforcement learning (RL) process stable and effective, we introduce a regression warmup stage that equips the tracker with basic box regression capability before policy optimization. We additionally incorporate a layer-aligned temporal token propagation strategy to improve semantic consistency across frames with negligible computational overhead. Extensive experiments on multiple benchmarks demonstrate that RELO achieves state-of-the-art performance on large-scale tracking benchmarks while maintaining real-time inference speed on GPU. In particular, RELO-L256 attains 75.1\% AUC on LaSOT ( 14 ) without template updates and 57.5\% AUC on the more challenging LaSOT ext benchmark ( 13 ) , showing strong generalization. In summary, our contributions are as follows: • We reformulate target localization in visual tracking as a Markov decision process and instantiate this formulation in RELO, without relying on handcrafted spatial priors. • We introduce a practical training strategy for RELO, including a regression warmup stage and a layer-aligned temporal token propagation mechanism. • We show that RELO achieves strong accuracy and efficiency, validating that directly optimizing task-oriented rewards offers a strong alternative to conventional prior-driven localization. 2 Related Work In this section, we position RELO with respect to the two closely related lines of research: dense-prediction-based and RL-based visual tracking. 2.1 Dense-Prediction-Based Visual Tracking Modern visual object trackers are largely built on discriminative dense prediction, with one-stream Transformer-based trackers emerging as a particularly strong and simple instantiation. These methods typically extract spatial representations using deep visual backbones ( 16 ; 32 ; 58 ) and predict the target state from the resulting search-region feature map. However, a common characteristic is that target localization is usually learned through handcrafted spatial priors rather than directly end-to-end optimized. Center-based localization. Most recent trackers fall into this category ( 50 ; 48 ; 41 ; 29 ; 25 ; 17 ; 60 ; 5 ; 56 ) . They typically combine a classification-style localization branch, which estimates the confidence of target presence at each spatial location, with a regression branch that predicts the final bounding box. The localization branch is supervised by a center-based prior, usually implemented as either a binary target map ( 9 ; 50 ) or a Gaussian-smoothed heatmap centered at the target position ( 52 ; 26 ) . Corner-based localization. Several earlier trackers instead adopt corner-based formulations ( 10 ; 38 ; 6 ; 15 ) . These methods predict probability distributions for the top-left and bottom-right corners, and recover the bounding box by taking expectations over the corresponding spatial distributions. Although this avoids an explicit center heatmap, it still introduces handcrafted geometric bias, since the supervisory signal is defined around annotated corners rather than directly around tracking performance. In contrast, RELO formulates target localization as a Markov decision process and optimizes it with RL using task-oriented rewards, thereby removing manually designed spatial priors and aligning training more closely with testing. 2.2 RL-Based Visual Tracking RL ( 40 ) has been explored in visual object tracking from several perspectives, but rarely as a mechanism that directly drives target localization. Early RL-based trackers ( 53 ; 34 ) treat tracking as iterative bounding box refinement, where an agent selects discrete translation or scale actions to adjust

Read the complete text