Performance-guided Reinforced Active Learning for Object Detection

Zhixuan Liang, Xingyu Zeng, Rui Zhao, Ping Luo · 2026

Read Performance-guided Reinforced Active Learning for Object Detection by Zhixuan Liang, Xingyu Zeng, Rui Zhao, Ping Luo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-15688.txt.

Introduction

Zhixuan Liang, Xingyu Zeng, Rui Zhao, and Ping Luo improve active learning for object detection in a 2026 arXiv paper. Active learning trains strong models with minimal labeling effort. It picks the most informative examples for humans to annotate. Most methods judge informativeness by data distribution or content alone. That judgment does not directly track downstream performance. In detection, the score that matters is mean average precision.

The authors propose performance-guided reinforced active learning, per the title. The guide is the task metric itself, not a proxy. Reinforcement learning steers which images get picked for labeling. If picks track real gains, annotation budgets stretch further. Labeling images for detection is slow and costly. A smarter picker saves money on every training round.

Catalog record

CREATORZhixuan Liang, Xingyu Zeng, Rui Zhao, Ping Luo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-15688

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Performance-guided Reinforced Active Learning for Object Detection Zhixuan Liang, Xingyu Zeng, Rui Zhao, Ping Luo Abstract Active learning (AL) strategies aim to train high-performance models with minimal labeling efforts, only selecting the most informative instances for annotation. Current approaches to evaluating data informativeness predominantly focus on the data’s distribution or intrinsic information content and do not directly correlate with downstream task performance, such as mean average precision (mAP) in object detection. Thus, we propose Performance-guided ( i.e. mAP-guided) Reinforced Active Learning for Object Detection (MGRAL), a novel approach that leverages the concept of expected model output changes as informativeness. To address the combinatorial explosion challenge of batch sample selection and the non-differentiable correlation between model performance and selected batches, MGRAL skillfully employs a reinforcement learning-based sampling agent that optimizes selection using policy gradient with mAP improvement as reward. Moreover, to reduce the computational overhead of mAP estimation with unlabeled samples, MGRAL utilizes an unsupervised way with fast look-up tables, ensuring feasible deployment. We evaluate MGRAL’s active learning performance on detection tasks over PASCAL VOC and COCO benchmarks. Our approach demonstrates the highest AL curve with convincing visualizations, establishing a new paradigm in reinforcement learning-driven active object detection. † † address: ⋆ The University of Hong Kong † SenseTime Research 1 Introduction While recent years have seen significant progress in model architectures, attention is increasingly shifting towards more efficient data utilization strategies. Among them, Active Learning (AL) stands out by training high-performance models with minimal labeling, particularly when field data arrives continuously and annotation is costly. By strategically selecting and annotating the most informative samples, active learning substantially improves data efficiency Figure 1: Overview of Performance-guided (mAP-guided) Reinforced Active Learning (MGRAL) for object detection. The RL-based agent learns to select most informative samples directly using performance gains ( \Delta\text{mAP} ) as reward. Therefore, the central question becomes how to identify most informative data. Prior methods select the data points that are complementary to the currently labeled set in feature space, no matter characterized by distributional coverage or intrinsic “information”. Although various definitions of this measure have been proposed and utilized as query strategies [ 23 , 1 , 24 , 25 , 21 , 17 , 6 , 22 , 16 ] , they do not align directly with downstream metrics, such as mean Average Precision (mAP) in object detection. EMOC [ 8 ] addresses this by measuring expected model output change, which matches our objective, but it was developed for Gaussian process regression and is not readily applicable to deep learning architectures. In this paper, we propose Performance-guided (mAP-guided) Reinforced Active Learning (MGRAL) for object detection, directly using mAP changes to guide data sampling. However, different from EMOC not using deep learning downstream task model, our optimization pipeline must handle discrete, non-differentiable batch selection from the unlabeled pool, which makes it unable to update the selected samples through gradient from downstream metrics straightly. We therefore employ a reinforcement learning-based sampling agent that uses mAP variation ( \Delta mAP) as the reward to enable optimizing the selection process through policy gradient technique. This approach achieves efficient exploration of possible batch combinations, maximizing mAP improvement per selected batch. Because candidate samples are unlabeled when estimating potential mAP changes, we substitute an unsupervised surrogate for the detector to approximate downstream performance. It is effective as we only need to capture the relative influence among candidate data batches instead of correct labels to guide the active learning process. To mitigate the high cost made by reinforcement learning (RL)-based agent of retraining the whole detector from scratch once per RL iteration, we introduce a fast lookup-table accelerator. Empirical results demonstrate our AL method’s efficiency on both Pascal VOC [ 7 ] and MS COCO [ 13 ] across two detector backbones. Visualizations further show that selection criteria should vary across training stages, and only downstream performance provides a reliable signal. Our contributions can be summarized as: (1) We propose MGRAL, extending the concept of expected model output changes for deep active learning, directly aligning the sampling strategy with mAP improvement. (2) We introduce a reinforcement learning-based agent for efficient batch selection, addressing the combinatorial explosion and non-differentiable challenge in optimizing from mAP improvement. (3) We develop practical techniques for MGRAL, and evaluate it on multiple downstream benchmarks and different backbones, revealing our method’s superior performance and the key effect that expected model output change played in it. 2 Related Works 2.1 Active Learning for Object Detection Earliest attempts in active learning for object detection include LL4AL [ 23 ] adapting instance loss predictions and AL4DeepDetection [ 2 ] combining uncertainty metrics for foreground objects and background pixels. CDAL [ 1 ] then enhances sample representativeness through spatial context, while MIAL [ 24 , 20 ] employs adversarial classifiers and an unsupervised framework. EBAL [ 21 ] integrates uncertainty and diversity but faces challenges with computational complexity and class imbalance. More recent works including ComPAS [ 16 ] , MEH+HUA [ 17 ] , MSS [ 6 ] and PPAL [ 22 ] focus more on efficient sample uncertainty definition and provides new solutions to batch selection problem. However, these methods do not directly align with the task model’s performance. Our work, instead, directly utilizes mAP improvement to guide the selection process, addressing the limitations of previous approaches in balancing various metrics and handling batch selection effectively. 2.2 Reinforcement Learning in Active Learning Reinforcement learning has been adopted to learn better query strategies in active learning, aiming to maximize task model performance. Techniques include imitation learning [ 14 ] , bi-directional RNNs [ 5 ] , Deep Q-Networks [ 11 , 4 ] , and policy gradient for simultaneous learning of data representation and selection heuristics [ 3 ] have been utilized to address data representativeness and develop efficient sampling strategies. However, these methods struggle with integrating multi-instance uncertainty within a single image and batch sample selection for object detection tasks. Our approach addresses these challenges by leveraging \Delta\text{mAP} as the reward to optimize the batch sample selection strategy. 3 Methodology 3.1 MGRAL Active Learning Pipeline As depicted in Fig. 1 , to handle the non-differentiable link between sample selection and downstream performance changes, we embed a reinforcement learning-based sampling agent into classical pool-based active learning framework, optimizing selections for mAP improvement. Training. MGRAL uses a nested optimization scheme: the outer RL loop trains the sampler; the inner detector loop estimates mAP gains for each RL step. For each AL round (cycle t ), [Inner Loop] we build a lookup table by training multiple detector variants in parallel (Sec. 3.5 ). The results are pre-recorded and serve for efficiently approximating performance gains. [Outer Loop] The sampling agent (Sec. 3.2 ) then runs RL iterations ( i ), selecting candidate sets from the unlabeled pool and receiving feedback from the estimated mAP improvements (Sec. 3.3 ). This feedback updates the policy via our stabilized policy-gradient optimization (Sec. 3.4 ). Inference (active selection). The trained agent traverses the unlabeled pool with its sequence-model-based architecture, computing for each image a selection score from image features and historical selection context, and finally chooses the top- B (budget) images for annotation. This design enables MGRAL to efficiently bridge sample selection and

Read the complete text