Positive-First Most Ambiguous: A Simple Active Learning Criterion for Interactive Retrieval of Rare Categories
Kawtar Zaher [email protected], Olivier Buisson [email protected], Alexis Joly [email protected] · 2026
Read Positive-First Most Ambiguous: A Simple Active Learning Criterion for Interactive Retrieval of Rare Categories by Kawtar Zaher [email protected], Olivier Buisson [email protected], Alexis Joly [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-24480.txt.
Introduction
Zaher and co-authors present Positive-First Most Ambiguous, an active learning rule for finding rare classes in large unlabelled image sets. Standard active learning assumes balanced classes and large labelling budgets, which fails in skewed settings like wildlife work. Their rule picks samples near the class boundary, but favours likely positives. This speeds the finding of subtle visual classes while keeping each batch informative. In live search tests, it returns small batches with a high share of positives, unlike methods that oversample negatives.
Catalog record
| CREATOR | Kawtar Zaher [email protected], Olivier Buisson [email protected], Alexis Joly [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45093 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.24480
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Positive-First Most Ambiguous: A Simple Active Learning Criterion for Interactive Retrieval of Rare Categories Kawtar Zaher [email protected], Olivier Buisson [email protected], Alexis Joly [email protected] Abstract Real-world fine-grained visual retrieval often requires discovering a rare concept from large unlabeled collections with minimal supervision. This is especially critical in biodiversity monitoring, ecological studies, and long-tailed visual domains, where the target may represent only a tiny fraction of the data, creating highly imbalanced binary problems. Interactive retrieval with relevance feedback offers a practical solution: starting from a small query, the system selects candidates for binary user annotation and iteratively refines a lightweight classifier. While Active Learning (AL) is commonly used to guide selection, conventional AL assumes symmetric class priors and large annotation budgets, limiting effectiveness in imbalanced, low-budget, low-latency settings. We introduce Positive-First Most Ambiguous (PF-MA), a simple yet effective AL criterion that explicitly addresses the class imbalance asymmetry: it prioritizes near-boundary samples while favoring likely positives, enabling rapid discovery of subtle visual categories while maintaining informativeness. Unlike standard methods that oversample negatives, PF-MA consistently returns small batches with a high proportion of relevant samples, improving early retrieval and user satisfaction. To capture retrieval diversity, we also propose a class coverage metric that measures how well selected positives span the visual variability of the target class. Experiments on long-tailed datasets, including fine-grained botanical data, demonstrate that PF-MA consistently outperforms strong baselines in both coverage and classifier performance, across varying class sizes and descriptors. Our results highlight that aligning AL with the asymmetric and user-centric objectives of interactive fine-grained retrieval enables simple yet powerful solutions for retrieving rare and visually subtle categories in realistic human-in-the-loop settings. 1 Introduction Fine-grained visual discovery in large-scale collections is increasingly important: in real-world scenarios, users seek a specific and visually subtle concept (e.g., a rare species, an object variant, or an unusual visual context) hidden among millions of irrelevant items. This typically induces a severely imbalanced setting where relevant instances may occur at extremely low frequencies (e.g., 1 in 1000 or 1 in 10000). Figure 1 : Comparison of selected samples. MA (left): near-boundary negatives are oversampled. MP (middle): only positives far from the boundary are selected. PF-MA (right): balance between relevant positives and negatives around the boundary. We consider interactive retrieval, where the system progressively constructs a novel user-defined class-of-interest using minimal supervision. The overall interaction loop is shown in Fig. 2 : starting from a small initial query, the system proposes candidate images, the user provides binary Relevance Feedback (RF) 28 , and a lightweight binary classifier refines the target concept over iterations based on the user feedback. The objective is not to train a general-purpose model, but to quickly retrieve relevant and diverse instances of the user-defined class under strict constraints: (i) severe class imbalance (rare positives vs. abundant negatives), (ii) tiny per-iteration annotation budgets, (iii) and low latency between iterations (few seconds). Constraints (ii) and (iii) reflect human limitations and user fatigue: users can only label a few samples and wait a few seconds for the next batch. Figure 2 : Interactive retrieval process. Active Learning (AL) 36 naturally guides sample selection, but conventional AL assumes symmetric class priors and labeling utility, aiming to improve global classification by answering “which samples should be labeled to train the best overall model?” . In interactive fine-grained retrieval, sample utility is inherently asymmetric: querying a predicted-positive sample provides substantially higher expected utility than querying a predicted-negative one, as it instantly improves user satisfaction, particularly under tight annotation budgets. Standard strategies ignore this asymmetry, wasting queries on ambiguous negatives. This creates a tension between informativeness and relevance : uncertainty-focused strategies sample near the decision boundary, naturally crowded with negatives, while confidence-based selection often returns redundant positives with limited training value, as illustrated in Fig. 1 . The key question becomes: ”which samples should be labeled to rapidly and broadly characterize the rare class-of-interest?” . The goal is not to improve general classification, but to rapidly and effectively construct the user’s class-of-interest. To tackle this problem, we introduce Positive-First Most Ambiguous (PF-MA) , a simple AL criterion designed for interactive class retrieval under severe class imbalance and real-world constraints. PF-MA prioritizes ambiguous samples while explicitly favoring likely positives, enabling rapid rare class discovery without sacrificing boundary refinement, and adapting to the asymmetric nature of our problem. This equilibrium, shown in Fig. 1 , ensures that the system consistently presents the user with relevant examples, enhancing early satisfaction and visual confirmation, while also exposing the model to hard negatives that are crucial for learning precise decision boundaries. Moreover, interactive retrieval also requires diversity : users benefit from covering different visual modes of the class (e.g., pose, context, appearance), not only from retrieving more redundant positives. We therefore propose a class coverage metric that evaluates retrieval quality beyond the number of discovered positives, measuring how broadly the retrieved set spans the underlying class manifold. We demonstrate the effectiveness of PF-MA across long-tailed datasets, including fine-grained images, using modern pretrained visual descriptors. Beyond the intrinsic rarity of the user-defined class in our highly imbalanced setting, we leverage long-tailed datasets to model the unknown and variable size of this class, ranging from a handful of instances to several hundreds. Despite its simplicity, PF-MA consistently improves early-stage retrieval coverage and classifier quality across classes of varying sizes. Our results highlight the practical benefits for human-in-the-loop fine-grained retrieval: high positive ratios, diverse retrieval, and rapid user satisfaction with minimal supervision. 2 Related Work Interactive Retrieval and Relevance Feedback Interactive retrieval aims to construct a user-defined class with minimal supervision through an iterative process. Unlike Novel Class Discovery 40 , which identifies all novel categories, interactive retrieval focuses on retrieving a single concept defined by the user. It is related to Few-Shot Learning (FSL) 37 which learns from limited examples, but FSL assumes a fixed label space and optimizes overall classification accuracy. In contrast, interactive retrieval operates without predefined categories and emphasizes retrieving diverse and relevant instances. Content-Based Image Retrieval (CBIR) 26 ; 19 ; 38 is closely related, yet often targets instance-level similarity. Early systems relied on handcrafted features and relevance feedback (RF) to iteratively refine queries 33 ; 12 ; 28 . With deep learning, learned embeddings significantly improved retrieval 5 . Recent works explore interactive learning with deep embeddings and user-in-the-loop refinement 41 . However, these similarity-driven strategies frequently produce near-duplicate suggestions, limiting diversity and training signal. Human-in-the-Loop, Interactive Learning and Active Learning Human-in-the-loop learning incorporates user feedback to refine model predictions 27 . To address the near-duplicate limitation, Active Learning (AL) 36 selects informative samples to reduce annotation effort. Classic AL focuses on improving model performance with minimal labeling using uncertainty 23 ; 34 ; 39 , diversity 10 ; 43 , or representativeness strategies such as CoreSet methods 35 and unlabeled data modeling 35 ; 17 . These approaches are primarily designed for supervised classification with fixed label spaces and balanced datasets, and typically assume symmetric labeling utility across classes, disregarding the class-of-interest, while optimizing global accuracy. Recent deep AL methods 24 further rely on repeated retraining or stochastic forward passes, introducing additional computational overhead, unfeasible in our low latency setting. In retrieval settings, AL has been explored to guide interactive feedback