Evaluating Few-Shot Pill Recognition Under Visual Domain Shift
Wai Ip Chu, Giacomo Tarroni, Ling Li · 2026
Read Evaluating Few-Shot Pill Recognition Under Visual Domain Shift by Wai Ip Chu, Giacomo Tarroni, Ling Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10833.txt.
Introduction
Wai Ip Chu, Giacomo Tarroni, and Ling Li evaluate few-shot pill recognition under visual domain shift, from a deployment perspective. Automated pill recognition aims to cut adverse drug events, yet real scenes bring clutter, overlapping pills, and reflections. Their two-stage detector gets base training, then few-shot fine-tuning with one, five, or ten labeled examples per new class, and is tested on a separate cluttered dataset. Semantic recognition adapted fast, saturating with even a single example. Under overlap and occlusion, though, localization and recall fell sharply.
Catalog record
| CREATOR | Wai Ip Chu, Giacomo Tarroni, Ling Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39827 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.10833
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Evaluating Few-Shot Pill Recognition Under Visual Domain Shift Wai Ip Chu, Giacomo Tarroni, Ling Li Abstract Adverse drug events are a significant source of preventable harm, which has led to the development of automated pill recognition systems to enhance medication safety. Real-world deployment of these systems is hindered by visually complex conditions, including cluttered scenes, overlapping pills, reflections, and diverse acquisition environments. This study investigates few-shot pill recognition from a deployment-oriented perspective, prioritizing generalization under realistic cross-dataset domain shifts over architectural innovation. A two-stage object detection framework is employed, involving base training followed by few-shot fine-tuning. Models are adapted to novel pill classes using one, five, or ten labeled examples per class and are evaluated on a separate deployment dataset featuring multi-object, cluttered scenes. The evaluation focuses on classification-centric and error-based metrics to address heterogeneous annotation strategies. Findings indicate that semantic pill recognition adapts rapidly with few-shot supervision, with classification performance reaching saturation even with a single labeled example. However, stress testing under overlapping and occluded conditions demonstrates a marked decline in localization and recall, despite robust semantic classification. Models trained on visually realistic, multi-pill data consistently exhibit greater robustness in low-shot scenarios, underscoring the importance of training data realism and the diagnostic utility of few-shot fine-tuning for deployment readiness. I Introduction Adverse drug events (ADEs) remain a significant source of preventable patient harm, particularly when patients manage complex medication regimens involving multiple concurrent prescriptions [ 3 ] . Such scenarios create challenging visual environments for computational systems, as medications must be identified under conditions that differ substantially from controlled acquisition setups. Pills are often stored together in dosette boxes or pill organizers, resulting in cluttered scenes with overlapping objects and reflective surfaces [ 4 ] . These conditions introduce occlusions, ambiguous boundaries, and visual artefacts, which present substantial challenges for conventional computer vision systems developed under idealized assumptions. Pill recognition is a fine-grained visual classification and detection task characterized by high intra-class similarity and low inter-class variability. Many pills share similar shapes, colors, and surface textures, making their distinction challenging even under controlled imaging conditions. In real-world scenarios, these challenges are exacerbated by partial visibility, specular reflections, and background variation [ 17 ] . Although modern deep learning models can achieve high accuracy with large, well-curated datasets, assembling such datasets in healthcare contexts is often costly and logistically complex [ 8 ] . These constraints motivate the adoption of few-shot learning, which adapts models using only a small number of labeled examples and has become increasingly relevant for medical and healthcare-related vision tasks. Few-shot learning has been investigated for pill recognition through both metric-based and meta-learning paradigms. Metric-based methods construct embedding spaces that enable recognition of novel classes using similarity measures, whereas meta-learning approaches optimize models for rapid adaptation across diverse tasks. Previous research, including studies utilizing datasets such as CURE, has demonstrated promising results with these techniques under controlled conditions [ 9 ] . However, these evaluations are typically performed with training and test data from closely related distributions, often involving isolated pills, limited background variation, and minimal object interaction [ 13 ] . Consequently, reported performance primarily reflects within-domain generalization and may overstate robustness in practical deployment scenarios [ 10 ] . More broadly, domain generalization and cross-dataset evaluation are recognized as significant challenges in computer vision, especially for safety-critical applications [ 12 ] . Although several studies have investigated dataset bias and performance degradation under distribution shift [ 16 ] , systematic cross-dataset evaluation is uncommon in few-shot object detection and remains largely unexplored for pill recognition [ 11 ] . Most existing few-shot pill recognition studies do not evaluate how models trained on visually homogeneous data adapt to environments with clutter, occlusion, and multiple object interactions, even though these conditions are central to real-world medication handling [ 10 ] . This study examines few-shot pill recognition from a deployment-oriented perspective, emphasizing generalization under realistic domain shift rather than proposing new architectures or benchmarks. The goal is not to establish state-of-the-art few-shot detection performance, but to interrogate robustness, failure modes, and supervision requirements under conditions representative of real-world deployment. Our investigation reveals the following key contributions: • We demonstrate that few-shot adaptation enables reliable semantic pill recognition under severe cross-dataset domain shift. Foreground classification accuracy saturates even in the 1-shot regime, despite substantial differences in scene structure, annotation style, and visual complexity. This shows that modern detectors learn transferable semantic representations for pills that generalize beyond the distributions assumed in prior few-shot benchmarks. • We provide evidence that training data realism, rather than dataset size or shot count alone, is a dominant factor in few-shot generalization under clutter and occlusion. Models base-trained on visually realistic, multi-pill data consistently exhibit higher robustness and lower failure rates in overlapping scenes, revealing a strong interaction between base-domain visual statistics and low-shot adaptation behavior. • We identify a systematic failure mode in few-shot pill recognition, whereby semantic classification remains robust while localization and recall degrade sharply under heavy overlap. This decoupling, revealed through classification-centric and error-based metrics, highlights limitations that are obscured by conventional average-precision-focused evaluations and clean benchmark settings. • We establish few-shot fine-tuning as a diagnostic tool for deployment readiness, showing that varying supervision levels expose stability-robustness trade-offs and diminishing returns beyond intermediate shot counts. This reframes few-shot learning not only as a data-efficient adaptation strategy but also as a method for probing domain sensitivity and supervision requirements in real-world pill recognition systems. Together, these contributions provide critical insight into the role of data realism and evaluation protocol in the practical design and assessment of pill recognition systems for healthcare deployment. II Methods Automated pill recognition systems are increasingly proposed for integration into healthcare workflows such as medication dispensing, adherence monitoring, and safety verification [ 5 ] . In practical settings, however, annotated training data are often scarce, and test conditions may differ substantially from those observed during model development. These discrepancies include the presence of multiple pills per image, pill overlap, visually cluttered backgrounds, reflections from pill organizers, and non-uniform lighting. Such conditions challenge conventional object detection pipelines trained under controlled imaging assumptions [ 7 ] . This study explores few-shot learning (FSL) as a deployment-oriented adaptation and evaluation tool for pill recognition under realistic imaging conditions. Instead of comparing datasets based on absolute performance, the analysis examines how a pre-trained object detector adapts to novel pill classes with only a small number of labeled examples and evaluates this adaptation under the visual complexity characteristic of real-world medication handling. II-A Datasets Two datasets, CURE and MEDISEG, are employed for base training. These datasets differ substantially in visual complexity and annotation granularity. The study intentionally leverages these differences to investigate the impact of base-domain realism on few-shot adaptation under domain shift. CURE is a large-scale pill recognition dataset comprising 8,973 images across 196 pill classes. Each image depicts a single pill instance photographed under controlled conditions, with minimal background clutter and no object overlap [ 9 ] . Annotations are provided as full-image bounding boxes, ensuring that each image corresponds to a single pill instance. Consequently, CURE emphasizes fine-grained appearance cues in visually homogeneous settings but does not expose models to multi-object interactions or occlusion during base training.