RF-DETR Object Detection vs YOLOv12 : A Study of Transformer-based and CNN-based Architectures for Single-Class and Multi-Class Greenfruit Detection in Complex Orchard Environments Under Label Ambiguity
Ranjan Sapkota, Rahul Harsha Cheppally, Ajay Sharda, Manoj Karkee · 2025
Read RF-DETR Object Detection vs YOLOv12 : A Study of Transformer-based and CNN-based Architectures for Single-Class and Multi-Class Greenfruit Detection in Complex Orchard Environments Under Label Ambiguity by Ranjan Sapkota, Rahul Harsha Cheppally, Ajay Sharda, Manoj Karkee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-13099.txt.
Introduction
Ranjan Sapkota and co-authors compare two object detection models for green fruit in orchards, in a research paper from 2025. Young green fruit blend with leaves, hide behind each other, and carry ambiguous labels, which makes counting and thinning decisions hard for growers. A fair model comparison under those real conditions was missing.
A custom data set holds single-class and two-class labels for occluded and clear fruit. One model uses a transformer design with global context, and the other a convolutional design tuned for local detail and edge use. The transformer model led on the main accuracy scores for both label setups. The convolutional model led on one stricter score and runs leaner. The study guides orchard vision choice.
Catalog record
| CREATOR | Ranjan Sapkota, Rahul Harsha Cheppally, Ajay Sharda, Manoj Karkee |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-13099
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RF-DETR Object Detection vs YOLOv12 : A Study of Transformer-based and CNN-based Architectures for Single-Class and Multi-Class Greenfruit Detection in Complex Orchard Environments Under Label Ambiguity Ranjan Sapkota, Rahul Harsha Cheppally, Ajay Sharda, Manoj Karkee Abstract This study presents a comprehensive comparison between RF-DETR object detection and YOLOv12 object detection models for greenfruit recognition in complex orchard environments characterized by label ambiguity, occlusion, and background camouflage. A custom dataset was developed featuring both single-class (greenfruit) and multi-class (occluded and non-occluded greenfruits) annotations to assess model performance under real-world conditions. The RF-DETR object detection model, leveraging a DINOv2 backbone with deformable attention mechanisms, excelled in global context modeling, which proved particularly effective for identifying partially occluded or visually ambiguous greenfruits. Conversely, the YOLOv12 model employed CNN-based attention mechanisms to enhance local feature extraction, optimizing it for computational efficiency and edge deployment suitability. In the single-class detection scenarios, RF-DETR achieved the highest mean Average Precision (mAP@50) of 0.9464, showcasing its robust capability to accurately localize greenfruits within cluttered scenes. Despite YOLOv12N achieving the highest mAP@50:95 of 0.7620, RF-DETR object detection model consistently outperformed in managing complex spatial scenarios. In multi-class detection, RF-DETR again led with an mAP@50 of 0.8298, demonstrating its effectiveness in distinguishing between occluded and non-occluded fruits, whereas YOLOv12L topped the mAP@50:95 metric with 0.6622, indicating superior classification under detailed occlusion conditions. The analysis of model training dynamics revealed RF-DETR’s rapid convergence, particularly in single-class scenarios where it plateaued at fewer than 10 epochs, underscoring the efficiency and adaptability of transformer-based architectures to dynamic visual data. These results confirm RF-DETR’s suitability for accuracy-critical agricultural tasks, while YOLOv12 remains ideal for speed-sensitive deployments. Journal: Elsevier Keywords: Object Detection, , RF-DETR object detection model (Roboflow Detection Transformer), , YOLOv12 object detection model, , You Only Look Once, , Transformers, , CNNs, , Greenfruit Detection, , Deep Learning , , Machine-Vision 1 Introduction Figure 1: Classification of object detection methodologies: Top features state-of-the-art CNN-based and Transformer-based methods, widely adopted; Vision Language Models are emerging. Also includes Hybrid, Sparse Coding, and Traditional Feature-based approaches. As illustrated in Figure 1 , the field of object detection over the past decade, propelled by breakthroughs in deep learning, have shifted from basic pattern recognition to sophisticated systems capable of complex image understanding. The object detection approaches can be branched into six primary methodologies as illustrated in Figure 1 , each with unique strengths and applications in various domains of technology and automation. This evolution is vital for overcoming common visual recognition challenges in fields requiring high precision and adaptability, such as autonomous driving [ 1 , 2 ] , healthcare [ 3 ] , security surveillance [ 4 ] , and notably in agriculture [ 5 , 6 ] , where accurate and efficient object detection supports advancements like automated field monitoring [ 7 ] and robotic harvesting [ 8 ] . The six approaches depicted in Figure 1 are Convolutional Neural Networks (CNNs) [ 9 ] , Transformer-based models [ 10 , 11 ] , Vision Language Model-based approaches [ 12 , 13 ] , Hybrid models such as RetinaMask and EfficientDet [ 14 , 15 ] , Sparse Coding and Dictionary Learning models, and Traditional Feature-based approaches. Among them, CNNs, including the YOLO (You Only Look Once) series [ 16 , 17 ] and R-CNN (Region-based CNN) family such as Mask R-CNN [ 18 ] , have become staples in practical deployments due to their proficient handling of spatial hierarchies. Transformer-based models like DETR (Detection Transformer) such as dynamic DETR [ 19 ] and deformable DETR [ 20 ] utilize self-attention mechanisms to treat images as sequences of patches, which helps in integrating a global context and eliminates the need for Non-Maximum Suppression (NMS) [ 21 ] , streamlining post-processing [ 22 ] . Vision Language Models (VLMs) such as CLIP (Contrastive Language-Image Pre-training) represent an emerging area that integrates textual and visual data, aiming to enhance robustness through multimodal learning, though their application in real-world scenarios, particularly in robotics and automation, is still developing. On the other hand, Hybrid models such as RetinaMask, Sparse Coding models like Online Dictionary Learning, and Traditional Feature-based methods such as Histogram of Oriented Gradients (HOG) are increasingly considered obsolete [ 23 , 24 ] . These have been superseded by more advanced systems that provide not only greater accuracy but also the capability to perform in real-time, a critical requirement for latency-sensitive operations such as those found in modern agricultural settings. As object detection continues to evolve, the focus remains on technologies that combine high precision with efficient processing, positioning CNN and Transformer-based models as the current state-of-the-art in the field. Figure 2: CNN vs Transformer-based model performance comparison focusing on YOLOv12 (CNN-based) and RF-DETR (Transformer-based) architectures: (a) RF-DETR object detection model benchmark evaluation with YOLO11, YOLOv8 and other DETR-based object detection models ; (b)RF-DETR evaluation on the RF100-VL dataset, highlighting domain adaptability and edge deployment potential. ; and (c) Performance overview of recent CNN-based models, includ- ing YOLOv6 through YOLOv12, Gold-YOLO, RT-DETR, RT-DETRv2, and YOLO-MS. b) RF-DETR benchmark results on the MS COCO dataset, surpassing 60% mAP Among the six primary object detection approaches illustrated in Figure 1 , CNN-based and Transformer-based models have emerged as the most widely adopted and actively developed over the past five years. These two paradigms now dominate both research and real-world applications due to their scalability, accuracy, and adaptability. This ongoing dominance has sparked a competitive evolution between the two approaches, particularly with the release of powerful Transformer-based models such as RF-DETR, developed by Roboflow. RF-DETR integrates architectural innovations from Deformable DETR and LW-DETR, and utilizes a DINOv2 backbone, offering superior global context modeling and domain adaptability. The model eliminates reliance on anchor boxes and Non-Maximum Suppression (NMS), supporting end-to-end training and real-time inference. With two variants, Base (29M) and Large (128M), RF-DETR offers scalability from edge deployment to high-performance scenarios. The model has been shown to outperform YOLOv11 and is the only model to surpass 60% mAP on the COCO dataset to date. Its performance on COCO and RF100-VL benchmarks is visualized in Figure 2 a and 2 b. However, despite its promise, RF-DETR has not yet been officially benchmarked against YOLOv12, the latest and most advanced model in the YOLO family. A comparative evaluation is warranted, particularly since YOLOv12 builds upon the strengths of YOLOv11, YOLOv10, and Gold-YOLO RT-DETR, as illustrated in Figure 2 c. 1.1 CNN-based Object Detection Approaches CNNs have been pivotal in advancing object detection, significantly since AlexNet catalyzed the field in 2012 [ 25 ] . These networks utilize hierarchical layers of convolutions, pooling, and nonlinear activations to effectively learn feature representations from images [ 26 ] . Unlike transformers that handle global relationships via attention mechanisms, CNNs are structured to excel at extracting local features, which is facilitated by their inherent inductive biases such as translation equivariance and the establishment of spatial hierarchies [ 27 ] . This fundamental architectural distinction makes CNNs particularly well-suited for scenarios demanding real-time processing and edge computing deployments, albeit with a noted limitation in their ability to model global contextual information comprehensively [ 28 , 29 ] . The progress in CNN architectures for object detection has been