ODverse33: Is the New YOLO Version Always Better? A Multi Domain benchmark from YOLO v5 to v11
Tianyou Jiang, Yang Zhong · 2025
Read ODverse33: Is the New YOLO Version Always Better? A Multi Domain benchmark from YOLO v5 to v11 by Tianyou Jiang, Yang Zhong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-14314.txt.
Introduction
Tianyou Jiang and Yang Zhong ask whether each new version of a famous real-time object detector really beats the last. The YOLO family is used everywhere, from farm cameras to operating rooms. Versions now arrive so fast that users cannot test them all.
The pair built a benchmark of thirty-three datasets across eleven domains, from driving to microscopy. They ran versions from the fifth to the eleventh on the same footing. They also summarize what each version actually changed inside. The gains, they find, do not land evenly across domains. A version that wins on street scenes may stall on underwater shots. Their tables give users a grounded way to pick a version. Newer, in short, is not always better.
Catalog record
| CREATOR | Tianyou Jiang, Yang Zhong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-14314
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ODverse33: Is the New YOLO Version Always Better? A Multi Domain benchmark from YOLO v5 to v11 Tianyou Jiang, Yang Zhong Abstract You Look Only Once (YOLO) models have been widely used for building real-time object detectors across various domains. With the increasing frequency of new YOLO versions being released, key questions arise. Are the newer versions always better than their previous versions? What are the core innovations in each YOLO version and how do these changes translate into real-world performance gains? In this paper, we summarize the key innovations from YOLOv1 to YOLOv11, introduce a comprehensive benchmark called ODverse33, which includes 33 datasets spanning 11 diverse domains (Autonomous driving, Agricultural, Underwater, Medical, Videogame, Industrial, Aerial, Wildlife, Retail, Microscopic, and Security), and explore the practical impact of model improvements in real-world, multi-domain applications through extensive experimental results. We hope this study can provide some guidance to the extensive users of object detection models and give some references for future real-time object detector development. † † footnotetext: * Equal contribution and corresponding authors 1 Introduction With the rapid advancement in both artificial intelligence and computer vision, object detection has seen remarkable breakthroughs, garnering considerable attention in recent years [ 55 ] . One of the most influential models in this field is You Only Look Once (YOLO), first introduced by Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi in 2016 [ 34 ] . YOLO quickly gained traction due to its impressive detection accuracy and fast inference speed, making it a leading choice for real-time object detection tasks. Since its inception till January 1, Year 2025, YOLO has evolved through 11 major versions (YOLOv1 to YOLOv11). While the original authors developed YOLOv1 through YOLOv3, subsequent versions (YOLOv4 to YOLOv11) were developed by different teams within the active YOLO community. Despite sharing the core YOLO concepts, many later Figure 1 : Evaluation results of YOLOv5 to YOLOv11. (a) Performance on the COCO validation set (reported in their original projects) and on our ODverse33 validation and test sets. (b) Performance on small, medium, and large objects in the ODverse33 test set. (c) Inference speed per image using a single NVIDIA A100 GPU and number of parameters for each YOLO model, where the size of the circles represents the product of these two metrics. versions integrate innovative techniques and architectural changes, reflecting diverse contributions and advancements in object detection methodologies. Evaluations of object detectors, including the YOLO series, have traditionally focused on the Common Objects in Context (COCO) dataset, where higher YOLO versions have consistently demonstrated improved performance in their original projects or papers [ 24 ] . However, a significant gap remains in understanding how different YOLO versions perform across various domain-specific datasets, making this a topic of strong research interest. Is the new version of YOLO always better than the previous version on different tasks? How should practitioners select the best YOLO version for their specific projects? What are the core improvements in each YOLO version, and how do these changes translate into real-world performance gains? This paper aims to address these questions through a comprehensive analysis and in-depth discussion. While the YOLO series has been widely adopted across various real-world ap-plications, a common misconception still persists that newer YOLO versions are inherently superior, much like the assumption that the latest hardware updates always lead to better performance. However, this is not necessarily the case. Adding a sleek spoiler doesn’t necessarily improve a car’s performance, the effec-tiveness of a YOLO model should depend on its alignment with the task at hand, not just its novelty. While YOLOv1 through YOLOv5 marked significant evolu-tionary milestones, later versions focused on refining existing architectures to further improve model performance and inference efficiency. These refinements, though valuable, may not always translate into robust improvements for broader object detection tasks beyond COCO benchmark. As shown in Figure 1 , our re-sults indicate that post-YOLOv5 versions exhibit fluctuating performance across domain-specific applications, sometimes failing to surpass their predecessors. In this paper, we aim to reveal the evolution of YOLO series models, provide a comprehensive benchmark, offer guidance for those involved in object detection applications, and propose references for the development of future real-time object detectors. To achieve this, we summarize the key innovations introduced in YOLOv1 through YOLOv11 and conduct extensive evaluations of the models from YOLOv5 to YOLOv11. Unlike most existing benchmarks that focus primarily on the COCO dataset, our study spans a variety of domains and conditions. We trained and evaluated YOLO models on 33 datasets, covering a wide range of applications, including aerial imagery, autonomous driving, medical imaging, microscopy, underwater imaging, agriculture, industry, video games, wildlife, retail, and security. This diverse, multidisciplinary dataset collection allows us to assess each YOLO version’s performance across a broad spectrum of real-world challenges. Meanwhile, to ensure a fair and consistent benchmark, we standardized experimental setups, including uniform dataset splits, consistent data augmentation techniques, and the same hyperparameter configurations for each YOLO model. Each model was trained for 300 epochs on each dataset, and their evaluation metrics on the test sets—mAP 50 , mAP 50-95 , mAP {}_{\text{small}} , mAP {}_{\text{medium}} , and mAP {}_{\text{large}} were calculated and compared. We refer to this benchmark as the ODverse33 benchmark, which encompasses 33 object detection datasets across various domains. More details about the benchmark and related information about the datasets and experiments involved in this paper can be found at https://github.com/SkyCol/ODverse33 . 2 Related works Figure 2 : Timeline of YOLO’s development, reflecting the earliest release time of their code repository or pre-print. 2.1 YOLO: You Only Look Once model series YOLO (You Only Look Once) was first introduced by Joseph Redmon and Ali Farhadi et al. in 2015 [ 34 ] , marking a major advancement in real-time object detection by unifying the tasks of bounding box prediction and class probability estimation into a single-stage network. Unlike traditional two-stage detectors like R-CNN and Faster R-CNN [ 12 , 37 ] , YOLO achieves rapid detection speeds by predicting both bounding boxes and class probabilities directly from full images in a single forward pass. A timeline of YOLO’s development is shown in Figure 2 , where the time reflects the earliest release time of their code repository or pre-print. In 2016 and 2018, the original authors introduced YOLOv2 and YOLOv3, respectively, further enhancing YOLO model’s detection capability [ 35 , 36 ] . YOLOv2, also known as YOLO9000, introduced anchor boxes for more precise detection and enabled real-time recognition across over 9000 classes by pre-training its Darknet-19 backbone on a general classification task. During training, it incorporated a multi-scale training approach, which allows the model to adapt to various input sizes, improving versatility. With k-means clustering, YOLOv2 optimized anchor box sizes, while multi-scale training enhances versatility across various input sizes. YOLOv3 improved accuracy by incorporating a deeper architecture (Darknet-53) and multi-scale predictions, which detects objects at three different scales to capture varying sizes more effectively, similar to the Feature Pyramid Networks (FPN) [ 23 ] . In 2020, Alexey Bochkovskiy and his collaborators introduced YOLOv4 [ 2 ] , delivering substantial improvements in architecture and training techniques. YOLOv4 upgraded the backbone to CSPDarknet-53, incorporated Cross Stage Partial (CSP) connections [ 47 ]