Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models
Glenn Jocher, Jing Qiu, Mengyu Liu, Shuai Lyu, Muhammet Esat Kalfaoglu · 2026
Read Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models by Glenn Jocher, Jing Qiu, Mengyu Liu, Shuai Lyu, Muhammet Esat Kalfaoglu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03748.txt.
Introduction
Jocher and co-authors release YOLO26, a unified family of real-time vision models. Real-time vision needs accuracy, speed, and easy deployment across hardware. Older YOLO detectors lean on non-maximum suppression at inference. Heavy heads from Distribution Focal Loss, long training, and missed small objects are further limits. YOLO26 attacks each of those limits directly.
A dual-head design gives native end-to-end inference with no suppression step. Removing the focal loss yields a lighter head with free regression range. Training mixes a hybrid Muon-SGD optimizer, a progressive loss, and a label plan that guarantees small objects get positive assignments. Task heads cover segmentation, pose, and oriented detection in one pipeline. Five scales span tiny to extra-large models. COCO accuracy runs from about 41 to 58 mAP at low latency. An open-vocabulary variant, YOLOE-26, adds text and visual prompting.
Catalog record
| CREATOR | Glenn Jocher, Jing Qiu, Mengyu Liu, Shuai Lyu, Muhammet Esat Kalfaoglu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 93 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.03748
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models Glenn Jocher, Jing Qiu, Mengyu Liu, Shuai Lyu, Muhammet Esat Kalfaoglu Abstract Real-time vision demands models that are accurate, efficient, and simple to deploy across diverse hardware. The YOLO family has become widely deployed for this reason, yet most YOLO detectors still rely on non-maximum suppression at inference, carry heavy detection heads due to Distribution Focal Loss, require long training schedules, and can leave the smallest objects without positive label assignments. We present Ultralytics YOLO26, a unified real-time vision model family that addresses these limitations through coordinated architecture and training advances. Architecturally, YOLO26 uses a dual-head design for native NMS-free end-to-end inference and removes DFL entirely, yielding a lighter head with unconstrained regression range. For training, three techniques jointly improve accuracy while reducing cost: MuSGD, a hybrid Muon–SGD optimizer adapted from large language model training; Progressive Loss, which shifts supervision toward the inference-time head; and STAL, a label assignment strategy that guarantees positive coverage for small objects. Beyond detection, YOLO26 introduces task-specific head and loss designs for instance segmentation, pose estimation, and oriented detection, producing consistent gains across tasks and scales. The family spans five scales (n/s/m/l/x) and supports detection, instance segmentation, pose estimation, classification, and oriented detection in a single pipeline, with an open-vocabulary extension, YOLOE-26, for text-, visual-, and prompt-free inference. Across all scales, YOLO26 achieves 40.9–57.5 mAP on COCO at 1.7–11.8,ms T4 TensorRT latency, advancing the accuracy–latency Pareto front over prior real-time detectors, while YOLOE-26x reaches 40.6 AP on LVIS minival under text prompting. Code and models are available at https://github.com/ultralytics/ultralytics . 1 Introduction Real-time object detection is a cornerstone of practical computer vision, powering applications from autonomous driving and robotics to surveillance and augmented reality, often on edge devices with tight latency and power budgets. The field has advanced through several architectural shifts, but the central challenge remains unchanged: improving accuracy without sacrificing deployment simplicity and runtime efficiency. Two-stage pipelines such as Faster R-CNN 60 set strong accuracy baselines but at considerable inference cost. One-stage detectors (SSD 44 , RetinaNet 39 , YOLOv3 59 , Ultralytics YOLOv5 28 ) traded proposal generation for dense prediction, drastically reducing latency. Anchor-free designs such as FCOS 66 and Ultralytics YOLOv8 69 further simplified the detection head, while YOLOv10 78 introduced consistent dual assignments to enable NMS-free inference. In parallel, DETR 5 cast detection as end-to-end set prediction, and its real-time descendants (RT-DETR 98 , D-FINE 55 , DEIM 21 , RF-DETR 62 ) have narrowed the accuracy gap with CNN-based detectors on standard benchmarks. However, these transformer-based models often depend on large pretrained vision backbones, deformable attention or other custom operators, or fixed input resolutions, which can complicate deployment across heterogeneous hardware targets and reduce portability across edge-oriented inference backends. Throughout these shifts, the YOLO family has remained the most widely deployed real-time detector in industry. Two structural properties underpin this durability. First, deployment universality : YOLO models rely on standard convolutional operators, enabling native export to TensorRT 53 , ONNX 54 , CoreML 1 , TFLite 17 , OpenVINO 24 , NCNN 51 , and ExecuTorch 57 across cloud, mobile, and embedded platforms; Sec. 3.5 summarizes the broader export surface 72 . Second, multi-task unification : a common backbone and neck support object detection, instance segmentation, image classification, pose estimation, and oriented bounding box (OBB) detection within a single training and deployment stack 71 . These properties make the YOLO paradigm a strong foundation for continued progress, provided its remaining limitations are addressed directly. Despite this strong position, several concrete limitations persist across current YOLO-family detectors. (a) NMS dependency and suboptimal dual-head training. Most CNN-based detectors still rely on non-maximum suppression at inference. YOLOv10 78 introduced a dual-head design to remove NMS, but applies identical loss weights to both heads throughout training: the one-to-one branch, the only head used at inference, receives the same optimization pressure as the dense one-to-many branch, leaving it under-trained relative to what targeted supervision could achieve. (b) DFL parameter bloat and range limitation. The Distribution Focal Loss (DFL) module adopted since YOLOv8 69 expands bounding-box regression from 4 scalars to 4K logits per spatial location (typically K{=}16 ), inflating head parameters, an especially unfavorable trade-off for nano-scale models where the head can dominate total parameter count. For instance, YOLO11n with DFL has 2.6M parameters and 6.5 GFLOPs, whereas removing DFL reduces these figures to only 2.3M parameters and 5.2 GFLOPs, a 12% parameter and 20% FLOP reduction from this single module alone. DFL also imposes a finite regression range of (K{-}1)\times\text{stride} pixels per side, which becomes restrictive for large objects at high resolution. D-FINE 55 addresses this by increasing K , but the additional bins further compound head cost. (c) Long training schedules. Standard SGD recipes still require roughly 600 epochs to reach competitive COCO accuracy, making rapid iteration expensive. Recent work on the Muon optimizer 40 has demonstrated roughly 2\times computational efficiency over AdamW in large language model training, yet no prior work has adapted Muon to object detection. (d) TAL zero-assignment for small objects. The Task-Aligned Learning (TAL) 14 label-assignment strategy selects candidate anchors based on geometric containment within ground-truth boxes. After downsampling, objects whose spatial extent falls below the minimum stride have no anchor inside their box, receiving zero positive assignments and zero gradient signal. In this regime, the smallest objects can receive no positive candidates and contribute no localization or classification signal during training. We present Ultralytics YOLO26 , a unified real-time vision model family built on YOLO11 70 . At the detector level, YOLO26 adopts a dual-head design for native NMS-free inference and removes DFL entirely, yielding a lighter regression head with unconstrained range. To recover localization quality and improve optimization, YOLO26 combines three complementary training components: MuSGD , a hybrid Muon–SGD optimizer 40 ; Progressive Loss , which gradually shifts supervision toward the inference-time one-to-one head; and STAL (Small-Target-Aware Label Assignment), which guarantees positive candidate coverage for tiny objects under TAL. The resulting family spans five size variants (n/s/m/l/x) and supports detection, instance segmentation, classification, pose estimation, and OBB detection. Beyond the shared detector, YOLO26 introduces task-specific extensions for instance segmentation, pose estimation, and oriented detection through a multi-scale proto pathway with auxiliary semantic supervision, an RLE-based uncertainty-aware keypoint objective 33 , and a revised long-edge angle formulation with dedicated angle supervision. Across scales, the YOLO26 family combines these extensions with the shared detector improvements, improving over YOLO11 by up to +3.7 mask AP on COCO instance segmentation, +7.2 pose AP on COCO keypoints, and +3.4 mAP on DOTA-v1.0 OBB detection. Beyond closed-set detection, we extend the YOLO26 family to open-vocabulary scenarios by building YOLOE-26 , which instantiates the open-vocabulary formulation of YOLOE 79 on the YOLO26 detector. YOLOE-26 retains the three inference modes of YOLOE (text-prompted, visual-prompted, and prompt-free) while introducing a stronger detector backbone, an upgraded text encoder (MobileCLIP2 13 ), a pseudo-label data engine, and decoupled segmentation training. On LVIS minival, YOLOE-26x reaches 40.6 AP under text prompting, surpassing DetCLIP-T 92 by +6.2 AP (see Sec. S7 ). Figure summarizes the accuracy–latency trade-off on COCO val2017 . Across all model scales, YOLO26 sits on or advances the Pareto front, with the strongest AP–latency trade-off at the medium,