Marginalized Generalized IoU (MGIoU): A Unified Objective Function for Optimizing Any Convex Parametric Shapes

Duy-Tho Le, Trung Pham, Jianfei Cai, Hamid Rezatofighi · 2025

Read Marginalized Generalized IoU (MGIoU): A Unified Objective Function for Optimizing Any Convex Parametric Shapes by Duy-Tho Le, Trung Pham, Jianfei Cai, Hamid Rezatofighi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16443.txt.

Introduction

Duy-Tho Le, Trung Pham, Jianfei Cai, and Hamid Rezatofighi introduced MGIoU in twenty twenty-five, a unifying loss for fitting parametric shapes in computer vision. Overlap, measured by intersection over union, is the canonical score, but optimizing it is messy: regression losses ignore it, direct versions are unstable or shape-limited, and each field handcrafts its own fix.

MGIoU projects convex shapes onto their surface normals and computes a one-dimensional normalized overlap there — simple, fast, and fully differentiable. A plus variant handles unstructured convex shapes, and a minus variant penalizes overlap for tasks like collision-free path prediction. On standard benchmarks the losses beat existing ones while computing ten to forty times faster, and they keep clean metric properties like scale invariance.

Catalog record

CREATORDuy-Tho Le, Trung Pham, Jianfei Cai, Hamid Rezatofighi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-16443

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Marginalized Generalized IoU (MGIoU): A Unified Objective Function for Optimizing Any Convex Parametric Shapes Duy-Tho Le, Trung Pham, Jianfei Cai, Hamid Rezatofighi Abstract Optimizing the similarity between parametric shapes is crucial for numerous computer vision tasks, where Intersection over Union (IoU) stands as the canonical measure. However, existing optimization methods exhibit significant shortcomings: regression-based losses like L1/L2 lack correlation with IoU, IoU-based losses are unstable and limited to simple shapes, and task-specific methods are computationally intensive and not generalizable across domains. As a result, the current landscape of parametric shape objective functions has become scattered, with each domain proposing distinct IoU approximations. To address this, we unify the parametric shape optimization objective functions by introducing Marginalized Generalized IoU (MGIoU) , a novel loss function that overcomes these challenges by projecting structured convex shapes onto their unique shape Normals to compute one-dimensional normalized GIoU. MGIoU offers a simple, efficient, fully differentiable approximation strongly correlated with IoU. We then extend MGIoU to MGIoU + that supports optimizing unstructured convex shapes. Together, MGIoU and MGIoU + unify parametric shape optimization across diverse applications. Experiments on standard benchmarks demonstrate that MGIoU and MGIoU + consistently outperform existing losses while reducing loss computation latency by 10-40x. Additionally, MGIoU and MGIoU + satisfy metric properties and scale-invariance, ensuring robustness as an objective function. We further propose MGIoU - for minimizing overlaps in tasks like collision-free trajectory prediction. Code is available at https://ldtho.github.io/MGIoU/ Figure 1 : MGIoU and its variants computation. Predicted ( P , red) and ground-truth ( G , blue) shapes are projected onto their unique Normals ( P_{i},G_{i} ) to calculate one-dimensional overlaps, measuring intersection (purple segments) relative to union (green segments). To reduce visual clutter, some panels depict projections onto unique Normals of only one shape. Examples: MGIoU {}^{+}[ A) unstructured quadrilaterals, B) general polygons ] , MGIoU [ C) ellipses, D) 3D cuboids ] , and MGIoU {}^{-}[ E) rotated rectangles ] . Best viewed zoomed in. 1 Introduction Parametric shape optimization is a core problem in computer vision, robotics, and graphics, with applications spanning 2D/3D object detection [ 25 , 38 , 37 , 4 ] , 6D pose estimation [ 5 , 15 ] , shape registration, and trajectory prediction [ 33 , 29 , 30 ] . At its heart, this optimization seeks to maximize the similarity between predicted and ground-truth shapes, commonly quantified using the Intersection over Union (IoU). IoU measures the overlap between two parametric shapes relative to their combined area or volume, providing a standardized and scale-invariant metric widely adopted for evaluating shape similarity. Despite its popularity, directly optimizing IoU (or its improved variant, Generalized IoU (GIoU) [ 27 ] ) for arbitrary convex shapes presents significant challenges. Computing the IoU/GIoU analytically for many convex shapes, e.g . two 3D ellipsoids, is non-trivial. Even for simpler shapes like rotated rectangles, 2D polygons or 3D cuboids [ 23 ] , analytical calculation is feasible, but using IoU/GIoU directly as an objective function within gradient-based optimization remains challenging and computationally expensive. Moreover, certain shape parameters, such as rotation angles, lead to erratic gradient landscapes, making optimization unstable when using IoU/GIoU losses. Consequently, aside from a few cases — such as axis-aligned bounding boxes [ 27 ] or 3D bounding boxes with 1 rotation DoF [ 8 , 43 ] — IoU/GIoU-based optimization has not been widely adopted for general shape optimization tasks. As a result, the objective function for parametric shape optimization is highly fragmented in different application domains. Different applications either: (i) rely on basic losses shown to work empirically, such as simple parametric regression losses (e.g., L-norms), OKS [ 19 ] , or Chamfer distance for shape vertices [ 21 ] , or (ii) adopt approximations of IoU/GIoU, such as Gaussian-based models for rotated boxes [ 36 , 37 , 39 ] , or vertex-based approaches for quadrilaterals [ 4 , 12 , 18 , 16 ] . However, these approaches often lack direct correlation with the true IoU/GIoU and in some cases, they may violate important properties, such as scale invariance or metric consistency. Moreover, many of these methods require extensive task-specific tuning [ 37 , 38 ] , ultimately leading to suboptimal shape alignment or over-fitted to specific datasets. Thus, there is currently no unified objective function that works effectively across different applications and shape parameterizations while maintaining strong correlation with standard IoU-based metrics. In this work, we introduce Marginalized Generalized IoU (MGIoU) , a novel geometric loss function explicitly designed to enable flexible and stable optimization of arbitrary convex parametric shapes in any dimension. MGIoU simplifies complex shape overlap calculations into efficient marginalization of 1D GIoU operations by projecting the shapes onto their associated normals ( e.g . , edge normals in 2D, face normals in 3D, or semi-axes for ellipses/ellipsoids). These projections allow for a differentiable, robust, and stable overlap evaluation, even when the shapes do not physically overlap in space. MGIoU unifies shape optimization into a single coherent loss term applied directly to shape normals, enabling holistic adjustment of shape vertices and parameters ( e.g . , position, size, and orientation) without requiring balancing or fine-tuning of multiple separate loss terms. Its simplicity allows seamless integration into existing optimization pipelines, making it a drop-in replacement across diverse applications such as 2D oriented object detection, 3D shape estimation, and polygonal shape fitting. Furthermore, the mathematical construction of MGIoU ensures compliance with essential scale-invariance and metric properties, reinforcing its theoretical soundness as a reliable measure. We propose three variations of this objective function, making MGIoU applicable across a wide range of tasks. The primary version, MGIoU, is designed for optimizing structured convex shapes, where both the source and target shapes share the same parametric domain, e.g . both being rectangles, ellipses, cuboids etc . (see Fig. 1 (C, D)). An extended version, denoted as MGIoU + , supports optimization between unstructured convex shapes, where the source and target may belong to different shape families or have differing numbers of vertices, such as convex polygons or polyhedra with varying and arbitrary vertex and edge counts (see Fig. 1 (A, B) as two examples). To accommodate this, we use a vertex-based parametrization for both shapes, under the condition that the target shape always has more vertices than the source. In MGIoU + , we introduce an additional convexity regularizer to ensure predicted shapes remains convex during optimization. We also propose a complementary loss, denoted as MGIoU - , which is specifically designed for minimizing overlap between two arbitrary convex shapes. This objective is particularly useful for applications such as agent trajectory prediction, where the goal is to reduce the risk of collisions between predicted trajectories. Unlike standard shape alignment objectives, MGIoU - directly minimizes the IoU between the predicted and reference shapes. By applying the same shape projection mechanism, it robustly penalizes overlaps, encouraging safer trajectory predictions. We further extend MGIoU - to handle sequences of shapes, enabling spatio-temporal optimization for tracking purposes. We evaluate the effectiveness of MGIoU and its variants, MGIoU + and MGIoU - , across multiple applications on standard benchmarks. Our results consistently show improved performance over traditional losses, while also significantly reducing the computational overhead compared to more complex existing

Read the complete text