SoftPQ: Robust Instance Segmentation Evaluation via Soft Matching and Tunable Thresholds

Ranit Karmakar, Simon F. Nørrelykke · 2025

Read SoftPQ: Robust Instance Segmentation Evaluation via Soft Matching and Tunable Thresholds by Ranit Karmakar, Simon F. Nørrelykke (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-12155.txt.

Introduction

Ranit Karmakar and Simon Nørrelykke set out SoftPQ, a new score for how well a tool cuts an image into its single things. Their paper is from two thousand twenty-five.

Old scores work by hard yes-or-no rules. A guess that falls just short of a fixed overlap mark counts as a full miss, the same as a wild one. That hides the gap between near and poor work, and gives no credit for slow gains. SoftPQ grades the match on a scale. Two tunable marks set a zone where a match counts in part. The score stays easy to read, and gives a fairer view of how a tool grows.

Catalog record

CREATORRanit Karmakar, Simon F. Nørrelykke
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-12155

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SoftPQ: Robust Instance Segmentation Evaluation via Soft Matching and Tunable Thresholds Ranit Karmakar, Simon F. Nørrelykke Abstract Segmentation evaluation metrics traditionally rely on binary decision logic: predictions are either correct or incorrect, based on rigid IoU thresholds. Detection-based metrics such as F1 and mAP determine correctness at the object level using fixed overlap cutoffs, while overlap-based metrics like Intersection over Union (IoU) and Dice operate at the pixel level, often overlooking instance-level structure. Panoptic Quality (PQ) attempts to unify detection and segmentation assessment, but it remains dependent on hard-threshold matching—treating predictions below the threshold as entirely incorrect. This binary framing obscures important distinctions between qualitatively different errors and fails to reward gradual model improvements. We propose SoftPQ a flexible and interpretable instance segmentation metric that redefines evaluation as a graded continuum rather than a binary classification. SoftPQ introduces tunable upper and lower IoU thresholds to define a partial matching region and applies a sublinear penalty function to ambiguous or fragmented predictions. These extensions allow SoftPQ to exhibit smoother score behavior, greater robustness to structural segmentation errors, and more informative feedback for model development and evaluation. Through controlled perturbation experiments, we show that SoftPQ captures meaningful differences in segmentation quality that existing metrics overlook, making it a practical and principled alternative for both benchmarking and iterative model refinement. A Preprint Keywords Instance Segmentation \cdot Segmentation Evaluation Metrics \cdot Soft Matching \cdot Over-segmentation \cdot Under-segmentation \cdot Structured Prediction \cdot Computer Vision Evaluation 1 Introduction Evaluating instance segmentation performance remains a core task in computer vision. Standard metrics such as the F1 score, Intersection over Union (IoU), mean Average Precision (mAP), and Panoptic Quality (PQ) are widely used to assess segmentation quality. While these metrics are effective for benchmarking, they tend to be highly sensitive to minor prediction (segmentation) discrepancies. While useful for rigor, this sensitivity can obscure improvements and yield counterintuitive results during development. As illustrated in Figure 1 , predicted masks with visibly and qualitatively different segmentation errors can yield similar scores when using conventional metrics. This inability to differentiate between error types makes it difficult to trust these metrics for diagnostic or iterative model improvement. Moreover, small changes in the predicted mask can lead to large variations in metric scores, undermining their interpretability and stability. To address these limitations, we introduce Soft Panoptic Quality (SoftPQ) , a principled extension and generalization of the PQ metric [ 1 ] . SoftPQ incorporates two tunable IoU thresholds—an upper and a lower bound—that define a range for valid matches. The upper threshold identifies strong matches between predicted and ground truth segments, similar to the original PQ formulation. The lower threshold captures "soft" matches associated with over- and under-segmentation, allowing for a more nuanced interpretation of partial overlaps. Users can tune these parameters based on task specific sensitivity. Notably, setting both thresholds to 0.5 recovers the original PQ as a special case of SoftPQ . In addition to threshold tuning, SoftPQ modifies how match quality is aggregated. It introduces a sublinear penalty to ambiguous matches by applying a normalization factor of \sqrt{n+1} where n is the number of partially matched segments. This penalizes soft matches while still reflecting gradual improvements in segmentation quality. We evaluate SoftPQ using a series of controlled synthetic experiments designed to illustrate common segmentation failure modes, including over- and under-segmentation. Our results show that SoftPQ provides a more robust and interpretable assessment across diverse error types. Compared to traditional metrics, it better captures the qualitative differences between outputs, offering a more useful signal for model evaluation and development. Figure 1: Limitations of Standard Segmentation Metrics in Handling over- and under-segmentation: Synthetic examples illustrating common segmentation errors—partial segmentation, over-segmentation, and ghost segments—for both single-object and multi-object cases. Despite there being qualitative differences in the predicted masks, traditional metrics (F1, mAP, IoU, PQ) often produce same or similar scores, highlighting their limited ability to distinguish between qualitatively distinct errors. 2 Related Work Segmentation evaluation metrics are commonly categorized into detection-based and segmentation-based types. Detection-based metrics, such as the F1 score [ 11 ] and mAP [ 10 ] , assess how many true labels are accurately detected based on threshold matching. mAP also allows different threshold levels between 0.5 and 0.9 for controlled scoring. In contrast, segmentation-based metrics like IoU [ 11 ] and Dice coefficient [ 11 ] focus on the degree of pixel-level overlap between predicted and ground truth masks. Each class of metric captures a different aspect of performance and has its limitations when used in isolation. To bridge this gap, Panoptic Quality (PQ) [ 1 ] was introduced as a unified metric that combines detection accuracy (via recognition quality, RQ) and segmentation quality (SQ) into a single score. Originally proposed for panoptic segmentation, PQ has since been adopted in instance segmentation as a balanced metric that accounts for both instance-level correctness and mask quality [ 12 ] . However, its reliance on a fixed IoU threshold for matching predictions with ground truth makes it sensitive to small changes in mask shape or placement, which can lead to counterintuitive score drops in edge cases involving over-segmentation, under-segmentation, or ambiguous boundaries. To mitigate such issues, several alternative strategies have been proposed. Boundary IoU [ 2 ] emphasizes alignment of object contours rather than overall region overlap, improving relevance in settings with complex or fine-grained structures. SortedAP [ 3 ] redefines average precision using instance-wise rankings to promote more stable and interpretable evaluations. In specialized domains like medical imaging, metric adaptations often incorporate task-specific priors or modify loss functions to better reflect domain-relevant errors [ 6 , 8 ] . Recent work has also challenged the suitability of PQ in biomedical contexts, where the assumption of hard thresholds may fail to reflect gradual or partial improvements in segmentation quality [ 5 , 4 ] , a critical point when fine-tuning an image analysis pipeline. Broader reviews of segmentation metrics [ 7 , 9 ] highlight the need for more flexible and interpretable metrics that are sensitive to context-specific requirements and qualitative performance differences. In this setting, we propose SoftPQ which extends PQ by introducing tunable IoU thresholds and a sublinear aggregation function to handle partial matches. This formulation maintains the attractive features of PQ while offering a more stable and adjustable evaluation of segmentation performance. SoftPQ is designed to support both fine-grained analysis during model development and consistent benchmarking in diverse real-world applications. 3 Proposed Method: Soft Panoptic Segmentation with SoftPQ We propose SoftPQ an extension of the PQ metric, designed to provide a more flexible and interpretable framework for evaluating segmentation quality. Unlike the standard PQ metric, which uses a fixed IoU threshold to determine matching between predicted and ground truth segments, SoftPQ introduces two tunable parameters: • Upper IoU threshold ( h ): Defines the minimum IoU required for a segment to be considered a full match, i.e., a true positive. • Lower IoU threshold ( l ): Allows partial matches (soft-matching) to be incorporated into the score in case of over- and under-segmentation. This dual-threshold formulation relaxes the binary matching assumption in traditional PQ and provides a more continuous assessment of segmentation quality. Setting h>0.5 uniquely identifies matched segments, while l<0.5 enables the incorporation of partial overlaps due

Read the complete text