Instance Awareness of Multi-class Semantic Segmentation Loss Functions
Soumya Snigdha Kundu, Florian Kofler, Marina Ivory, Hendrik Möller, Jonathan Shapey, Tom Vercauteren, [0.5em] King’s College London · 2026
Read Instance Awareness of Multi-class Semantic Segmentation Loss Functions by Soumya Snigdha Kundu, Florian Kofler, Marina Ivory, Hendrik Möller, Jonathan Shapey, Tom Vercauteren, [0.5em] King’s College London (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-24276.txt.
Introduction
Kundu and five colleagues extend instance-sensitive segmentation losses to multi-class problems, where rare classes get too little signal. Blob and CC losses were built for single-class segmentation so small lesions match large ones. A one-vs-rest decomposition repurposes them for class imbalance, since uniform averaging lets each class contribute equally. Inverse-size weighting, unstable globally, works when confined inside each component's loss. On BraTS-METS 2025, multi-class CC loss lifts foreground Dice to 0.64 with rare-class gains, while blob loss leads on Panoptic Quality and recognition.
Catalog record
| CREATOR | Soumya Snigdha Kundu, Florian Kofler, Marina Ivory, Hendrik Möller, Jonathan Shapey, Tom Vercauteren, [0.5em] King’s College London |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30201 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.24276
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Instance Awareness of Multi-class Semantic Segmentation Loss Functions Soumya Snigdha Kundu, Florian Kofler, Marina Ivory, Hendrik Möller, Jonathan Shapey, Tom Vercauteren, [0.5em] King’s College London Abstract Instance-sensitive losses for semantic segmentation such as blob loss and CC loss were designed to address instance imbalance , ensuring small lesions generate the same gradient as large ones, but operate only on single-class segmentation. In multi-class settings, class imbalance poses an additional problem: rare classes with few instances receive a disproportionately small share of the training signal. We show that extending instance-sensitive losses to multi-class segmentation via a one-vs-rest class decomposition repurposes them to also address class imbalance, as uniform averaging over classes ensures each class contributes equally regardless of frequency. We further show that inverse-size weighting, which destabilizes training when applied globally due to weight imbalances across rare and common classes, becomes effective when integrated within the per-component loss, confining the reweighting to each component’s spatial context. On the BraTS-METS 2025 dataset (260 test cases), multi-class CC loss improves foreground Dice ( 0.64 ± 0.26 0.64\pm 0.26 vs. 0.59 ± 0.27 0.59\pm 0.27 baseline) and rare-class Dice, while maintaining Panoptic Quality at DSC threshold 0.5. Multi-class blob loss achieves the best Panoptic Quality at threshold 0.5 ( 0.40 ± 0.24 0.40\pm 0.24 vs. 0.38 ± 0.25 0.38\pm 0.25 baseline) and recognition quality ( 0.53 ± 0.29 0.53\pm 0.29 vs. 0.49 ± 0.30 0.49\pm 0.30 ). Integrating inverse-size weighting within the per-component loss increases rare-class Dice to 0.44 ± 0.36 0.44\pm 0.36 at the cost of reduced detection quality. Figure 1 : 3D rendering of a representative BraTS-METS case. (a) Semantic reference label with four tumor sub-regions overlaid on the T1c brain volume. (b–e) Per-class instance decomposition via connected components: (b) SNFH (15 components), (c) ET (17 components), (d) RC (1 component), (e) NETC (4 components). Numbers index individual instances for each class. 1 Introduction Brain metastases are the most common intracranial tumors, affecting 10–26% of patients who die from cancer 9 , with an estimated 98,000–170,000 new cases per year in the United States alone 11 . Stereotactic radiosurgery (SRS) is a standard treatment for both intact and resected brain metastases, requiring precise target delineation to deliver tumoricidal doses while sparing surrounding tissue 8 . For resected metastases, adjuvant SRS to the resection cavity reduces 12-month local recurrence from approximately 50% to under 30% 6 , making accurate cavity segmentation directly relevant to treatment planning. Individual metastases range from sub-millimeter micro-metastases to lesions exceeding several centimeters, and SRS dose prescription depends directly on target volume 8 , making accurate size estimation essential. Beyond SRS targeting, longitudinal monitoring of individual tumor sub-regions enables quantitative tracking across timepoints 7 . The BraTS-METS 2025 challenge formalizes this clinical need by defining four segmentation regions: whole tumor (WT), tumor core (TC), enhancing tumor (ET), and resection cavity (RC) 7 . Automated segmentation of these sub-regions is therefore essential for scalable, consistent treatment planning and follow-up, but standard segmentation networks trained with voxel-level losses such as Dice + cross-entropy (DC+CE) perform well on common structures and struggle with rare ones. Multi-class Dice loss partially addresses class imbalance by computing a separate score per class and averaging uniformly, giving each class equal weight regardless of voxel count, but it still operates at the voxel level and does not distinguish between instances. Figure 2 : Overview of instance-sensitive loss decomposition. (a) Gradient contribution by class (color) and instance size (hatch). DC+CE is dominated by large instances in common classes; inverse-size weighting shifts to small instances but RC remains underrepresented; Blob/CC with class decomposition equalizes both classes and instances; adding IWL within components further emphasizes small instances while preserving class balance. (b) CC loss partitions the volume into Voronoi cells V_{k}^{c} . (c) Blob loss pairs each component S_{k}^{c} with the background. On the BraTS-METS 2025 test set (a randomized split, as the original test set is not publicly available), resection cavity (RC) exhibits triple imbalance: present in only 10% of cases, contributes 32 components versus 1,739 for ET, and represents 295K voxels versus 15.5M for whole tumor. While Dice loss normalizes per class, the cross-entropy component averages gradients across all foreground voxels, so classes with fewer voxels receive proportionally smaller updates, compounding the effect of case- and instance-level rarity. Under DC+CE training, the baseline nnU-Net 2 achieves an RC Dice Similarity Coefficient (DSC) of only 0.15 and a foreground-mean Panoptic Quality 4 at DSC threshold 0.5 ( PQ_{0.5}^{\text{DSC}} ) of 0.38. Instance-sensitive losses address a related but different problem: instance imbalance , where small lesions are under-represented relative to large ones within the same class. One of the earliest efforts was inverse-size weighting 10 , where each voxel is assigned a weight inversely proportional to its connected-component size, amplifying gradients from small instances. More recently, Blob loss 5 and CC loss 1 have taken this further by decomposing the label map into connected components and evaluating an independent loss per component, ensuring each instance contributes equally to the gradient. Connected components derived from the binary label map serve as a proxy for instances in the absence of explicit instance masks, allowing these losses to be instance-sensitive without requiring ground-truth instance annotations. However, all three methods were developed for single-class segmentation and equalize gradients only within each class, not across classes with vastly different frequencies. To our knowledge, this work is among the first to address instance and class imbalance simultaneously within a single loss for multi-class segmentation. We observe that extending instance-sensitive losses to multi-class segmentation, via a one-vs-rest decomposition that treats each foreground class independently, repurposes these losses to solve class imbalance in addition to instance imbalance. By averaging per-component losses uniformly across classes, instances from each class receive equal gradient weight regardless of how rare the class is (see Fig. 2 ). A 78 mm 3 micro-metastasis in a common class and a 2,617 mm 3 resection cavity in a rare class contribute equally to the training signal. We further integrate inverse-size weighting within the per-component loss, rather than applying it as a standalone global mechanism, allowing it to interact with the component-level decomposition. Our evaluation reveals a systematic trade-off between voxel-level overlap (DSC) and instance-level detection ( PQ_{0.5}^{\text{DSC}} ): methods that up-weight rare classes improve DSC at the cost of lower PQ_{0.5}^{\text{DSC}} . Our contributions are as follows: • We show that instance-aware losses (blob loss, CC loss), when applied to multi-class segmentation via one-vs-rest class decomposition, are repurposed from solving instance imbalance to also solving class imbalance, because uniform averaging over classes ensures each class contributes equally regardless of frequency. • We show that inverse-size weighting, which destabilizes training when applied globally to multi-class segmentation due to weight imbalances across rare and common classes (small instances in rare classes receive much higher weights than large instances in common classes), becomes effective when integrated within the per-component loss, increasing blob loss RC Dice from 0.22 to 0.44, as the reweighting is confined to each component’s spatial context. Table 1 : Dataset statistics for BraTS-METS 2025 (260 test cases), illustrating the triple imbalance affecting RC: it is rare at the case, instance, and voxel levels. Region Cases present Components Median size (mm 3 ) Total voxels WT