MASSeg : 2nd Technical Report for 4th PVUW MOSE Track
Xuqiang Cao, Linnan Zhao, Jiaxuan Zhao, Fang Liu, Puhua Chen, Wenping Ma · 2025
Read MASSeg : 2nd Technical Report for 4th PVUW MOSE Track by Xuqiang Cao, Linnan Zhao, Jiaxuan Zhao, Fang Liu, Puhua Chen, Wenping Ma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-10254.txt.
Introduction
Xuqiang Cao and colleagues report a competition system for following objects through messy video, which placed second in its challenge track. Complex scenes defeat segmenters through tiny objects, occlusions, and clutter. Writing in twenty twenty-five, the team improves an existing framework and builds an enriched training set. The added data emphasizes occlusions, busy backgrounds, and small targets. Training mixes augmentations that keep frames consistent with ones that deliberately break consistency, hardening the model. At inference, a mask scaling strategy adapts output size to the object and its occlusion level.
Scores on the challenge test set are reported in the standard overlap and boundary measures. Both land in the high range the leaderboard rewards. The report is a concise engineering account: data, augmentation, and output scaling, in that order, carried the result.
Catalog record
| CREATOR | Xuqiang Cao, Linnan Zhao, Jiaxuan Zhao, Fang Liu, Puhua Chen, Wenping Ma |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 14 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-10254
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MASSeg : 2nd Technical Report for 4th PVUW MOSE Track Xuqiang Cao, Linnan Zhao, Jiaxuan Zhao, Fang Liu, Puhua Chen, Wenping Ma Abstract Complex video object segmentation continues to face significant challenges in small object recognition, occlusion handling, and dynamic scene modeling. This report presents our solution, which ranked second in the MOSE track of CVPR 2025 PVUW Challenge . Based on an existing segmentation framework, we propose an improved model named MASSeg for complex video object segmentation, and construct an enhanced dataset, MOSE+ , which includes typical scenarios with occlusions, cluttered backgrounds, and small target instances. During training, we incorporate a combination of inter-frame consistent and inconsistent data augmentation strategies to improve robustness and generalization. During inference, we design a mask output scaling strategy to better adapt to varying object sizes and occlusion levels. As a result, MASSeg achieves a J score of 0.8250 , F score of 0.9007 , and a J&F score of 0.8628 on the MOSE test set.The code is available at https://github.com/cxqNet/MASSeg . 1 Introduction Pixel-level scene understanding is a fundamental task in computer vision, aiming to recognize the category, semantics, and instance of each pixel. With the rapid growth of video content, this task has extended from static images to dynamic videos, drawing increasing attention to video object segmentation (VOS). VOS requires segmenting and tracking target objects across frames with only the first-frame mask provided. It has been widely applied in autonomous driving, augmented reality, video editing, and data annotation. As task complexity increases, memory-based VOS methods have become the mainstream ding2020semantic . These models maintain a memory bank to store visual features and adopt attention or feature matching to segment targets in future frames. Representative works include STM Oh2019STM , which introduces Figure 1 : Representative challenges in complex video object segmentation. The examples showcase small object motion (left), appearance confusion with occlusion (middle), and densely cluttered scenes (right), which reflect typical situations encountered in the MOSE dataset. explicit memory for pixel-level matching; STCN Cheng2021STCN , which improves efficiency with unmasked frames and L2 affinity; and XMem Cheng2022XMem , which mimics human memory with sensory, working, and long-term stages. Cutie Tang2022Cutie enhances robustness in crowded scenes via object-level memory and object transformers, achieving state-of-the-art results on YouTube-VOS and DAVIS. To better evaluate model robustness in real-world scenarios, Henghui Ding et al. proposed and organized the MOSE ding2024pvuw2024challengecomplex and MeViS MeViS tracks. These tracks target two distinct but complementary aspects of video understanding: MOSE focuses on complex video object segmentation under challenging visual conditions, while MeViS emphasizes motion expression-guided segmentation based on natural language descriptions. To better evaluate model robustness in real-world scenarios, Henghui Ding et al., in collaboration with CVPR, introduced the Pixel-level Video Understanding in the Wild (PVUW) Challenge. The challenge focuses on pixel-level scene understanding and features two complementary tracks: MOSE MOSE , which targets complex multi-object segmentation under challenging visual conditions, and MeViS MeViS , which emphasizes motion expression-guided segmentation based on natural language descriptions. The MOSE track achieved significant progress in the PVUW 2024 Challenge ding2024pvuw2024challengecomplex . Figure 2 : Overview of our method. The MOSE dataset includes 2,149 videos and 5,200 annotated instances over 36 categories, with over 430K pixel-level masks. MOSE emphasizes real-world challenges such as occlusions, fast motion, dense small objects, similar appearances, and frequent reappearance, providing a rigorous benchmark beyond prior datasets like DAVIS and YouTube-VOS. As illustrated in Figure 1 , the MOSE dataset contains challenging scenarios such as small objects, heavy occlusions, and similar instances, which significantly impact segmentation robustness. Mainstream methods, e.g., Cutie, achieve over 84% J&F on YouTube-VOS but drop significantly on MOSE, revealing generalization limitations. MOSE has thus emerged as a valuable platform for driving progress in robust, real-world VOS. To address MOSE-specific challenges, we propose an improved method based on the SAM2 framework ravi2024sam2 , incorporating a mask output control strategy and retraining on an enhanced dataset MOSE+ , which includes diverse small object instances and complex scenes. We further adopt inter-frame consistent and inconsistent augmentations and tuned inference strategies to improve robustness across scales and occlusions. Our method achieves a J score of 0.8250 , F score of 0.9007 , and a J&F of 0.8628 on the MOSE test set, ranking 2 nd overall. 2 Method Our overall solution is illustrated in Figure 2 . Based on the data characteristics of the MOSE dataset, we construct a customized dataset named MOSE+ and design a series of targeted data augmentation strategies, aiming to simulate appearance variations, pose perturbations, illumination changes, and structural blur commonly found in real-world video scenarios. During inference, a mask confidence control strategy is adopted, followed by a fusion process over video frames to obtain the final segmentation results. The detailed components are described below. 2.1 Baseline Model We adopt a transformer-based segmentation framework featuring object-guided attention, mask-aware memory, and spatiotemporal reasoning. The model effectively captures temporal cues and spatial details through dual memory modules and multi-scale decoding, enabling robust performance under challenging scenarios such as occlusion, motion blur, and small-object clutter. This strong baseline lays a solid foundation for our enhancement strategies. Figure 3 : Visualization of our data augmentation strategies. Each column shows an example before (top) and after (bottom) applying augmentation. From left to right: (a) geometric transformation (e.g., affine distortion), (b) color jittering with inconsistent color shift, and (c) grayscale conversion. These augmentations simulate realistic variations in pose, illumination, and appearance, improving robustness and generalization of the model in complex scenarios. 2.2 Loss Function To achieve high-precision segmentation and temporal consistency, we design a multi-task loss that combines pixel-wise accuracy, region-level overlap, classification discriminability, and robustness to occlusion. The total loss is defined as: \mathcal{L}_{total}=\lambda_{1}\mathcal{L}_{CE}+\lambda_{2}\mathcal{L}_{Dice}+\lambda_{3}\mathcal{L}_{Sim}+\lambda_{4}\mathcal{L}_{MaskIoU} (1) where \mathcal{L}_{CE} denotes cross-entropy loss for foreground-background classification, \mathcal{L}_{Dice} enhances region consistency, \mathcal{L}_{Sim} enforces similarity between memory and query features, and \mathcal{L}_{MaskIoU} constrains predicted mask quality. These losses are computed across multiple frames and candidate masks to jointly supervise spatiotemporal modeling. In our implementation, the weights \lambda_{i} are empirically set for balanced optimization. This formulation improves segmentation performance on small or occluded targets and supports temporal consistency in long sequences. 2.3 Data Augmentation To improve generalization and robustness, we introduce a set of targeted augmentation strategies during training. Unlike static image tasks, video segmentation demands consistency across frames while simulating realistic variations. Our approach integrates both frame-consistent and frame-inconsistent perturbations: • Consistent geometric transformations : Random horizontal flipping, affine transformations (rotation, shear), and multi-scale resizing are applied across all frames in a clip to simulate viewpoint and object deformation. • Mixed color perturbations : Brightness, contrast, and saturation changes are applied globally, while grayscale conversion and inconsistent color jittering are selectively applied to individual frames, enhancing robustness to lighting changes and visual ambiguity. • Normalization : Images are transformed into tensors and normalized using ImageNet mean and standard deviation for stable convergence and pretrained compatibility. These augmentations significantly improve the model’s ability to handle structure variation, appearance change, and dynamic scenes in MOSE-like scenarios. 2.4 Inference Strategy To enhance the robustness and adaptability of the model in complex video object segmentation, we design and apply a series of strategies during inference. Mask Confidence Control Strategy. We observe that the quality of predicted masks can be significantly