VisualCent : Visual Human Analysis using Dynamic Centroid Representation

Niaz Ahmad Youngmoon Lee Guanghui Wang Department of Computer Science, Toronto Metropolitan University, Toronto, Canada. Department of Robotics, Hanyang University, Ansan, South Korea. · 2025

Read VisualCent : Visual Human Analysis using Dynamic Centroid Representation by Niaz Ahmad Youngmoon Lee Guanghui Wang Department of Computer Science, Toronto Metropolitan University, Toronto, Canada. Department of Robotics, Hanyang University, Ansan, South Korea. (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-19032.txt.

Introduction

Niaz Ahmad, Youngmoon Lee, and Guanghui Wang introduce VisualCent, a 2025 framework that finds people's poses and outlines in images through one shared idea: the centroid. Analyzing several people at once is a scaling headache for vision systems. Bottom up methods detect body joints first, then group them into persons, which suits crowded scenes.

VisualCent builds its keypoint maps around disk shaped marks and a key centroid that pins down each joint's best location. For segmentation it adds a mask centroid, a moving center that gathers pixels into the right person's outline. That dynamic center copes with fast motion and heavy occlusion, where people overlap. On two standard benchmarks the method beat existing systems on accuracy scores and ran faster per frame, the authors report. A single centroid idea serving both pose and masking keeps the system compact enough for real time use.

Catalog record

CREATORNiaz Ahmad Youngmoon Lee Guanghui Wang Department of Computer Science, Toronto Metropolitan University, Toronto, Canada. Department of Robotics, Hanyang University, Ansan, South Korea.
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-19032

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VisualCent : Visual Human Analysis using Dynamic Centroid Representation Niaz Ahmad Youngmoon Lee Guanghui Wang Department of Computer Science, Toronto Metropolitan University, Toronto, Canada. Department of Robotics, Hanyang University, Ansan, South Korea. Abstract We introduce VisualCent , a unified human pose and instance segmentation framework to address generalizability and scalability limitations to multi-person visual human analysis. VisualCent leverages centroid-based bottom-up keypoint detection paradigm and uses Keypoint Heatmap incorporating Disk Representation and KeyCentroid to identify the optimal keypoint coordinates. For the unified segmentation task, an explicit keypoint is defined as a dynamic centroid called MaskCentroid to swiftly cluster pixels to specific human instance during rapid changes in human body movement or significantly occluded environment. Experimental results on COCO and OCHuman datasets demonstrate VisualCent ’s accuracy and real-time performance advantages, outperforming existing methods in mAP scores and execution frame rate per second. The implementation is available on the project page 1 1 1 https://sites.google.com/view/niazahmad/projects/visualcent . I Introduction Human pose estimation and body segmentation are crucial for human-computer interaction and real-time image/video analytics. The primary goals are identifying individuals and their activities based on 2D joint positions and body shapes. The main challenges include handling an unknown number of overlapping, occluded, or entangled individuals and managing the rapid increase in computational complexity as the number of individuals grows. Human-to-human interactions further complicate spatial associations due to limb contacts and obstructions, necessitating an efficient, scalable, and accurate unified model for human pose and segmentation. We propose VisualCent , a new centroid-based unified representation for human pose estimation and instance-level segmentation. It first detects individual keypoints in a centroid-based bottom-up manner and then it employs the high confidence keypoints as dynamic centroids for mask pixels to perform instance-level segmentation. Unlike top-down approaches [ 6 , 15 , 23 , 12 ] , VisualCent detects human structures without requiring a box detector or incurring runtime complexity. Fig. 1: Visual performance of VisualCent . (a) represents KeyCentroid to identify precise keypoint coordinates. (b) represents MaskCentroid with dynamic high confident keypoint to cluster the mask pixels to the correct instance. (c) represents an illustration of a binary mask generated with the help of MaskCentroid (d) exemplifies the instance-level segmentation. (e) shows a unified representation of human pose and estimation (zoom in). Although VisualCent adopts a bottom-up approach similar to other models [ 11 , 9 , 14 , 46 ] , it introduces significant gains. While existing models [ 11 ] rely on human poses to refine pixel-wise clustering for segmentation, they struggle with segmentation accuracy. Other models [ 9 , 14 , 46 ] face challenges such as i) computational overhead due to person detectors [ 14 ] , ii) scalability limitations in instance-level segmentation [ 46 ] , or iii) excessive model complexity [ 9 ] , making them unsuitable for crowded scenarios and real-time applications. In contrast, VisualCent overcomes these issues by avoiding person detectors and mitigating segmentation performance and scalability problems associated with clustering methods that lack pixel centroids. VisualCent addresses these challenges by introducing two key contributions: KeyCentroid and MaskCenroid. KeyCentroid is to determine the optimal keypoint coordinates with the keypoint disk by generating the vector field within the disk using a regression technique. In this way, VisualCent restricts the classifier to a particular region of the keypoint that helps with intricate keypoints of the human body as shown in Fig. 1 (a). MaskCenroid employs the high-confidence keypoints as dynamic clustering anchors. This method allows the grouping of pixels based on proximity to these centroids as shown in Fig. 1 (b), significantly reducing the number of computations and enhancing scalability. MaskCentroids dynamically adjust to rapid movements and occlusions, maintaining the model’s real-time capabilities without the processing overhead of exhaustive pixel-wise relationships. This design choice enables VisualCent to operate efficiently in dense, multi-person environments without compromising accuracy or runtime performance. Fig. 1 (c) illustrates the binary mask generated based on MaskCenroid concept. Fig. 1 (d) shows the instance mask of the human body. Fig. 1 (e) displays the final visual output of the VisualCent human pose and instance-level segmentation in the video stream. II Related Work Human Pose Estimation: Approaches for human pose estimation can be classified into two categories: top-down or bottom-up. The top-down approach first runs a human detector and then identifies keypoints. Representative works include HRNet [ 7 ] , RMPE [ 10 ] , Multiposenet [ 21 ] , Hourglass [ 29 ] , convolutional pose machine [ 39 ] , CPN [ 6 ] , Mask r-cnn [ 14 ] , simple baseline [ 41 ] , CSM-SCARB [ 35 ] , RSN [ 3 ] , and Graph-PCNN [ 37 ] . The top-down approach explores human pose within a person detector, achieving satisfactory performance but at the cost of high computational expense. The bottom-up approach like DeepCut [ 31 ] and DeeperCut [ 16 ] , unlike the top-down counterpart, detects the keypoint in a one-shot manner. It formulates the association between keypoints as an integer linear scheme which takes a longer processing time. Part-affinity field techniques like OpenPose [ 4 ] and other extensions, such as PersonLab [ 11 ] , and HGG [ 18 ] have been developed based on grouping techniques that often fail in the crowd. VisualCent aims to specifically improve hard keypoint detection in crowded and occluded cases by introducing the keypoint heatmaps using keypoint disks and KeyCentroid. Instance-level Segmentation: Instance-level segmentation is done in either single-stage [ 8 , 27 , 2 , 40 ] or multi-stage [ 14 , 33 ] . The single-stage approach generates intermediate and distributed feature maps based on the input image. InstanceFCN [ 8 ] produces instance-sensitive scoring maps and applies the assembly module to the output instance. This approach is based on repooling and other non-trivial computations (e.g., mask voting), which is not suitable for real-time processing. YOLACT [ 2 ] generates a set of mask prototypes and uses coefficient masks, but this method is critical for obtaining a high-resolution output. The multi-stage approach follows the detect-then-segment paradigm. It first performs box detection, and then pixels are classified to obtain the final mask in the box region. Mask R-CNN [ 14 ] extends Faster R-CNN [ 33 ] by adding a branch for predicting segmentation masks for each Region of Interest. The subsequent work in [ 26 ] improves the accuracy of Mask R-CNN by enriching the Feature Pyramid Network [ 25 ] . In contrast, our segmentation pipeline introduces MaskCentroid, a dynamic clustering point that helps cluster the mask pixels to a particular instance under the rapid changes in human-body movements. Fig. 2: The overview of the proposed system. F^{*} indicates the KeyCentroid K_{c} operation. F^{\prime} indicates the MaskCentroid M_{c} operation using the Explicit keypoint E_{k} predicted by K_{c} . Visual Human Analysis: The paradigm has attracted significant attention in recent years, especially in the domains of human visual analysis and image generation [ 43 ] . Mask R-CNN [ 14 ] was a pioneering method in human visual analysis, but it suffers from high computational costs due to its top-down nature. PersonLab [ 11 ] and Pose Plus Seg [ 9 ] are closest to VisualCent . Both of them can be

Read the complete text