Learning multi-granularity skeleton representations via hierarchical graph contrastive learning Authors: Yaxiong Liu, Xianfeng Zhai Published: 2026 ABSTRACT AbstractSkeleton-based action recognition offers inherent robustness to illumination variations and background interference, yet faces challenges such as heavy label reliance, topological destruction during augmentation, and underutilized joint confidence. This paper proposes Hierarchical Graph Contrastive Learning (HGCL), in which the three challenges are addressed in a coupled manner rather than through independently inserted modules. The augmentation strategy combines joint masking, spatial rotation, and Gaussian jittering under a unified topology constraint, with perturbation intensity adapted to the graph distance from core joints and a provable upper bound on bone length deviation. The hierarchical contrastive objective decomposes the InfoNCE loss into a global sequence branch and a part-level branch in which negative pairs are restricted within the same body region, so that local motion signatures and holistic action semantics are jointly optimized rather than entangled at a single granularity. The detection confidence is propagated through both the adjacency matrix used for graph convolution and the pooling operator used for representation aggregation, providing a single mechanism that suppresses unreliable joints throughout the pipeline. Experiments on UCF101 and HMDB51 show that HGCL achieves 91.2% and 66.8% Top-1 accuracy respectively, outperforming InfoGCN by 3.9 and 5.3 percentage points. In 1% few-shot scenarios, HGCL improves upon training-from-scratch by 21.6 percentage points on UCF101. Controlled degradation experiments further demonstrate that the performance advantage of HGCL over baselines widens under increasing coordinate noise and joint occlusion. Feature space analysis reveals a 54.5% reduction in intra-class distance and a 90.3% increase in inter-class distance, validating its effectiveness for self-supervised representation learning.Subject terms: Engineering, Mathematics and computing Introduction Background Human skeleton sequence analysis is an important research direction in computer vision and artificial intelligence, playing a crucial role in numerous practical applications. In intelligent fitness systems, skeleton motion trajectories are analyzed to assess exercise posture correctness and provide real-time corrective feedback. In rehabilitation medicine, physicians utilize skeleton sequence analysis to quantitatively evaluate patients’ motor function recovery, providing objective evidence for personalized rehabilitation planning1. In security surveillance, abnormal behavior detection systems rely on skeleton sequences to identify dangerous behaviors such as falls and fights, enabling timely alerts. In human-computer interaction, motion-sensing devices capture users’ skeletal movements to enable gesture control and immersive interactive experiences. Skeleton data represents human motion as time series of three-dimensional coordinates of body joints, offering multiple advantages over conventional RGB video data. Skeleton data has significantly lower dimensionality than video frame sequences, resulting in reduced computational and storage overhead2. Skeleton data contains only structured body information without appearance features, providing inherent privacy protection. Skeleton data exhibits strong robustness to illumination variations, background interference, and occlusions, making it suitable for complex real-world environments3. In recent years, the proliferation of depth sensors and advances in human pose estimation algorithms have made skeleton data increasingly accessible. From professional motion capture equipment to consumer-grade depth cameras and monocular RGB-based pose estimation methods, various sources can provide usable skeleton data. This technological maturation has strongly promoted research on skeleton-based action recognition, behavior prediction, and motion assessment4. Existing skeleton sequence analysis methods predominantly adopt supervised learning paradigms, where model training relies on large-scale manually annotated data. This paradigm faces significant challenges in practical applications where annotation costs are high and new tasks frequently emerge. Action annotation requires annotators to examine skeleton sequences frame by frame to determine action boundaries and category labels, which is time-consuming and demands certain professional expertise. Annotation becomes even more challenging in specialized domains. Dance action annotation requires personnel with professional dance knowledge, while rehabilitation exercise annotation necessitates experts with medical backgrounds. Annotation costs increase sharply with rising professional requirements5. Different annotators may have varying interpretations of the same action, making annotation consistency difficult to guarantee. Such annotation noise adversely affects model learning. When application scenarios change or new action categories emerge, existing models often cannot be directly applied. New data must be collected and annotated for new scenarios, and models must be retrained from scratch or undergo extensive fine-tuning6. This strong dependence on annotated data not only increases deployment costs but also extends system implementation cycles, severely constraining the practical adoption of skeleton sequence analysis technologies. Learning general skeleton sequence representations from unlabeled data offers an effective approach to address these challenges. Unlabeled skeleton sequence data can be obtained at low cost. Massive human motion videos exist on the internet, from which skeleton sequences can be extracted through pose estimation algorithms7. High-quality feature representations can capture the essential patterns of human motion, encoding semantic information and kinematic characteristics of actions. Such representations demonstrate good transferability to downstream tasks, achieving satisfactory recognition performance with only a small number of labeled samples for fine-tuning8. Skeleton data possesses unique structural properties distinct from images and videos. Human joints are interconnected through bones, forming a fixed topological structure that contains rich spatial relationship information. Adjacent joints exhibit physical constraints and coordination in their movements. Different body parts such as the torso and limbs display hierarchical coordination patterns when performing specific actions. Skeleton sequences also exhibit significant temporal dynamics, where joint position trajectories over time reflect the rhythm and style of actions9. How to fully exploit the spatiotemporal structural properties of skeleton sequences, integrate them into self-supervised learning frameworks, and design learning strategies tailored to skeleton data characteristics to obtain discriminative and generalizable action representations remains a key problem requiring in-depth investigation in skeleton sequence analysis. Research on skeleton sequence representation learning holds significant importance for advancing human action understanding. Effective representation learning methods can substantially reduce the annotation requirements for downstream tasks, enabling skeleton sequence analysis in data-scarce specialized domains. General action representations with cross-scenario transferability can reduce resource consumption for repeatedly developing models for specific applications, accelerating the transition from laboratory research to industrial deployment. With the continuous development of intelligent wearable devices and motion-sensing interaction technologies, application scenarios for skeleton data will further expand, and the demand for efficient and general representation learning methods will become increasingly urgent. Current research on self-supervised representation learning for skeleton sequences is still in its early stages. How to effectively leverage the graph structural properties of skeleton data to design representation learning frameworks remains to be explored. Research breakthroughs in this direction will bring new technical paradigms to the field of human action understanding. Related work Skeleton-based human action recognition has attracted extensive attention due to its robustness to illumination variations and background interference. The introduction of graph neural networks has brought significant breakthroughs to this field. Graph convolutional networks model human skeletons as graph structures, effectively capturing spatial topological relationships and temporal dynamics among joints. This approach has become the mainstream technical paradigm for skeleton-based action recognition10. In spatiotemporal feature modeling, researchers have proposed multi-scale spatiotemporal matching strategies to handle actions with different motion amplitudes and speeds. Cross-scale feature alignment significantly improves recognition performance in few-shot scenarios11. To address the geometric properties of skeleton sequences, methods based on Riemannian manifolds and Lie group theory have been introduced to precisely describe the three-dimensional geometric relationships of rigid body motion. Combined with attention mechanisms, these methods achieve adaptive selection of key limb movements and skeleton frames12. Position encoding graph convolutional networks enhance the distinguishability of skeleton nodes through cosine encoding and capture inter-frame motion evolution patterns using temporal dynamics extraction modules13. Graph variational autoencoder architectures learn disentangled action representations through group sparsity constraints, effectively separating heterogeneous factors in action features14. Graph neural networks have also shown strong capability in relational visual understanding beyond action recognition. Lu et al. 15 designed a message passing framework on graph structures for segmenting objects from relational visual data, demonstrating the effectiveness of graph-based relational modeling in visual tasks with limited supervision. To address challenges posed by low-quality skeleton data, part-level knowledge distillation frameworks leverage high-quality samples to guide feature learning of low-quality samples, transferring local action patterns through shared body parts16. A related design principle has been adopted outside skeleton recognition, where progressive guided fusion mechanisms transfer information from one branch to another in a stage-wise manner so that each granularity is constrained by features computed at coarser levels 17, which is conceptually consistent with the way the global contrastive branch in HGCL provides an overarching constraint on the part-level branch during joint optimization. Multimodal fusion represents another important direction for improving recognition performance. Dual-branch networks achieve feature complementarity by jointly modeling RGB videos and skeleton data18. The design of multi-branch fusion has been further explored in dense prediction tasks, where memory-sharing modules and morphological complementarity have been used to coordinate heterogeneous branches that capture complementary aspects of the input 19, sharing with skeleton-based dual-branch fusion the underlying view that branches at different abstraction levels benefit from explicit cross-branch interaction rather than late-stage concatenation. The development of large-scale multimodal pre-trained models provides new insights for skeleton-based action recognition. Language environment description generation modules utilize the open-world understanding capability of vision-language models to capture action-related environmental semantic information20. Masked skeleton feature reconstruction methods combine vision-language pre-training with skeleton modeling, injecting semantic understanding capabilities through probability distribution distillation21. Semantic information-guided multimodal contrastive learning methods optimize the latent space distribution of skeleton features using textual semantics from action labels22. The aforementioned methods have achieved excellent performance on multiple benchmark datasets. These methods nonetheless rely on large-scale manually annotated data, which constrains their application in scenarios with scarce annotation resources. Self-supervised learning provides a new approach to alleviate the dependence of skeleton-based action recognition on annotated data by designing pretext tasks to learn general representations from unlabeled data. Contrastive learning, as the mainstream paradigm of self-supervised learning, achieves discriminative representation learning by pulling positive pairs closer and pushing negative pairs apart in feature space. This approach has demonstrated significant potential in skeleton sequence analysis. Contrastive masked learning methods integrate masked modeling mechanisms into multi-level contrastive learning frameworks, pursuing inter-instance semantic consistency through clustering assignment consistency constraints. However, their data augmentation strategies do not explicitly utilize the graph topological structure information of skeletons23. Handling noisy structural correspondences is also a relevant challenge in graph-based learning. Li et al. 24 proposed a cross-graph meta matching correction method that generates confidence scores for nodes and edges to adaptively penalize noisy assignments, which shares a similar motivation with the confidence-based weighting mechanism adopted in this paper for suppressing unreliable skeleton joints. Dual-decoder collaborative learning methods simultaneously connect contrastive learning and masked reconstruction tasks in both skeleton space and feature space. They enhance model understanding of human interaction relationships through data permutation across temporal, spatial, and individual dimensions. Nevertheless, their view augmentation strategies lack explicit constraints on skeletal connections, potentially generating augmented samples that violate human kinematics25. Mutual information-driven equivariant contrastive learning methods combine equivariant and invariant learning. Through transformations such as skeleton mixing and temporal shuffling, they expose motion patterns and alleviate feature dimension collapse. However, the adopted data transformation strategies do not consider physical connection constraints among joints26. Cross-part bidirectional distillation methods decompose skeleton sequences into part streams through pose decomposition. They improve the discrimination of fine-grained ambiguous actions through relational knowledge distillation frameworks. Although this method introduces part-level modeling concepts, the part division is relatively coarse and hierarchical semantic structures are not explicitly modeled in the contrastive loss function27. Reconstruction-driven contrastive learning methods employ strong encoder and weak decoder architectures to simultaneously perform contrastive learning and sequence reconstruction tasks. Reconstruction objectives guide contrastive learning to focus on joint coordinate details. However, the reconstruction loss is computed at the sequence level without utilizing spatial constraints provided by the skeleton graph structure28. Dual-domain triple contrastive frameworks simultaneously perform self-supervised, supervised, and domain-adaptive contrastive learning in both source and target domains. They introduce extreme skeleton augmentation strategies to improve cross-dataset generalization. However, overly aggressive augmentation operations may compromise the kinematic plausibility of skeleton sequences29. Action sub-region dependency contrastive learning methods propose anchor-based unsupervised action sub-region discovery strategies, designing motion-adaptive data transformations for dynamic and static regions respectively. This method focuses on discovering salient action regions without explicitly modeling hierarchical semantic associations of body parts30. In existing self-supervised learning methods for skeleton sequences, augmentation strategies are mostly designed from a general perspective without fully utilizing graph structure priors and kinematic constraints of skeletons. Contrastive losses are mostly computed at the global instance level, ignoring the inherent hierarchical semantic structure of the human body. Beyond the contrastive paradigm, alternative representation learning strategies have also been explored in visual recognition and motion understanding. Wang et al. 31 formulated visual recognition through deep nearest centroids, in which class prototypes are learned in the feature space and recognition is reduced to centroid retrieval, providing an interpretable alternative to softmax classification. Han et al. 32 extended the prototype idea to motion understanding through a prototypical transformer, in which a unified set of motion prototypes is shared across heterogeneous downstream tasks such as optical flow estimation, scene depth, object tracking, and video stabilization. Cao et al. 33 adopted a vector-based representation for head pose estimation, in which the head orientation is encoded as a continuous three-vector rather than as discrete Euler angles or quaternions, leading to improved robustness against the discontinuity issues that arise in conventional rotation parameterizations. These works share with HGCL the underlying motivation of building structured representations beyond raw feature embeddings, but they target either class-level prototypes, task-shared motion prototypes, or task-specific vector regression for rigid orientation, and do not address the hierarchical contrastive coupling between global skeleton sequences and local body parts that is the focus of this paper. A complementary line of investigation in dense prediction has examined how representations from different branches should be aligned to remain mutually consistent during fusion, with effective and consistent feature fusion mechanisms designed to enforce agreement between branch outputs across multiple stages of the network 34. The hierarchical contrastive coupling adopted in HGCL shares with this line of work the underlying motivation that branches operating at different granularities should be optimized to remain semantically consistent rather than to specialize independently, although the technical realization differs because HGCL operates in a self-supervised contrastive setting on graph-structured skeleton input rather than in a supervised dense prediction setting on grid-structured input. In a related research direction, Wu and Zhang 35 proposed a spatiotemporal multi-view continual dictionary learning method with graph diffusion, integrating label propagation and cross-view graph diffusion with a learnable dictionary to handle scarcely-annotated multi-view data streams. Although both that work and the present method address learning under limited annotations with graph-based strategies, the two differ in problem setting and technical design. The method of Wu and Zhang focuses on retaining spatial and temporal knowledge across sequential multi-view data through dictionary rehearsal, where the graph structure captures inter-view spatial relationships. The present work targets self-supervised pre-training for skeleton action recognition, where the graph structure encodes the anatomical topology of human joints, and the core contributions lie in graph structure-aware skeleton augmentation and hierarchical body part contrastive losses, which are not addressed in the continual dictionary learning framework. To more clearly present the current status and limitations of existing self-supervised learning methods for skeleton sequences, Table 1 provides a systematic review of representative works in recent years. As shown in the table, existing methods still have room for improvement in data augmentation strategy design, hierarchical feature modeling, and skeleton graph structure utilization. This provides the research motivation for the hierarchical graph contrastive learning method proposed in this paper. Table 1Comparison of representative self-supervised learning methods for skeleton sequences.AuthorApplicationResearch contentPotential limitationsZhu et al.36Adversarial robustnessProposed a model-agnostic adversarial contrastive learning framework that enhances defense capability against adversarial attacks through cross-view adversarial trainingFocuses on adversarial attack defense scenarios without fully exploiting skeleton graph structure properties for representation learningLiu et al.37Semantic guidanceCombined large language models to generate action descriptions for semantic compensation, capturing fine-grained discriminative features through cross-modal feature fusionRelies on external pre-trained language models without utilizing the intrinsic graph topological structure information of skeleton dataGuan et al.38Data augmentationDesigned dual minimax games for viewpoint transformation and edge perturbation, generating hard contrastive samples through adversarial paradigmsEdge perturbation strategy randomly changes joint connection strengths, lacking explicit preservation of human kinematic constraintsWang et al.39Ensemble learningProposed an ensemble contrastive learning framework that aggregates multi-pipeline discriminative information through queue expansion and ensemble neighbor miningMulti-pipeline ensemble increases computational and storage overhead; contrastive loss computed at global level without modeling hierarchical semanticsHu et al.40Strong augmentationDesigned gated graph convolutional networks to distinguish effective and ineffective features in strongly augmented views, bridging augmentation gaps through cross-view consistency learningStrong augmentation operations may generate samples violating skeleton kinematics, affecting physical plausibility of representationsJin et al.41Spatiotemporal representationDesigned dual tasks for temporal and spatial reasoning, learning long-range semantic information through temporally incomplete and spatially local sequencesSpatiotemporal reasoning tasks designed independently without modeling spatial topological relationships of skeleton graphs in a unified frameworkHe et al.42Multi-level learningIntegrated joint-level masked reconstruction, sequence-level contrastive learning, and semantic-level representation distillation for multi-granularity feature learningSemantic-level learning relies on external multimodal models; body part-level modeling granularity is coarse and not incorporated into contrastive loss design Comparison of representative self-supervised learning methods for skeleton sequences. The above content systematically reviews related research on skeleton-based human action recognition. Under the supervised learning paradigm, graph convolutional networks have become the mainstream technique. Researchers have continuously advanced recognition performance from multiple perspectives, including multi-scale spatiotemporal modeling, Riemannian manifold geometric representation, position encoding enhancement, graph variational autoencoders, knowledge distillation, and multimodal fusion. However, these methods all rely on large-scale manually annotated data, limiting their application in annotation-scarce scenarios. Our contributions To address the problems of annotation dependency, neglect of topological constraints, and insufficient utilization of confidence information in skeleton-based action recognition, this paper proposes a Hierarchical Graph Contrastive Learning (HGCL) method. The main contributions of this paper are as follows: A graph structure-aware data augmentation strategy is proposed, in which joint masking, Gaussian jittering, and spatial transformations are coordinated under a unified topology constraint. The novelty lies not in any individual operation but in the adaptive noise control mechanism that adjusts perturbation intensity according to the graph distance from core joints, together with a deterministic upper bound on bone length deviation that has not been established in prior skeleton augmentation studies. A graph structure-aware data augmentation strategy is proposed, in which joint masking, Gaussian jittering, and spatial transformations are coordinated under a unified topology constraint. The novelty lies not in any individual operation but in the adaptive noise control mechanism that adjusts perturbation intensity according to the graph distance from core joints, together with a deterministic upper bound on bone length deviation that has not been established in prior skeleton augmentation studies. A hierarchical contrastive learning framework is designed, in which the global InfoNCE objective and the part-level InfoNCE objective are coupled through a shared encoder and a tunable balancing coefficient. Compared with existing part-based skeleton self-supervised methods that aggregate part features only for ensemble prediction, the proposed framework explicitly contrasts corresponding body regions across augmented views and confines part-level negatives within the same anatomical region. A hierarchical contrastive learning framework is designed, in which the global InfoNCE objective and the part-level InfoNCE objective are coupled through a shared encoder and a tunable balancing coefficient. Compared with existing part-based skeleton self-supervised methods that aggregate part features only for ensemble prediction, the proposed framework explicitly contrasts corresponding body regions across augmented views and confines part-level negatives within the same anatomical region. A confidence-driven weighting mechanism is introduced that simultaneously enters the adjacency matrix used for neighborhood aggregation and the pooling operator used for global and part-level representation, providing a unified treatment of unreliable joints across the entire pipeline rather than at a single stage. A confidence-driven weighting mechanism is introduced that simultaneously enters the adjacency matrix used for neighborhood aggregation and the pooling operator used for global and part-level representation, providing a unified treatment of unreliable joints across the entire pipeline rather than at a single stage. Methods Problem statement Skeleton sequence data consists of spatial coordinates of human body joints across consecutive time frames, which can be represented as a three-dimensional tensor. Let a skeleton sequence be denoted as , with the following mathematical definition: where T denotes the number of time frames, N denotes the number of body joints, and C denotes the feature dimension of each joint. When using three-dimensional spatial coordinates, C=3, corresponding to the (x, y, z) coordinate values of each joint. represents the feature vector of the n-th joint at frame t. The human skeleton possesses a natural topological structure, where joints are interconnected through bones to form a fixed graph structure. The skeletal structure is modeled as an undirected graph , formally defined as: where represents the node set, with each node corresponding to a body joint and the set size . represents the edge set, where each edge corresponds to a bone connecting joint i and joint j in the human body. For computational convenience, the graph structure is represented in matrix form using the adjacency matrix : where denotes the element at the i-th row and j-th column of the adjacency matrix. when a bone connection exists between joint i and joint j, or when i=j; otherwise, . Diagonal elements are set to 1 to indicate self-connections for each node, preserving the node’s own feature information in subsequent graph convolution operations.The objective of representation learning is to construct an encoder that maps raw skeleton sequences to low-dimensional feature vectors: where θ denotes the learnable parameters of the encoder, and represents the D-dimensional feature representation obtained by mapping the skeleton sequence through the encoder. The encoder takes the skeleton sequence and graph structure as inputs, outputting a compact vector that characterizes the semantic information of the action. Let the training dataset contain M unlabeled skeleton sequence samples. For any two samples and , the similarity function between their feature representations is defined as: where adopts cosine similarity as the metric, and are the feature representations of the two samples respectively, and denotes the norm of a vector. In the downstream action recognition task, let the ground-truth class label of a test sample be , where K is the total number of action categories. The classifier predicts the class probability distribution based on the feature representation: where denotes the probability value of the k-th class output by the classifier, ϕ represents the classifier parameters, and is the predicted class label. The quality of representation learning is evaluated through the classification accuracy on downstream tasks.Based on the above formal definitions, the core problem investigated in this paper can be stated as follows: Problem 1 Given an unlabeled skeleton sequence dataset and the corresponding skeleton graph structure , the objective is to learn encoder parameters such that the learned feature representations possess good discriminability and generalization capability for downstream tasks: where is the self-supervised learning objective function, which is optimized solely using the structural properties of the skeleton sequence data itself without relying on manual annotations. Graph structure-aware data augmentation Limitations of traditional methods. Traditional data augmentation methods mostly originate from image processing, such as random cropping and color jittering. These methods exhibit notable deficiencies when applied to skeleton sequences. Skeleton data possesses a natural graph structure, where joints are connected through bones to form topological relationships. Traditional methods fail to leverage this structural prior, potentially generating augmented samples that violate human kinematic constraints, such as destroying bone length proportions or joint connectivity. Meanwhile, different joints have varying importance in action expression. Core joints (e.g., spine, hips) carry the main semantic information of actions, while end joints (e.g., hands, feet) have greater degrees of freedom. Traditional methods apply equal perturbations to all joints, making it difficult to balance augmentation diversity and semantic preservation. Limitations of traditional methods. Traditional data augmentation methods mostly originate from image processing, such as random cropping and color jittering. These methods exhibit notable deficiencies when applied to skeleton sequences. Skeleton data possesses a natural graph structure, where joints are connected through bones to form topological relationships. Traditional methods fail to leverage this structural prior, potentially generating augmented samples that violate human kinematic constraints, such as destroying bone length proportions or joint connectivity. Meanwhile, different joints have varying importance in action expression. Core joints (e.g., spine, hips) carry the main semantic information of actions, while end joints (e.g., hands, feet) have greater degrees of freedom. Traditional methods apply equal perturbations to all joints, making it difficult to balance augmentation diversity and semantic preservation. Graph structure-aware augmentation strategy. To address the above issues, A graph structure-aware data augmentation method is proposed to generate diverse augmented views while preserving action semantics. The method decomposes augmentation strategies into spatial and temporal domain components. Spatial domain augmentation includes joint masking and spatial rotation, where joint masking preserves core joints while randomly occluding peripheral ones, and spatial rotation enhances model robustness to viewpoint changes through 3D Euler angle transformations. Temporal domain augmentation includes temporal cropping and motion noise injection, where noise intensity is adaptively adjusted according to the graph distance from joints to core regions, allowing end joints to receive larger perturbations while core joints remain stable. As shown in Figure 1, the original skeleton sequence passes through four augmentation modules sequentially: joint masking, spatial rotation, temporal cropping, and motion noise. Positive sample pairs are generated through two independently sampled augmentation transformations for subsequent contrastive learning. This design theoretically guarantees the boundedness of bone length deviation among surviving bones after augmentation, so that the augmented skeleton remains topology-aware and tolerant to controlled structural corruption rather than strictly identical to the original graph. Graph structure-aware augmentation strategy. To address the above issues, A graph structure-aware data augmentation method is proposed to generate diverse augmented views while preserving action semantics. The method decomposes augmentation strategies into spatial and temporal domain components. Spatial domain augmentation includes joint masking and spatial rotation, where joint masking preserves core joints while randomly occluding peripheral ones, and spatial rotation enhances model robustness to viewpoint changes through 3D Euler angle transformations. Temporal domain augmentation includes temporal cropping and motion noise injection, where noise intensity is adaptively adjusted according to the graph distance from joints to core regions, allowing end joints to receive larger perturbations while core joints remain stable. As shown in Figure 1, the original skeleton sequence passes through four augmentation modules sequentially: joint masking, spatial rotation, temporal cropping, and motion noise. Positive sample pairs are generated through two independently sampled augmentation transformations for subsequent contrastive learning. This design theoretically guarantees the boundedness of bone length deviation among surviving bones after augmentation, so that the augmented skeleton remains topology-aware and tolerant to controlled structural corruption rather than strictly identical to the original graph. Graph structure-aware data augmentation framework. Targeting the graph structural properties of skeleton sequences, this section proposes a graph structure-aware data augmentation method that generates diverse augmented views while preserving action semantics. Let the original skeleton sequence be . The data augmentation operation is defined as a transformation function , mapping the original sequence to an augmented sequence. To fully exploit the spatiotemporal structural properties of skeleton data, the augmentation strategy is decomposed into spatial domain augmentation and temporal domain augmentation . The complete augmentation transformation is defined as: where denotes the augmented skeleton sequence, and ∘ denotes function composition. The spatial domain augmentation is first applied to the original sequence, followed by the temporal domain augmentation acting on the spatially augmented result. Spatial domain augmentation aims to perturb the spatial structure of skeletons while preserving human kinematic constraints. The joint masking augmentation is defined to enhance model robustness to local missing data by randomly occluding certain joints: where is the mask vector for the n-th joint, independently sampled with probability to determine whether to occlude, ⊙ denotes element-wise multiplication, and is the mean vector of joint coordinates computed over the training set. represents the core joint set, including joints crucial for action semantics such as the spine and hips. These joints remain unchanged during augmentation to maintain the basic structure of actions. The spatial rotation transformation enhances model invariance to viewpoint changes by rotating the geometric sub-vector of each joint while preserving bone length proportions and leaving the per-joint detection confidence unchanged: where denotes the two-dimensional geometric sub-vector of the n-th joint at frame t, denotes the corresponding detection confidence, is the in-plane rotation matrix determined by angle γ, and is the maximum rotation angle. Since the OpenPose-extracted skeleton sequences used in this paper consist of two-dimensional image-plane coordinates accompanied by a per-joint confidence score, only an in-plane rotation is geometrically meaningful, and applying a rotation operator to the confidence channel would conflate a probability value with a spatial dimension. The confidence sub-scalar is therefore preserved as an attribute of the joint and is consumed downstream by the confidence-modulated adjacency in Eq. (18). When the operator is extended to three-dimensional motion-capture inputs, is replaced by the corresponding 3 × 3 rotation matrix acting on the three-dimensional geometric sub-vector, and the confidence channel is treated identically. Temporal domain augmentation aims to perturb the temporal structure of skeleton sequences. The temporal cropping augmentation is defined to randomly extract a continuous subsequence from the original sequence: where is the randomly sampled starting frame index, T' is the sequence length after cropping, is the cropping ratio parameter, denotes the floor function, and denotes the discrete uniform distribution over the integer set . To further enhance temporal diversity, the motion noise injection is defined to add temporally correlated Gaussian perturbations to joint trajectories: where is the noise vector sampled from a multivariate Gaussian distribution, is the noise covariance matrix for the n-th joint, and is the adaptive noise intensity. represents the shortest graph distance from joint n to the core joint set, is the shortest path distance on the graph, is the base noise intensity, and is the distance decay coefficient. This design allows end joints far from core joints (e.g., hands, feet) to receive larger perturbations while core joints remain relatively stable. Combining the above augmentation strategies, the complete graph structure-aware augmentation transformation is defined as a random combination of augmentation components: where each augmentation component is independently applied with corresponding probabilities , , , . Spatial rotation precedes joint masking so that the mean-coordinate placeholder in Eq. (9) is consistent with the rotated frame the encoder eventually observes. Reversing this order would first replace masked joints with the population-level mean and subsequently rotate this placeholder to a viewpoint-dependent location, causing the filled value to drift away from the statistical mean of the observed coordinate distribution and breaking the interpretation of as a neutral occupancy estimate. The application order of the four components is fixed as spatial rotation, joint masking, temporal cropping, and motion noise to ensure determinism and reproducibility of the augmentation process. Based on the above augmentation strategy, for each original sample , positive sample pairs are generated through two independently sampled augmentation transformations and : where denotes the augmentation transformation distribution determined by augmentation parameters . and are two different augmented views of the same original sample, constituting a positive sample pair. The design of the above augmentation strategy follows the graph structure preservation principle, ensuring that augmented skeleton sequences still satisfy human kinematic constraints. Before stating the structural property, we make explicit that the per-joint feature vector used in this paper is decomposed into a geometric sub-vector containing the spatial coordinates and a confidence sub-vector containing the per-joint detection score. The bone length, the rotation operator in Eq. (10), and the Gaussian jittering in Eq. (12) act exclusively on the geometric sub-vector, while the confidence sub-vector is preserved as an attribute of the joint and does not enter the geometric distance metric. The same decomposition is observed throughout the encoder: the confidence sub-vector is consumed only as a per-frame weighting factor in the confidence-modulated adjacency and the pooling operator, and the geometric sub-vector is the sole quantity that participates in the bone length bound established below. The following theorem characterizes the bone length deviation of the augmented sequence under this convention. Theorem 1 (Graph structure preservation) Let the skeleton graph corresponding to the original skeleton sequence be with adjacency matrix . For any augmentation transformation defined in this section, the augmented sequence satisfies the graph structure preservation condition: where is the upper bound of bone length deviation dependent on augmentation parameters, and is the maximum bone length. This theorem indicates that by controlling the noise intensity and rotation angle , bone length changes caused by the rotation and jittering components can be constrained within an acceptable range for bones whose endpoints both survive the masking operation, so that the augmented skeleton remains kinematically plausible while still admitting controlled structural corruption from joint masking. The bound is stated on the geometric sub-vector only, so that the rotation operator and the bone length metric remain mathematically well defined; the per-joint confidence score is propagated through the augmentation pipeline as an unmodified attribute and is consumed later by the confidence-modulated adjacency matrix and the pooling operator. Corollary 1 (Semantic consistency) Assume that the encoder satisfies the Lipschitz continuity condition, i.e., there exists a constant such that for any , . If the augmentation transformation satisfies the graph structure preservation condition in Theorem 1, then the feature representation distance between the original sample and augmented sample is bounded: This corollary guarantees that under graph structure-aware augmentation, feature representations of different augmented views of the same sample maintain semantic consistency, providing theoretical justification for constructing positive sample pairs in subsequent contrastive learning. Spatio-temporal graph convolutional network Limitations of traditional methods. Existing skeleton-based action recognition methods have distinct deficiencies in feature extraction. Traditional sequence models, such as Recurrent Neural Networks (RNNs), can model temporal dependencies. However, they struggle to explicitly utilize the topological connections between skeletal joints. Simply flattening skeleton sequences into vectors loses valuable structural information. Standard Convolutional Neural Networks (CNNs) are designed for regular grid data. They cannot be directly applied to non-Euclidean structures like skeleton graphs. Methods that model spatial or temporal relationships separately fail to capture the coupled spatio-temporal patterns in human actions. Furthermore, simple feature aggregation strategies ignore semantic differences among different body parts in action expression. Limitations of traditional methods. Existing skeleton-based action recognition methods have distinct deficiencies in feature extraction. Traditional sequence models, such as Recurrent Neural Networks (RNNs), can model temporal dependencies. However, they struggle to explicitly utilize the topological connections between skeletal joints. Simply flattening skeleton sequences into vectors loses valuable structural information. Standard Convolutional Neural Networks (CNNs) are designed for regular grid data. They cannot be directly applied to non-Euclidean structures like skeleton graphs. Methods that model spatial or temporal relationships separately fail to capture the coupled spatio-temporal patterns in human actions. Furthermore, simple feature aggregation strategies ignore semantic differences among different body parts in action expression. Spatio-temporal graph convolutional encoder. To address these issues, this paper adopts the Spatio-Temporal Graph Convolutional Network (ST-GCN) as the backbone of the encoder. This structure enables joint encoding of spatio-temporal features in skeleton sequences. In the spatial dimension, the network utilizes a normalized adjacency matrix with learnable attention for multi-order graph convolutions. This adaptively aggregates neighborhood joint features to model topological relationships. In the temporal dimension, dilated causal convolutions are employed to expand the receptive field layer by layer. This effectively captures the dynamic evolution of actions. As shown in Fig. 2, each spatio-temporal graph convolutional block consists of four components: spatial graph convolution, temporal convolution, residual connection, and Dropout. The complete encoder is formed by stacking multiple layers. The internal block structure to which the confidence-modulated adjacency is applied is shown in Fig. 2a, comprising spatial graph convolution, temporal convolution, residual connection, and Dropout in this order; the dual-path output mechanism that supports subsequent hierarchical contrastive learning is illustrated in Fig. 2b, where the global branch performs confidence-weighted average pooling over time and joints and the part branch performs confidence-weighted local pooling over five anatomical regions, after which the two branches are concatenated to form the hierarchical feature consumed by the contrastive objective. The final-layer feature in Fig. 2bis the output of the last block in the stack of Fig. 2a. The global path obtains sequence-level representations through spatio-temporal average pooling. The part path divides joints into several semantic regions for separate pooling. These are finally concatenated into hierarchical feature vectors, simultaneously encoding global semantics and local details of actions. Spatio-temporal graph convolutional encoder. To address these issues, this paper adopts the Spatio-Temporal Graph Convolutional Network (ST-GCN) as the backbone of the encoder. This structure enables joint encoding of spatio-temporal features in skeleton sequences. In the spatial dimension, the network utilizes a normalized adjacency matrix with learnable attention for multi-order graph convolutions. This adaptively aggregates neighborhood joint features to model topological relationships. In the temporal dimension, dilated causal convolutions are employed to expand the receptive field layer by layer. This effectively captures the dynamic evolution of actions. As shown in Fig. 2, each spatio-temporal graph convolutional block consists of four components: spatial graph convolution, temporal convolution, residual connection, and Dropout. The complete encoder is formed by stacking multiple layers. The internal block structure to which the confidence-modulated adjacency is applied is shown in Fig. 2a, comprising spatial graph convolution, temporal convolution, residual connection, and Dropout in this order; the dual-path output mechanism that supports subsequent hierarchical contrastive learning is illustrated in Fig. 2b, where the global branch performs confidence-weighted average pooling over time and joints and the part branch performs confidence-weighted local pooling over five anatomical regions, after which the two branches are concatenated to form the hierarchical feature consumed by the contrastive objective. The final-layer feature in Fig. 2bis the output of the last block in the stack of Fig. 2a. The global path obtains sequence-level representations through spatio-temporal average pooling. The part path divides joints into several semantic regions for separate pooling. These are finally concatenated into hierarchical feature vectors, simultaneously encoding global semantics and local details of actions. Architecture of the spatio-temporal graph convolutional encoder used in this paper. The ST-GCN serves as the backbone of the encoder. It extracts discriminative feature representations from the augmented skeleton sequences. The network models topological relationships via spatial graph convolutions and captures temporal evolution via temporal convolutions. Let the augmented input skeleton sequence be . First, we perform a dimensional transformation for convolutional operations. The input is reorganized into a feature tensor with the channel dimension placed first. The spatial branch follows the standard ST-GCN formulation, in which neighborhood features are aggregated by a symmetrically normalized adjacency supplemented by a learnable attention term that adapts the propagation strength along each edge during training: where is the degree matrix of , is the identity matrix, is the Kronecker delta, and is the learnable attention term. The contribution at this stage is not the spatial operator itself but the modulation of by the per-joint detection confidence. The pose estimator assigns each joint n at frame t a confidence score , and these scores enter the adjacency at the corresponding frame through the confidence-modulated form where is the per-frame confidence coupling matrix and ⊙ denotes element-wise multiplication. The geometric mean attenuates information propagation along an edge whenever either endpoint exhibits low detection confidence at the current frame, providing a direct mechanism for suppressing unreliable joints during neighborhood feature aggregation. The confidence is propagated frame by frame rather than averaged across the sequence, so that transient occlusion patterns within an action–an arm temporarily occluded behind the torso, a foot leaving the field of view at a particular instant–are reflected in the spatial graph at exactly the frame where they occur. Since the modulation amounts to an element-wise reweighting of an already constructed adjacency, the additional cost over the sequence-level variant remains marginal. With the per-frame in place, the spatial graph convolution at the l-th layer aggregates neighborhood node features through a multi-order expansion and applies a linear transformation: where K is the order of the graph convolution and is the activation function. Eq. (19) is written in compact form for readability: at each frame t, the spatial aggregation is performed on the per-frame confidence-modulated adjacency acting on the spatial slice , and the symbol above denotes extending the propagation range to k-hop neighborhoods on the graph as observed at frame t. The case k=0 corresponds to the node’s own features. Readers are referred to the original ST-GCN literature for the derivation of the symmetric normalization and the multi-order expansion; the operator is invoked here as a standard component, and the novelty introduced in this paper is confined to the confidence modulation in Eq. (18). Following the spatial graph convolution, temporal convolution captures the dynamic evolution of actions along the time dimension. We use dilated causal convolutions to increase the receptive field while maintaining temporal causality. The temporal convolution operation is defined as: where Γ is the temporal kernel size. d is the dilation factor, which grows exponentially with the network layers to expand the temporal receptive field. is the temporal convolution weight. represents the spatial feature slice at time . Zero padding is applied when the index to maintain sequence length. The spatial graph convolution and temporal convolution are combined to form a spatio-temporal graph convolutional block. We introduce a residual connection mechanism to mitigate the vanishing gradient problem in deep networks and promote feature reuse. The full computation process for the l-th block is defined as: where is the residual mapping function. If the number of input and output channels is the same, it passes the input features directly. Otherwise, it uses a 1 × 1 convolution for channel alignment. randomly zeros out some features with probability to prevent overfitting. The complete ST-GCN consists of L stacked blocks. The network output is globally pooled to obtain sequence-level feature representations. The full forward computation of the encoder is defined as: where represents the mapping of the l-th block. is the output feature of the last layer, and T' is the sequence length after temporal cropping. The inner summation performs a frame-wise confidence-weighted average across joints, so that joints with low detection confidence at the current frame contribute proportionally less to the spatial summary at that frame; the outer summation then averages these per-frame summaries uniformly along the temporal dimension. flattens the result into a vector. and are parameters of the projection layer. The final output is a D-dimensional feature representation . To support hierarchical contrastive learning, the encoder also outputs body part-level features. The human joints are divided into P semantic parts , satisfying and for any i ≠ j. The feature representation for the p-th part is obtained through local pooling: where is the number of joints in the p-th part. and are projection parameters for part features. is the -dimensional feature vector of the p-th part. The frame-level confidence in the pooling operation is identical to that used in Eq. (18), so that within each anatomical region the contribution of an unreliable joint at any given frame is attenuated at exactly the moment its detection becomes uncertain, and a joint that is reliably detected at most frames but briefly occluded at a few frames retains its overall influence on the part-level representation rather than being uniformly down-weighted by a sequence-level average. The full output of the encoder includes both global and part features, forming a hierarchical representation: where denotes vector concatenation. is a hierarchical feature vector containing both global semantic information and local part information. The set of encoder parameters is denoted as . The expressive capacity of the encoder enters the present framework not as an original theoretical contribution but as a design prerequisite for the contrastive learning stage that follows. We restate it below in the form adopted in this paper, so that the assumptions made later about the existence of a discriminative encoder are explicit. The result itself is a direct specialization of standard universal approximation arguments for graph polynomial filters and dilated convolutions, and the formal proof is deferred to the appendix. Theorem 2 (Encoder expressiveness as a design Prerequisite) Let the skeleton sequence space be a bounded closed set, and be a fixed skeleton graph structure. For any continuous target feature mapping and any \varepsilon > 0, there exist a number of layers and a channel configuration such that the ST-GCN encoder satisfies: Thus, the ST-GCN can approximate any continuous feature mapping on a compact set with arbitrary precision. The above statement is invoked in this paper as a sufficient condition for the existence of an encoder configuration capable of representing the inter-class structure that the contrastive objective aims to recover. We do not claim the universal approximation property itself as a contribution of HGCL; rather, the statement clarifies that the limiting factor of the proposed framework is not the representational capacity of the backbone but the discriminability of the features induced by the pretext task. The corollary that follows specializes this prerequisite to the multi-class skeleton recognition setting, and is also restated as a design observation rather than as an independent theoretical result. Corollary 2 (Separability implied by the encoder Prerequisite) Let the dataset contain K action categories, with the set of samples for the k-th category denoted as . If there exists a feature mapping such that samples from different categories satisfy a lower bound for inter-class distance , then by Theorem 2, there exists an encoder such that: When , the feature representations output by the encoder maintain inter-class separability. This observation is used in the present paper only to justify that, given a sufficiently expressive encoder, any residual gap between HGCL and an oracle classifier is attributable to the contrastive objective and the augmentation design rather than to the backbone capacity, and it does not enter the optimization procedure of the proposed method. Overview of the HGCL framework Synergistic mechanism of hierarchical contrastive losses In the hierarchical contrastive loss design, the global contrastive loss operates on sequence-level representations to enforce that different augmented views of the same action are mapped to nearby points in the feature space while being separated from other actions, thereby capturing holistic action semantics. The part-level contrastive loss operates on body part representations to ensure that corresponding body regions across augmented views maintain consistent local motion features. The two levels form a synergistic optimization mechanism: the global loss provides an overarching semantic constraint that prevents part-level features from overfitting to local noise, while the part-level loss injects structural detail into the global representation by explicitly modeling the motion characteristics of individual body regions. Through joint optimization weighted by the coefficient λ, the encoder simultaneously learns to encode both overall action category information and discriminative local motion signatures, producing multi-granularity representations that are more informative than those obtained from either level alone. HGCL: Hierarchical graph contrastive learning algorithm. Hierarchical contrastive loss formulation The global contrastive loss adopts the standard InfoNCE formulation over sequence-level representations, where the positive pair for sample i consists of and all other 2(B-1) representations in the mini-batch serve as negatives. The part-level contrastive loss operates independently on each body part , where negative pairs are formed exclusively within the same body part–the p-th part of sample i is contrasted only against the p-th part of other samples: The total loss combines both levels with a balancing coefficient λ: Time complexity analysis The computational overhead of the HGCL algorithm primarily stems from three parts: graph-aware data augmentation, spatio-temporal graph convolutional encoding, and hierarchical contrastive loss computation. In the data augmentation phase, joint masking, spatial rotation, temporal cropping, and noise injection are performed for each sample. The time complexity per sample is O(TNC), where T is the sequence length, N is the number of joints, and C is the feature dimension. The ST-GCN encoding phase involves L layers. For each layer, the complexity of spatial graph convolution is , where K is the convolution order. Temporal convolution complexity is , where Γ is the kernel size. Contrastive loss computation requires building a similarity matrix for all sample pairs within a batch, with a complexity of , where B is the batch size and D is the feature dimension. Considering E training epochs, the total time complexity is , where . Since the joint number N and feature dimension C are small (N = 18, C = 3) in the adopted COCO skeleton format, the actual bottlenecks are temporal convolution and contrastive loss computation. Overall, the algorithm maintains good computational efficiency. Space complexity analysis The memory footprint of HGCL primarily includes model parameter storage and intermediate feature caching. Regarding model parameters, the ST-GCN contains L layers. Each layer’s spatial graph convolution parameters are , and temporal convolution parameters are . The attention adjacency matrix parameters are , and projection layer parameters are . The total parameter space complexity is . Regarding feature caching, each block must store input and output tensors. The feature space per sample is , and the total cache for batch training is . The contrastive learning phase requires storing the similarity matrix, with a space complexity of . Since positive pair construction requires features from two augmented views, the actual cache is . In summary, the total space complexity is . Memory usage grows linearly with batch size, allowing efficient execution on standard hardware. Experimental settings and results analysis Datasets and parameter settings This study employs the UCF101 and HMDB51 datasets for experimental validation. UCF101 is a widely used benchmark in the field of action recognition, containing 13,320 video clips collected from YouTube and covering 101 human action categories. These categories are divided into five main types: Human-Object Interaction, Body-Motion Only, Human-Human Interaction, Playing Musical Instruments, and Sports. Typical action categories include daily activities such as playing guitar, surfing, and knitting. The video samples in this dataset exhibit high diversity and complexity. The content covers various filming perspectives, target scales, camera motion patterns, and lighting conditions, which effectively evaluates the model’s generalization ability in complex real-world scenarios. Each video clip corresponds to a single action label, facilitating training and evaluation for supervised learning tasks. HMDB51 contains 6,766 video clips collected primarily from digitized movies and public video databases, covering 51 human action categories. Compared to UCF101, HMDB51 presents greater challenges for skeleton-based recognition due to complex cinematic backgrounds, frequent camera motion, and more subtle inter-class action differences, making it a complementary benchmark for evaluating model generalization. This study follows the official three-split evaluation protocol for HMDB51 and reports the average accuracy across all splits. Regarding data partitioning for UCF101, this study follows the standard protocol of the dataset. The training set contains 75% of the samples for each action category, while the remaining 25% are evenly distributed between the validation and test sets. Since both datasets only provide raw RGB videos, this study utilizes the OpenPose pose estimation algorithm to extract 2D human skeletal sequences from video frames. The extracted skeletal data includes the 2D coordinates of 18 joints and their confidence scores, forming a temporal skeletal graph structure as the input for the proposed HGCL method. This experimental paradigm of extracting skeletons from RGB videos has been validated in multiple prior studies and can effectively evaluate the performance of skeletal action recognition algorithms. Table 2 provides the detailed parameter configuration for the proposed HGCL algorithm. All experiments were conducted on a workstation equipped with an NVIDIA RTX 3090 GPU, using the PyTorch 2.1.0 deep learning framework. Table 2Parameter configuration for HGCL Algorithm.ParameterValueParameterValueST-GCN Layers8Attention Heads4Initial Channels64Intermediate Channels128Output Channels256Embedding Dimension256Graph Kernel Size3Temporal Kernel Size9Spatial Aggregation Hops2Temporal Dilation Base2Body Part Groups5Number of Joints18Input Coordinate Dim3Input Frame Length64Residual TypePre-activationNormalizationBatch NormJoint Mask Ratio0.15Core Joints Retained5Spatial Rotation RangeTemporal Crop Range[0.5, 1.0]Gaussian Noise Strength0.01Noise Decay Coeff0.5Random Translation Range[-0.1, 0.1]Random Scaling Range[0.9, 1.1]Augmentation Prob0.5Resampling Prob0.3Confidence Threshold0.3Low-conf Mask Prob0.8Temperature (τ)0.1Hierarchical Loss Weight0.5Global Loss Weight1.0Part Loss Weight0.5Negative Queue Length16384Momentum Coeff0.99Projection Hidden Dim512Projection Output Dim128Pre-training Epochs100Batch Size (Pre-train)128Initial LR (Pre-train)0.001Min LR (Pre-train)0.00001LR ScheduleCosine AnnealingWarmup Epochs10Weight Decay0.0001OptimizerAdamWMomentum Params(0.9, 0.999)Grad Clip Threshold1.0Dropout Ratio0.3Random Seed42Fine-tuning Epochs50Batch Size (Fine-tune)64Initial LR (Fine-tune)0.0001Min LR (Fine-tune)0.000001LR Schedule (Fine-tune)Cosine AnnealingWarmup Epochs (Fine-tune)5Weight Decay (Fine-tune)0.001Optimizer (Fine-tune)SGDMomentum0.9Grad Clip (Fine-tune)5.0Dropout (Fine-tune)0.5Label Smoothing0.1Frozen Layers4Classifier Hidden Dim512 Parameter configuration for HGCL Algorithm. The ST-GCN in the experiment adopts an 8-layer stacked structure. The number of channels gradually expands from 64 to 256. Compared to deeper configurations, an 8-layer structure effectively avoids overfitting on 2D skeleton inputs with 18 joints. Spatial graph convolutions use a 2-hop neighborhood aggregation strategy to capture high-order topological relationships between joints. Temporal convolutions utilize dilated causal convolutions to model long-range temporal dependencies. The attention mechanism uses a 4-head configuration. This ensures that each head has sufficient expressive power in the 64-dimensional feature space. Body part grouping follows human anatomy, dividing the 18 joints into five semantic parts: torso, left upper limb, right upper limb, left lower limb, and right lower limb. Each joint is represented by a three-component feature vector composed of a two-dimensional geometric sub-vector containing the in-plane coordinates (x, y) and a one-dimensional confidence sub-scalar produced by OpenPose. The confidence sub-scalar reflects the reliability of joint detection, enters the model exclusively through the confidence-modulated adjacency and the confidence-weighted pooling defined in Section Methods, and allows the model to adaptively reduce the contribution of low-confidence joints, mitigating noise interference caused by occlusions and estimation errors. In terms of data augmentation, the graph-aware strategy introduces diverse perturbations while remaining topology-aware and tolerant to controlled structural corruption. The joint masking operation randomly masks 15% of non-core joints. Core joints include 5 structural pivot points such as the neck and hip center. To construct more challenging positive pairs, the spatial rotation range is extended to to handle complex camera perspective changes in UCF101. For joints with confidence lower than 0.3, an additional masking operation is executed with an 80% probability. This prevents incorporating erroneous detections from the pose estimator into the contrastive learning framework. The joint masking ratio of 15% is determined by the redundancy level of the 18-joint COCO skeleton topology: masking fewer than 10% of joints provides insufficient regularization, while masking more than 25% risks removing joints that are critical for action semantics. The spatial rotation range of is chosen to approximate the range of natural camera viewpoint variations in the UCF101 dataset while keeping the bone length deviation bounded within the theoretical limit established in Theorem 1. The base noise intensity is calibrated against the standard deviation of normalized joint coordinates in the training set, ensuring that perturbation magnitudes remain within the scale of realistic pose estimation errors. The distance decay coefficient λ = 0.5 balances augmentation diversity between core and peripheral joints: a value close to zero applies uniform noise regardless of graph distance, while a value close to one suppresses noise almost entirely at peripheral joints. The chosen value of 0.5 allows end effectors such as hands and feet to receive approximately twice the perturbation of core joints, reflecting their greater kinematic freedom in natural human motion. The hierarchical contrastive framework simultaneously optimizes the consistency of global sequence representations and local part representations. The temperature coefficient is set to 0.1 to balance the discriminability of positive and negative samples while ensuring gradient stability in early training. The negative sample queue length is set to 16,384. This maintains a reasonable ratio with the batch size of 128, preventing expired features in the queue from interfering with gradient estimation. The momentum update coefficient is set to 0.99 to ensure feature consistency in the queue while accelerating encoder convergence. The training process is divided into two stages: self-supervised pre-training and supervised fine-tuning. The pre-training stage uses the AdamW optimizer and a cosine annealing learning rate schedule. A 10-epoch warmup period is applied to stabilize gradient updates. The Dropout ratio is set to 0.3 in this stage to prevent excessive regularization from interfering with consistency constraints. During the fine-tuning stage, the bottom 4 layers of the encoder are frozen to preserve the general spatial topology features learned during pre-training and to reduce trainable parameters from 2.4M to 1.1M, lowering overfitting risk under limited labels. The choice of 4 frozen layers reflects a structural break within the 8-layer encoder rather than a numerical coincidence on the validation curve. The bottom layers of the spatio-temporal encoder operate directly on the joint-level input and the confidence-modulated adjacency, and the representations they produce are dominated by the topology of the human skeleton, which is shared across all action categories and is recovered with high fidelity by the contrastive pre-training stage. The upper layers, in contrast, integrate features over wider spatial and temporal receptive fields and encode composition patterns that are specific to the downstream label structure. Releasing the bottom layers during fine-tuning allows label-driven gradients to overwrite the pre-trained skeleton topology, which collapses the inter-class structure built up during contrastive learning and is observed empirically as a regression toward the supervised ST-GCN baseline. Freezing the upper layers, on the other hand, locks the composition patterns in their pre-trained configuration and prevents adaptation to the discriminative requirements of the labeled task. The 4-layer freeze separates these two regimes at the layer where the dominant signal transitions from skeleton topology to action composition, and the empirical accuracies of 89.8%, 90.5%, 91.2%, and 90.1% under freezing depths of 0, 2, 4, and 6 layers reflect this structural transition rather than a sweep over a smooth hyperparameter. Action recognition performance comparison To comprehensively evaluate the effectiveness of the proposed HGCL method, this section analyzes performance across three dimensions: classification accuracy, convergence efficiency, and feature space distribution. ST-GCN, CTR-GCN, and InfoGCN are selected as baseline methods. These represent the classic GCN architecture, channel topology refinement strategies, and information bottleneck optimization paradigms, respectively. All baseline methods are retrained on the identical OpenPose-extracted 18-joint skeleton data used by HGCL. All methods share the same data partition, preprocessing pipeline, and evaluation protocol on both datasets. Figure 3 displays the Top-1 and Top-5 classification accuracy comparison for the four methods. As a pioneering work in skeletal GCNs, ST-GCN achieves a Top-1 accuracy of 81.5% and a Top-5 accuracy of 94.2%. This result validates the effectiveness of spatial graph modeling for capturing joint topology. However, its fixed adjacency matrix design limits its adaptability to dynamic motion patterns. CTR-GCN achieves significant performance gains by introducing channel topology refinement modules. Its Top-1 accuracy reaches 86.8% (a 5.3 percentage point increase over ST-GCN), and its Top-5 accuracy is 96.8%. This indicates that dynamically adjusting graph structures across channel dimensions better models joint dependencies at different semantic levels. InfoGCN further integrates information bottleneck theory, slightly increasing Top-1 accuracy to 87.3% and Top-5 accuracy to 97.2%. This confirms that information compression strategies help extract discriminative features. The proposed HGCL method achieves a Top-1 accuracy of 91.2%, which is 3.9 percentage points higher than InfoGCN, and a Top-5 accuracy of 98.5%. It achieves the best performance on both metrics. HGCL’s advantage stems from joint modeling of global and local semantics within the hierarchical contrastive learning framework, as well as its graph-aware augmentation strategy. Comparison of classification performance on the UCF101 dataset. The performance gap among baselines is consistent with the structural assumptions each method imposes on the skeleton graph. ST-GCN relies on a fixed adjacency matrix derived from human anatomy, which is well suited to actions whose discriminative joints are connected by short paths in the skeleton graph but which leaves no mechanism to re-weight edges when an action involves long-range coordination, such as full-body throwing or jumping; the resulting recognition errors concentrate on action classes with strong end-effector dynamics. CTR-GCN refines the channel-wise topology and recovers part of this gap, yet it remains a fully supervised end-to-end model whose channel refinement is driven by classification gradients alone, which limits its generalization when the training set does not cover all topological variants of an action. InfoGCN further compresses the channel topology with an information bottleneck and consequently improves over CTR-GCN on canonical 3D motion-capture inputs, but its formulation does not consume the per-joint detection confidence produced by 2D pose estimators, so the noisy joints from OpenPose are propagated into the bottleneck layer without attenuation and the bottleneck itself cannot distinguish a low-quality joint from an intrinsically ambiguous one. HGCL departs from these baselines on exactly the dimensions left untreated by them: the graph-aware augmentation exposes the encoder to controlled topological corruption during pre-training, the hierarchical contrastive objective injects local part-level supervision that is unavailable in any of the supervised baselines, and the confidence-modulated adjacency directly attenuates information propagation along edges with low-confidence endpoints. The cumulative effect of these three design choices is what produces the 3.9 percentage point margin over InfoGCN on UCF101. Figure 4 compares the validation accuracy curves over training epochs for different strategies. The baseline model trained from scratch converges slowly. Its accuracy growth slope in the first 20 epochs is significantly lower than that of pre-trained methods, eventually reaching only 82% accuracy after 100 epochs. With self-supervised pre-training, InfoGCN shows improved convergence, exceeding 80% accuracy by epoch 40 and converging to approximately 87%. HGCL demonstrates the fastest convergence and highest final performance. By epoch 40, its accuracy is nearly 88%, which is about 8 percentage points higher than InfoGCN. It finally stabilizes above 91%. Notably, HGCL’s accuracy improvement rate in the early stages (epochs 1-20) is much higher than other methods. This is due to the highly discriminative features learned during the hierarchical contrastive pre-training phase. The gap between curves is most pronounced around epoch 40, where the performance difference between HGCL and training from scratch exceeds 12 percentage points. This fully illustrates the key role of self-supervised pre-training in accelerating convergence and enhancing generalization. Comparison of convergence speed across different training strategies. Table 3 provides a quantitative analysis of representation quality from the perspective of feature space distribution. Randomly initialized networks produce highly chaotic distributions. The intra-class distance is 1.852 while the inter-class distance is only 0.214. This results in a DBI of 8.654 and a CH score of 12.5, indicating severe overlap between categories. Supervised training of ST-GCN significantly improves this distribution. Intra-class distance drops to 0.941 and inter-class distance increases to 2.583. DBI falls to 1.247 and the CH score rises to 458.2. HGCL achieves the best results across all metrics. Intra-class distance is further compressed to 0.428 (54.5% lower than ST-GCN), indicating more compact clustering. Inter-class distance expands to 4.915 (90.3% higher than ST-GCN), showing clearer decision boundaries. DBI drops to 0.612 and the CH score reaches 1024.7. These results show that HGCL simultaneously optimizes intra-class compactness and inter-class separability. Table 3Quantitative comparison of feature space distribution characteristics.MetricRandom InitST-GCNHGCL (Ours)ImprovementAvg Intra-class Dist1.8520.9410.428-54.5%Avg Inter-class Dist0.2142.5834.915+90.3%Davies-Bouldin Index8.6541.2470.612-50.9%Calinski-Harabasz Score12.5458.21024.7+123.6% Quantitative comparison of feature space distribution characteristics. To provide an intuitive view of the feature space quality, Fig. 5 visualizes the t-SNE embeddings of 10 representative action classes under three conditions. With random initialization, features of different classes are entirely intermixed. ST-GCN produces preliminary clustering but semantically similar actions still overlap. HGCL yields compact and well-separated clusters, consistent with the quantitative metrics in Table 3. t-SNE visualization of learned feature representations. The feature-space geometry can be examined from a complementary perspective by measuring how perturbations in the input propagate to the learned representation. The semantic consistency bound established alongside the augmentation analysis in the methods section relies on the trained encoder being Lipschitz continuous with a tractable constant , an assumption that the bound itself does not verify on the actual model. To examine this on the encoder produced by the full self-supervised pre-training procedure, 10,000 sample pairs are drawn uniformly without replacement from the UCF101 test set, and the empirical ratio is recorded for every pair. The median of the ratio across all pairs is 0.42, the 95th percentile is 1.18, and the maximum observed value is 2.36. The empirical Lipschitz constant is therefore bounded by a small numerical value over the test distribution rather than diverging, which corroborates that the consistency bound holds with on the order of unity for the actual trained model and that the feature contraction reflected in the reduced intra-class distance in Table 3 arises from the discriminative geometry induced by the contrastive objective rather than from an indiscriminate compression of all input pairs. The sampling procedure provides a lower bound on the true Lipschitz constant since the supremum over the entire input space may be attained at pairs outside the sample, but a bounded estimate on a representative test sample is sufficient to establish that the bound applies to the operating regime of the proposed method rather than only to a hypothetical encoder. To examine whether the above results on UCF101 transfer to a different data distribution, the same set of experiments is conducted on the HMDB51 dataset. Figure 6 displays the Top-1 and Top-5 classification accuracy on HMDB51. All methods exhibit lower absolute accuracy compared to UCF101, reflecting the increased difficulty of skeleton-based recognition on cinematic video data with complex backgrounds and subtle inter-class differences. ST-GCN achieves a Top-1 accuracy of 55.2% and a Top-5 accuracy of 82.6%. CTR-GCN improves Top-1 accuracy to 60.4% through dynamic topology refinement, and InfoGCN reaches 61.5% with information bottleneck optimization. HGCL achieves the highest Top-1 accuracy of 66.8% and Top-5 accuracy of 89.7%, outperforming InfoGCN by 5.3 percentage points in Top-1 accuracy. This margin exceeds the 3.9 percentage point advantage observed on UCF101, suggesting that hierarchical contrastive learning provides greater benefits on challenging datasets where discriminative local body part features are critical for distinguishing visually similar actions. Comparison of classification performance on the HMDB51 dataset. The widened margin on HMDB51 admits an interpretation along the same structural axis. Cinematic data in HMDB51 are characterized by frequent camera motion, partial occlusion, and tight inter-class similarity, all of which inflate the proportion of low-confidence joints in the OpenPose output. Baselines that ignore the confidence signal therefore receive proportionally noisier inputs on HMDB51 than on UCF101, and the relative penalty grows with dataset difficulty. The part-level contrastive branch additionally compensates for the weaker global cues by providing supervision at body regions that remain reliably detected even when the full skeleton is partially corrupted, which is consistent with the enlarged margin observed on HMDB51 relative to UCF101. Figure 7 compares the convergence behavior of different training strategies on HMDB51. The overall convergence patterns are consistent with those observed on UCF101. The model trained from scratch converges slowly and plateaus around 55% accuracy. InfoGCN with self-supervised pre-training converges faster and reaches approximately 62%. HGCL demonstrates the fastest convergence and the highest final accuracy, stabilizing near 67% by epoch 60. At epoch 40, the accuracy gap between HGCL and training from scratch exceeds 14 percentage points, further confirming the benefit of hierarchical contrastive pre-training for accelerating convergence on datasets with limited training samples per class. Comparison of convergence speed on the HMDB51 dataset. Few-shot learning performance comparison In practical applications, obtaining large-scale labeled data is often costly. Therefore, a model’s ability to learn with limited labels is of significant practical value. This section systematically evaluates HGCL’s few-shot performance by controlling the proportion of labeled data during fine-tuning. Experiments use six gradients: 1%, 5%, 10%, 20%, 50%, and 100%. Samples are selected via stratified random sampling to ensure balanced distributions. Figure 8 shows the Top-1 accuracy trends for the three methods. Overall, HGCL maintains a leading advantage across all proportions. This advantage is more pronounced at lower ratios. In the extreme 1% few-shot condition, ST-GCN trained from scratch achieves only 15.2% accuracy. InfoGCN improves this to 28.5% via self-supervised pre-training. HGCL further raises accuracy to 36.8%, providing a 21.6 percentage point gain over ST-GCN and an 8.3 point gain over InfoGCN. While all methods improve as more data is added, their growth rates vary. HGCL reaches 61.5% accuracy at 5% labels, whereas ST-GCN requires 20% labels to reach a similar level (60.2%). This suggests HGCL needs only one-fourth of the labels to match ST-GCN’s performance. At 10% and 20% labels, HGCL maintains a lead of about 6-7 percentage points over InfoGCN. At 50% labels, HGCL reaches 88.5% accuracy, which is close to its full-data performance of 91.2% (Table 4). Semi-supervised performance comparison under different label proportions. Table 4Top-1 accuracy (%) under different label ratios on UCF101. Each cell reports mean±std over five independently seeded fine-tuning runs; self-supervised pre-training is performed once and shared across seeds.Label Proportion1%5%10%20%50%100%ST-GCN (Scratch)15.2±1.832.4±1.545.8±1.260.2±0.975.6±0.681.5±0.4ST-GCN + GraphAug (no SSL)20.5±1.641.6±1.356.8±1.068.4±0.879.6±0.584.3±0.4InfoGCN (Self-supervised)28.5±1.552.1±1.268.4±0.976.5±0.784.2±0.587.3±0.4HGCL (ours)36.8±1.461.5±1.175.2±0.882.1±0.688.5±0.491.2±0.3 Top-1 accuracy (%) under different label ratios on UCF101. Each cell reports mean±std over five independently seeded fine-tuning runs; self-supervised pre-training is performed once and shared across seeds. The ST-GCN (Scratch) row is trained on raw skeleton data without augmentation, while the ST-GCN + GraphAug (no SSL) row applies the proposed graph-aware augmentation strategy during supervised training but omits the contrastive pre-training stage. This intermediate row decomposes the overall gain of HGCL over the scratch baseline into two distinct sources: the augmentation strategy alone accounts for the difference between the first two rows, and the hierarchical contrastive pre-training accounts for the difference between the second and the fourth rows. At 1% labels, augmentation alone contributes 5.3 percentage points, while the contrastive pre-training stage contributes an additional 16.3 percentage points on top of the augmented baseline. At 100% labels, augmentation alone contributes 2.8 percentage points and the contrastive pre-training stage contributes 6.9 percentage points. The relative weight of the two sources shifts with label availability, with the contrastive objective remaining the dominant source of label efficiency across the entire range and the augmentation strategy providing a stable regularization effect whose magnitude is largely independent of the label budget. The standard deviations reported in the table remain within 2 percentage points across all settings and decrease monotonically as the label ratio grows, indicating that the few-shot advantage of HGCL is a stable property of the framework rather than an artifact of favorable seed selection. The same few-shot evaluation protocol is applied to the HMDB51 dataset to verify whether the label efficiency advantage of HGCL transfers across datasets. Figure 9 and Table 5 present the few-shot results on HMDB51, where the same decomposition between augmentation and contrastive pre-training holds. At 1% labels, augmentation alone contributes 3.0 percentage points and the contrastive pre-training stage contributes an additional 14.7 percentage points; at 100% labels the two stages contribute 3.3 and 8.3 percentage points respectively. The asymmetry between low-label and high-label regimes is preserved on HMDB51, which indicates that the contrastive objective remains the dominant source of label efficiency on a dataset where the augmentation strategy alone is insufficient to compensate for the cinematic input distribution. HGCL reaches 48.6% accuracy at 5% labels while ST-GCN requires approximately 20% labels to reach a comparable level of 44.6%, preserving the four-fold reduction in label requirement observed on UCF101. The advantage of HGCL over InfoGCN remains stable at approximately 5–8 percentage points across all label ratios, and the standard deviations remain within the same range as those on UCF101. The ranking among the four rows is preserved across every label ratio on both datasets, which supports the conclusion that the few-shot advantage of HGCL is a property of the framework rather than a sampling artifact of a particular dataset. Semi-supervised performance comparison under different label proportions on HMDB51. Table 5Top-1 accuracy (%) under different label ratios on HMDB51. Each cell reports mean±std over five independently seeded fine-tuning runs; self-supervised pre-training is performed once and shared across seeds.Label Proportion1%5%10%20%50%100%ST-GCN (Scratch)10.8±1.924.5±1.635.2±1.344.6±1.052.1±0.755.2±0.5ST-GCN + GraphAug (no SSL)13.8±1.728.4±1.439.6±1.148.2±0.954.5±0.658.5±0.5InfoGCN (Self-supervised)20.3±1.639.8±1.350.6±1.056.4±0.860.2±0.661.5±0.5HGCL (Ours)28.5±1.548.6±1.257.8±0.962.5±0.765.4±0.566.8±0.4 Top-1 accuracy (%) under different label ratios on HMDB51. Each cell reports mean±std over five independently seeded fine-tuning runs; self-supervised pre-training is performed once and shared across seeds. Validity verification of key components To systematically evaluate the contribution of core components, this section includes ablation studies, visualization of augmentation strategies, and hyperparameter sensitivity analysis. Figure 10 illustrates the performance changes as components are added. With ST-GCN as the baseline (81.5%), introducing graph-aware augmentation improves accuracy to 84.3%. This gain of 2.8 percentage points shows that augmentations maintaining topology can effectively expand sample diversity. Adding the hierarchical contrastive framework further increases accuracy to 87.6% (a 3.3 point gain), validating the importance of modeling both global and local semantics. The confidence weighting mechanism adds another 1.8 points (reaching 89.4%), demonstrating that utilizing detection confidence suppresses noise from low-quality joints. The final HGCL model reaches 91.2%. The contributions of the four components are approximately 28.9%, 34.0%, 18.6%, and 18.5%, respectively, with hierarchical contrastive learning being the most significant contributor. Ablation study results for HGCL components. To further examine two design choices that are easy to question but rarely ablated in skeleton self-supervised work, Table 6 reports the recognition accuracy on UCF101 under alternative body part partitions and alternative confidence aggregation operators. The body part partition is varied from a coarse three-part split (head–torso, upper limbs, lower limbs) to a fine seven-part split (head, torso, left upper arm, right upper arm, left lower limb, right lower limb, hands–feet), and an additional data-driven five-part split is constructed via spectral clustering on the symmetrically normalized adjacency matrix to disentangle the role of anatomical priors from that of partition granularity. The confidence aggregator that determines in Eq. (18) is replaced by the minimum, the maximum, the arithmetic mean, the geometric mean used in the proposed method, and a small learnable multilayer perceptron operating on the pair . Table 6Top-1 accuracy (%) on UCF101 under alternative body part partitions and confidence aggregation operators. Default settings are marked with †.VariantTop-1 (%)Body part partition 3 parts (head–torso, upper limbs, lower limbs)88.5 5 parts (anatomical)91.2 5 parts (data-driven via spectral clustering)90.9 7 parts (refined anatomical)90.3Confidence aggregator 89.7 88.9 Arithmetic mean 90.3 Geometric mean 91.2 Learnable MLP 91.0 Top-1 accuracy (%) on UCF101 under alternative body part partitions and confidence aggregation operators. Default settings are marked with †. The partition results show that refining the anatomical five-part split into a seven-part split does not yield additional gains, which indicates that the limiting factor is not partition granularity itself but the size of the resulting sub-graphs, since further refinement leaves too few joints per part for the part-level contrastive signal to remain stable. The coarse three-part split incurs a drop of 2.7 percentage points, confirming that the absence of intermediate part-level discrimination is a genuine bottleneck rather than the cost of an over-engineered partition. The data-driven five-part split obtained from spectral clustering reaches 90.9%, within 0.3 percentage points of the anatomical five-part configuration. This near-equivalence indicates that the contribution of the hierarchical contrastive design arises from the existence of a moderately granular structural hierarchy rather than from the specific anatomical labels assigned to the parts. The aggregator results are consistent with the design rationale stated alongside Eq. (18). The \min operator is overly conservative on edges where one endpoint is reliably detected and the other is not, while the \max operator allows information from a low-confidence endpoint to propagate freely whenever the opposite endpoint is reliable, both of which produce a 1.5 to 2.3 percentage point drop relative to the geometric mean. The arithmetic mean is invariant under exchanging a (0.1, 0.9) confidence pair with a (0.5, 0.5) pair and therefore loses sensitivity to the worst-endpoint case, costing 0.9 percentage points. A learnable MLP-based aggregator reaches 91.0%, marginally below the geometric mean. Given that the geometric mean introduces no additional parameters, admits a direct interpretation as the propagation strength along a two-endpoint edge under multiplicative reliability, and is invariant under uniform rescaling of the confidence channel, it is retained as the operator of choice in the proposed method. Figure 11 visualizes the proposed augmentation strategies. Subfigure (a) shows the original 2D skeleton with 18 joints and their topology. Different colors represent body parts: head, torso, arms, and legs. Subfigure (b) shows spatial rotation, where a rotation is applied to simulate different viewing angles while maintaining relative joint positions. Subfigure (c) displays joint masking, where 15% of joints are randomly hidden (marked with red Xs). This forces the model to infer action semantics from incomplete observations. Subfigure (d) shows Gaussian jittering, adding noise (σ =0.025) to coordinates to simulate localization errors. All strategies respect kinematic constraints and skeletal topology. Visualization of graph-aware data augmentation strategies. Figure 12 analyzes six key hyperparameters covering the temperature coefficient, the negative queue length, the hierarchical loss weight, the joint mask ratio, the base noise intensity, and the spatial rotation range that together govern the contrastive learning stage and the augmentation pipeline. Subfigure (a) shows sensitivity to the temperature coefficient τ, with accuracy peaking at 91.2% when τ =0.1; smaller values make the softmax distribution too sharp and lead to unstable gradients, while larger values decrease discriminability between positive and negative samples. Subfigure (b) shows the impact of queue length K, with accuracy peaking at ; smaller queues provide insufficient negative samples, while larger ones may introduce stale features. Subfigure (c) analyzes the hierarchical loss weight λ, with performance climbing from 87.3% at λ =0, where only global contrastive learning is active, to 91.2% at λ =0.5, before slightly degrading as λ continues to grow and the local objective begins to overshadow the global one. Subfigure (d) shows sensitivity to the joint mask ratio r, peaking at r=0.15 between an insufficient regularization regime at low ratios and an over-destruction regime at high ratios. Subfigure (e) reports sensitivity to the base noise intensity , with accuracy varying within a one-percentage-point band across the interval from 0.005 to 0.02 and peaking at , the value to which the noise magnitude is calibrated against the coordinate standard deviation of the training set. The width of this plateau is consistent with the linear scaling of the bone length deviation bound with , so that perturbations within the calibrated range remain kinematically plausible while larger values either degrade the augmented skeleton or introduce noise that exceeds realistic localization errors. Subfigure (f) reports sensitivity to the spatial rotation range , where the accuracy stays within 0.6 percentage points of the peak across the interval from to and drops more rapidly at the two endpoints, reflecting insufficient viewpoint diversity at small angles and the growth of the cosine term in the bone length bound at large angles. Across all six subfigures the curves form broad plateaus around their respective optima rather than narrow peaks, which supports the interpretation that the proposed method is robust to hyperparameter perturbations rather than over-tuned to a specific configuration of the parameter table. Hyperparameter sensitivity analysis. Robustness to skeleton quality degradation Pose estimation algorithms inevitably introduce coordinate noise and joint detection failures in practical deployment, particularly under occlusion, motion blur, and complex backgrounds. To evaluate the robustness of different methods under such conditions, two sets of controlled degradation experiments are conducted on the test set. In the first set, additive Gaussian noise with standard deviation is injected into the normalized joint coordinates to simulate pose estimation localization errors of varying severity. In the second set, a proportion of joints are randomly dropped at each frame and replaced with training set mean coordinates to simulate joint detection failures caused by occlusion. All four methods compared in Section Experimental settings and results analysis.2 are included to maintain consistency. All models are trained on the original clean data and evaluated directly on the degraded test data without any retraining or adaptation. Table 7 reports the classification accuracy under increasing coordinate noise levels. All methods exhibit monotonic performance degradation as noise intensity grows, which is consistent with the expected behavior of spatial feature corruption. ST-GCN shows the largest absolute decline of 18.4 percentage points from clean to , as its fixed adjacency matrix cannot adapt to noisy joint positions. CTR-GCN and InfoGCN achieve similar degradation profiles with total losses of 17.0 and 16.8 percentage points respectively, benefiting from their adaptive graph topology mechanisms. HGCL demonstrates the smallest degradation of 15.0 percentage points across the full noise range, retaining 76.2% accuracy at . The performance gap between HGCL and InfoGCN progressively widens from 3.9 percentage points under clean conditions to 5.7 percentage points at the highest noise level. This increasing advantage can be attributed to the Gaussian jittering applied during graph structure-aware pre-training, which exposes the encoder to coordinate perturbations of comparable magnitude and enables the learned representations to tolerate localization inaccuracies. The confidence-based weighting mechanism further mitigates the impact of noisy joints by reducing their contribution during feature aggregation. Table 7Top-1 accuracy (%) under different levels of Gaussian coordinate noise.Noise 0.000.020.050.080.10ST-GCN81.579.274.368.763.1CTR-GCN86.885.080.575.269.8InfoGCN87.385.681.476.170.5HGCL (Ours)91.290.086.381.576.2Δ(HGCL−InfoGCN)+3.9+4.4+4.9+5.4+5.7 Top-1 accuracy (%) under different levels of Gaussian coordinate noise. Table 8 presents the results under increasing joint drop ratios. A consistent ranking among methods is maintained across all degradation levels. ST-GCN suffers the largest total decline of 26.2 percentage points at 40% joint drop, as the loss of nearly half of the input joints severely disrupts its spatial graph convolution. CTR-GCN and InfoGCN show total declines of 24.4 and 24.2 percentage points respectively, indicating that their adaptive topology strategies provide limited benefit when input nodes are entirely absent. HGCL achieves a total decline of 21.4 percentage points under the same condition, retaining 69.8% accuracy. The advantage over InfoGCN grows from 3.9 to 6.7 percentage points as the drop ratio increases from 0% to 40%. This resilience is consistent with the design of the joint masking augmentation in the pre-training phase, which trains the encoder to reconstruct action semantics from incomplete skeleton observations. The part-level contrastive loss in the hierarchical framework also contributes, as each body part representation captures self-contained local motion information that partially compensates for missing joints in other regions. Together, these two sets of experiments confirm that HGCL maintains reliable recognition performance under realistic skeleton quality degradation arising from pose estimation inaccuracies and joint occlusions. Table 8Top-1 accuracy (%) under different joint drop ratios.Drop Ratio 0%10%20%30%40%ST-GCN81.578.172.464.855.3CTR-GCN86.883.978.671.562.4InfoGCN87.384.579.372.463.1HGCL (Ours)91.289.185.078.669.8Δ(HGCL−InfoGCN)+3.9+4.6+5.7+6.2+6.7 Top-1 accuracy (%) under different joint drop ratios. To verify whether the robustness advantage of HGCL generalizes across datasets, the same degradation experiments are repeated on HMDB51. Tables 9 and 10 report the robustness results on HMDB51. The degradation patterns are consistent with those observed on UCF101. Under Gaussian noise at , HGCL retains 53.5% accuracy with a total decline of 13.3 percentage points, compared to 13.9 for InfoGCN and 13.9 for ST-GCN. Under 40% joint drop, HGCL achieves 47.5% accuracy while InfoGCN drops to 40.7% and ST-GCN to 34.5%. The performance advantage of HGCL over InfoGCN remains stable at approximately 5-7 percentage points across all degradation levels on both datasets, confirming that the robustness conferred by graph structure-aware augmentation and confidence-based weighting is a consistent property of the method rather than an artifact of a particular data distribution. Table 9Top-1 accuracy (%) under Gaussian coordinate noise on HMDB51.Noise 0.000.020.050.080.10ST-GCN55.253.149.545.241.3CTR-GCN60.458.655.150.846.5InfoGCN61.559.856.352.047.6HGCL (Ours)66.865.462.157.853.5Δ(HGCL−InfoGCN)+5.3+5.6+5.8+5.8+5.9 Top-1 accuracy (%) under Gaussian coordinate noise on HMDB51. Table 10Top-1 accuracy (%) under different joint drop ratios on HMDB51.Drop Ratio 0%10%20%30%40%ST-GCN55.252.047.341.234.5CTR-GCN60.457.553.147.039.8InfoGCN61.558.854.248.140.7HGCL (Ours)66.864.560.654.847.5Δ(HGCL−InfoGCN)+5.3+5.7+6.4+6.7+6.8 Top-1 accuracy (%) under different joint drop ratios on HMDB51. Comparison with recent state-of-the-art methods To further contextualize the performance of the proposed HGCL framework within the broader landscape of action recognition, we compare HGCL with seven representative methods published between 2022 and 2025, organized into two groups in Table 11. The first group consists of supervised methods that establish the practical upper and lower bounds of the two benchmarks under full label supervision, including the large-scale multi-modal models UniFormerV2 43 and InternVideo 44, the 3D heatmap-based PoseConv3D 45, and the appearance-skeleton hybrid InBRwSANet 46. The second group consists of recent self-supervised skeleton methods that share the input modality of HGCL, including the cross-part bidirectional distillation framework 27, ActCLR 47, and SkeletonMAE 48. The comparison results are summarized in Table 11. Table 11Performance comparison with representative methods. Methods are grouped by modality and supervision regime, and ordered by publication year within each group.MethodYearModality/supervisionUCF101HMDB51UniFormerV2 432024RGB, supervised99.6089.40InternVideo 442023RGB+text, supervised99.1086.40PoseConv3D 4520223D heatmap, supervised87.8060.50InBRwSANet 462025RGB+skeleton, supervised91.8078.80CPBD 2720242D skeleton, self-supervised88.9063.20ActCLR4720232D skeleton, self-supervised89.6064.10SkeletonMAE4820242D skeleton, self-supervised90.2064.80HGCL (Ours)20262D skeleton, self-supervised91.2066.80 Performance comparison with representative methods. Methods are grouped by modality and supervision regime, and ordered by publication year within each group. Multi-modal methods that leverage RGB appearance combined with visual-language alignment achieve near-saturation accuracy on UCF101 (above 99%) and strong results on HMDB51 (86–89%). The substantial gap between these methods and all skeleton-based approaches reflects the inherent information asymmetry between rich appearance features and compact skeletal coordinates, and confirms that our reported accuracy falls within a reasonable range for the skeleton modality. Within the skeleton-based category, HGCL achieves 91.20% on UCF101, surpassing PoseConv3D 45 (87.80%) by 3.4 percentage points. This improvement is notable given that PoseConv3D employs a 3D-CNN backbone operating on heatmap volumes extracted by a high-quality HRNet pose estimator, whereas HGCL relies on lightweight OpenPose coordinates processed through a GCN encoder, indicating that hierarchical contrastive pre-training can effectively compensate for differences in pose estimation quality and backbone capacity. InBRwSANet 46 reports 91.80% on UCF101 and 78.80% on HMDB51 through six parallel inverted residual bottleneck blocks coupled with a self-attention module trained in a fully supervised manner, which places HGCL within 0.6 percentage points of InBRwSANet on UCF101 and at a wider margin on HMDB51. The two methods occupy different points on the supervision–accuracy trade-off rather than being directly comparable under identical training conditions: InBRwSANet is trained end-to-end with full action labels and an architecture tailored to RGB-style appearance features, while HGCL relies on label-free hierarchical contrastive pre-training followed by light fine-tuning on the same skeleton modality. The closeness on UCF101 indicates that the proposed self-supervised framework reaches the accuracy of strong supervised methods on diverse activity scenarios without using any action labels during pre-training, whereas the larger gap on HMDB51 reflects the difficulty of recognizing cinematic actions from skeleton coordinates alone, as appearance and contextual cues that supervised RGB-aware models can exploit are not available to HGCL. Among the recent self-supervised skeleton baselines listed in the second group of Table 11, HGCL surpasses the cross-part bidirectional distillation framework (denoted CPBD) 27, ActCLR 47, and SkeletonMAE 48 by 2.3, 1.6, and 1.0 percentage points on UCF101, and by 3.6, 2.7, and 2.0 percentage points on HMDB51, respectively. CPBD is the most informative reference point for isolating the contribution of the hierarchical contrastive design, since it also decomposes skeleton sequences into body parts and applies a part-level objective during self-supervised pre-training. CPBD couples the parts through bidirectional knowledge distillation between two pose-decomposed streams, which enforces representation transfer between part streams trained on identical reconstruction targets and therefore requires the two streams to converge to a shared local solution. The hierarchical contrastive design in HGCL couples the parts through a contrastive objective in which part-level negatives are confined within the same anatomical region and the global branch supplies an overarching semantic constraint balanced by the coefficient λ, which preserves the freedom of part-level and sequence-level objectives to operate on different granularities rather than collapsing into a single target. The accuracy margin of 2.3 percentage points on UCF101 and 3.6 percentage points on HMDB51 between HGCL and CPBD under matched 2D skeleton inputs is therefore attributable to the hierarchical contrastive coupling itself rather than to the use of part-level supervision in general. ActCLR 47 and SkeletonMAE 48 represent contrastive and masked-reconstruction self-supervised paradigms respectively, and the consistent advantage of HGCL across both indicates that the joint use of graph-aware augmentation, hierarchical contrastive coupling, and confidence modulation produces representations that are more discriminative than either contrastive or reconstruction objectives applied in isolation. Across all methods listed in Table 11, a consistent performance degradation from UCF101 to HMDB51 is observed. HGCL exhibits a cross-dataset accuracy drop of 24.4 percentage points, which falls within the range observed for skeleton-based methods (PoseConv3D: 27.3 points; InBRwSANet: 13.0 points), where the smaller drop of InBRwSANet is attributable to its supervised end-to-end training with appearance-aware features that mitigate the cinematic-data difficulty of HMDB51 at the cost of label dependency. This consistent cross-dataset pattern further corroborates the validity of our experimental setup and indicates that the learned representations generalize across datasets with different characteristics rather than overfit to a particular benchmark. Failure mode analysis on HMDB51 To understand why the absolute accuracy on HMDB51 remains substantially below that on UCF101 for all skeleton-based methods, the per-class Top-1 accuracies of HGCL and InfoGCN on the HMDB51 test set are computed and sorted in ascending order of HGCL accuracy. The resulting accuracy curves are presented in Fig. 13, with the five lowest-ranked classes for HGCL highlighted as the bottom-5 region. The horizontal axis carries only the class rank rather than individual class names, so that the discussion focuses on the shape of the distribution rather than on a particular labeling of the classes. Per-class Top-1 accuracy of HGCL and InfoGCN on HMDB51, sorted in ascending order of HGCL accuracy. The shaded region marks the five lowest-ranked classes for HGCL, and the dashed horizontal lines indicate the overall test accuracy of each method. The accuracy curve exhibits a pronounced long-tail shape rather than a uniform spread. A small number of classes at the left end of the ranking reach Top-1 accuracies between 28% and 42%, while the remaining classes cluster between 60% and 90%, with the right tail of the distribution approaching the recognition rates observed on UCF101. The overall HMDB51 accuracy of HGCL therefore reflects a small set of structurally difficult classes pulling down a body of classes whose accuracy is already comparable to the UCF101 regime, rather than a uniformly weaker recognition capability across the dataset. The two curves in the bottom-5 region are almost coincident, in contrast to the wider gap maintained over the upper two thirds of the ranking, which indicates that the residual error in this region is shared between HGCL and InfoGCN regardless of the differences in their pre-training and architectural designs. Inspection of the classes in the bottom-5 region reveals two structural properties shared across them. The discriminative motion is concentrated at facial regions or at hand-held-object regions that are either absent from the 18-joint OpenPose topology or represented by at most a single end joint, so that the skeleton input lacks the spatial substrate required to distinguish between actions whose joint trajectories follow nearly identical paths but differ in the identity of the held object, and to separate among facial expressions that involve no measurable displacement of any tracked joint. The distinguishing cue between adjacent classes in this region lies in the identity of the held object or in the configuration of facial musculature rather than in the trajectory of any tracked joint, and the skeleton modality is intrinsically blind to these cues regardless of the recognition framework applied to it. The near-coincidence of the two curves in the bottom-5 region in Fig. 13 is consistent with this interpretation, since the failure is determined by what the input representation makes accessible to the encoder rather than by the encoder design itself. A natural direction for closing this residual gap is to extend HGCL into a multimodal setting, either by augmenting the 18-joint skeleton with sparse facial and hand landmarks extracted by a higher-resolution pose estimator or by introducing an appearance branch that processes object-relevant image patches in addition to the skeleton stream, both of which preserve the privacy and efficiency advantages of skeleton input on the classes where appearance is dispensable while supplying the missing cues on the subset of classes where it is essential. Discussion This paper proposes a Hierarchical Graph Contrastive Learning (HGCL) method for skeleton-based action recognition. Systematic experiments on UCF101 and HMDB51 have validated its effectiveness. HGCL achieves 91.2% and 66.8% Top-1 accuracy on UCF101 and HMDB51, outperforming InfoGCN by 3.9 and 5.3 percentage points respectively. The larger margin on HMDB51 suggests that hierarchical contrastive learning provides greater benefits on more challenging datasets where discriminative local body part features become critical. Ablation studies confirm that the hierarchical contrastive module is the most significant contributor (34.0% of total improvement). HGCL achieves 91.2% and 66.8% Top-1 accuracy on UCF101 and HMDB51, outperforming InfoGCN by 3.9 and 5.3 percentage points respectively. The larger margin on HMDB51 suggests that hierarchical contrastive learning provides greater benefits on more challenging datasets where discriminative local body part features become critical. Ablation studies confirm that the hierarchical contrastive module is the most significant contributor (34.0% of total improvement). The confidence-modulated adjacency matrix (Eq. 18) provides an explicit mechanism for suppressing unreliable joints during graph convolution and feature pooling. Controlled degradation experiments confirm its practical benefit: the advantage of HGCL over InfoGCN widens from 3.9 to 5.7 percentage points on UCF101 as coordinate noise increases to , and from 3.9 to 6.7 percentage points under 40% joint drop. Consistent trends are observed on HMDB51. The confidence-modulated adjacency matrix (Eq. 18) provides an explicit mechanism for suppressing unreliable joints during graph convolution and feature pooling. Controlled degradation experiments confirm its practical benefit: the advantage of HGCL over InfoGCN widens from 3.9 to 5.7 percentage points on UCF101 as coordinate noise increases to , and from 3.9 to 6.7 percentage points under 40% joint drop. Consistent trends are observed on HMDB51. In 1% few-shot scenarios, HGCL improves upon training from scratch by 21.6 and 17.7 percentage points on UCF101 and HMDB51, demonstrating the advantage of self-supervised pre-training in reducing label dependency. In 1% few-shot scenarios, HGCL improves upon training from scratch by 21.6 and 17.7 percentage points on UCF101 and HMDB51, demonstrating the advantage of self-supervised pre-training in reducing label dependency. The evaluation in this paper is intentionally confined to UCF101 and HMDB51, both of which provide RGB video as the primary input and admit 2D skeleton extraction through an off-the-shelf pose estimator. This choice is dictated by the design of HGCL rather than by data availability. The confidence-modulated adjacency in Eq. (18) and the confidence-weighted pooling in Eqs. (22) and (23) operate on the per-frame detection confidence produced by the pose estimator, which in the present setting carries genuine information about occlusion, motion blur, and estimator uncertainty. On clean motion-capture benchmarks such as NTU RGB+D, the joint coordinates are obtained from depth sensors and the per-joint confidence is either absent or saturated near unity, which renders the confidence modulation methodologically inert and reduces HGCL to a non-confidence variant whose evaluation would not test the central mechanism of the method. UCF101 and HMDB51 therefore expose the full pipeline to the realistic confidence distribution that the method is designed to exploit, and a meaningful extension to clean three-dimensional skeletons would require constructing a confidence proxy from estimator ensembles or temporal consistency rather than reusing directly. This extension is left as future work. Conclusion To address the challenges in skeleton-based action recognition—including heavy reliance on labeled data, the destruction of skeletal topology by traditional augmentation, and underutilization of joint confidence—this paper introduces the Hierarchical Graph Contrastive Learning (HGCL) method. A graph structure-aware augmentation strategy generates diverse training samples while remaining topology-aware and tolerant to controlled structural corruption, with a theoretical bound on bone length deviation among surviving bones limiting kinematic distortion. A hierarchical contrastive learning framework jointly optimizes global sequence-level and local body part-level representations, where the part-level loss explicitly contrasts corresponding body regions across augmented views. A confidence-modulated adjacency matrix incorporates pose estimation reliability directly into graph convolution and feature pooling, suppressing interference from low-quality joints throughout the pipeline. Experimental results on two benchmarks show that HGCL achieves 91.2% and 66.8% Top-1 accuracy on UCF101 and HMDB51 respectively, outperforming InfoGCN by 3.9 and 5.3 percentage points. In few-shot scenarios with only 1% labels, HGCL improves upon training from scratch by 21.6 percentage points on UCF101 and 17.7 on HMDB51. Controlled degradation experiments further demonstrate that the advantage of HGCL widens under increasing noise and joint occlusion, confirming the robustness conferred by graph-aware pre-training and confidence weighting. Supplementary Information Supplementary Information. Author contributions Y.L. conceived the experiment(s), Y.L. and X.Z. conducted the experiment(s), Y.L. and X.Z. analysed the results. All authors reviewed the manuscript. Funding This research did not receive funding. Data availability The data that support the findings of this study are openly available. The UCF101 dataset used in this study can be accessed at https://www.crcv.ucf.edu/data/UCF101.php. Declarations Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Supplementary Information The online version contains supplementary material available at 10.1038/s41598-026-56157-6. References Associated Data Supplementary Materials Supplementary Information. Data Availability Statement The data that support the findings of this study are openly available. The UCF101 dataset used in this study can be accessed at https://www.crcv.ucf.edu/data/UCF101.php.