30 January 2023 Feature difference and feature correlation learning mechanism for skeleton-based action recognition
Ruxin Qing, Min Jiang, Jun Kong
Author Affiliations +
Abstract

In recent years, skeleton-based action recognition has become increasingly popular in the field of human action recognition, and graph convolutional networks (GCNs) have shown better advantages in this task. Many GCN-based methods are insufficient in the latent relationship between features, which affects the discriminability of features being not rich enough. These potential feature relationships can manifest as feature differences that change due to actions and feature correlations that interact with each other. Therefore, we propose a feature difference and feature correlation learning mechanism to learn discriminative augmentation features, including feature differences in actions and feature correlations between joints. First, we propose a temporal feature difference and correlation learning module (FDCL) (TFDCL). In adjacent temporal frames, we extract feature correlations between related parts. Feature differences are captured through changes in joints over the overall long-term timeline. Second, we propose a channel FDCL module. Different channels contain different types of features for actions. We use convolution operations to interact between channels, continuously extracting the strongest features to obtain feature maps. Third, we propose a temporal channel context topology (TCCT) module to dynamically learn global contextual features of all joints during motion. Finally, experiments are conducted on the NTU-RGBD 60 dataset and the kinetics-skeleton 400 dataset to verify the effectiveness of the network.

© 2023 SPIE and IS&T
Ruxin Qing, Min Jiang, and Jun Kong "Feature difference and feature correlation learning mechanism for skeleton-based action recognition," Journal of Electronic Imaging 32(1), 013011 (30 January 2023). https://doi.org/10.1117/1.JEI.32.1.013011
Received: 3 August 2022; Accepted: 6 January 2023; Published: 30 January 2023
Lens.org Logo
CITATIONS
Cited by 1 scholarly publication.
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Action recognition

Feature extraction

Convolution

Education and training

Bone

Matrices

Data modeling

Back to Top