A GCN and Transformer complementary network for skeleton-based action recognition

被引:0
|
作者
Xiang, Xuezhi [1 ,2 ]
Li, Xiaoheng [1 ]
Liu, Xuzhao [1 ]
Qiao, Yulong [1 ,2 ]
El Saddik, Abdulmotaleb [3 ]
机构
[1] School of Information and Communication Engineering, Harbin Engineering University, Harbin,150001, China
[2] Key Laboratory of Advanced Marine Communication and Information Technology, Ministry of Industry and Information Technology, Harbin,150001, China
[3] School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa,ON K1N 6N5, Canada
基金
中国国家自然科学基金;
关键词
Joints; (anatomy);
D O I
10.1016/j.cviu.2024.104213
中图分类号
学科分类号
摘要
Graph Convolution Networks (GCNs) have been widely used in skeleton-based action recognition. Although there are significant progress, the inherent limitation still lies in the restricted receptive field of GCN, hindering its ability to extract global dependencies effectively. And the joints that are structurally separated can also have strong correlation. Previous works rarely explore local and global correlations of joints, leading to insufficiently model the complex dynamics of skeleton sequences. To address this issue, we propose a GCN and Transformer complementary network (GTC-Net) that allows parallel communications between GCN and Transformer domains. Specifically, we introduce a graph convolution and self-attention combined module (GAM), which can effectively leverage the complementarity of GCN and self-attention to perceive local and global dependencies of joints for the human body. Furthermore, in order to address the problems of long-term sequence ordering and position detection, we design a position-aware module (PAM), which can explicitly capture the ordering information and unique identity information for body joints of skeleton sequence. Extensive experiments on NTU RGB+D 60 and NTU RGB+D 120 datasets are conducted to evaluate our proposed method. The results demonstrate that our method can achieve competitive results on both datasets. © 2024 Elsevier Inc.
引用
收藏
相关论文
共 50 条
  • [21] Fully Attentional Network for Skeleton-Based Action Recognition
    Liu, Caifeng
    Zhou, Hongcheng
    IEEE ACCESS, 2023, 11 : 20478 - 20485
  • [22] Skeleton-based Action Recognition with Graph Involution Network
    Tang, Zhihao
    Xia, Hailun
    Gao, Xinkai
    Gao, Feng
    Feng, Chunyan
    2022 26TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), 2022, : 3348 - 3354
  • [23] Convolutional relation network for skeleton-based action recognition
    Zhu, Jiagang
    Zou, Wei
    Zhu, Zheng
    Hu, Yiming
    NEUROCOMPUTING, 2019, 370 : 109 - 117
  • [24] Auto-Learning-GCN: An Ingenious Framework for Skeleton-Based Action Recognition
    Xin, Wentian
    Liu, Yi
    Liu, Ruyi
    Miao, Qiguang
    Shi, Cheng
    Pun, Chi-Man
    PATTERN RECOGNITION AND COMPUTER VISION, PRCV 2023, PT I, 2024, 14425 : 29 - 42
  • [25] A Spatiotemporal Fusion Network For Skeleton-Based Action Recognition
    Bao, Wenxia
    Wang, Junyi
    Yang, Xianjun
    Chen, Hemu
    2024 3RD INTERNATIONAL CONFERENCE ON IMAGE PROCESSING AND MEDIA COMPUTING, ICIPMC 2024, 2024, : 347 - 352
  • [26] Hypergraph Neural Network for Skeleton-Based Action Recognition
    Hao, Xiaoke
    Li, Jie
    Guo, Yingchun
    Jiang, Tao
    Yu, Ming
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 : 2263 - 2275
  • [27] Attentional weighting strategy-based dynamic GCN for skeleton-based action recognition
    Hu, Kai
    Jin, Junlan
    Shen, Chaowen
    Xia, Min
    Weng, Liguo
    MULTIMEDIA SYSTEMS, 2023, 29 (04) : 1941 - 1954
  • [28] Attentional weighting strategy-based dynamic GCN for skeleton-based action recognition
    Kai Hu
    Junlan Jin
    Chaowen Shen
    Min Xia
    Liguo Weng
    Multimedia Systems, 2023, 29 : 1941 - 1954
  • [29] Improved ELBO-assisted Transformer for Skeleton-Based Action Recognition
    Bhattacharjee, Arnab
    Chen, Wen-Hui
    Lin, Yu-Chen
    Lai, Kuan-Ting
    2023 IEEE 26TH INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS, ITSC, 2023, : 3997 - 4002
  • [30] Skeleton-based action recognition via spatial and temporal transformer networks
    Plizzari, Chiara
    Cannici, Marco
    Matteucci, Matteo
    COMPUTER VISION AND IMAGE UNDERSTANDING, 2021, 208 (208-209)