ATOM: Self-supervised human action recognition using atomic motion representation learning

被引:3
|
作者
Degardin, Bruno [1 ,2 ,3 ]
Lopes, Vasco [1 ,3 ]
Proenca, Hugo [1 ,2 ]
机构
[1] Univ Beira Interior, Covilha, Portugal
[2] IT Inst Telecomunicacoes, Aveiro, Portugal
[3] DeepNeuronic, Covilha, Portugal
关键词
Atomic dynamics; Self-supervised learning; Graph convolutional networks; Human pose; Skeleton-based action recognition; Human behavior understanding;
D O I
10.1016/j.imavis.2023.104750
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Self-supervised learning (SSL) is a promising method for gaining perception and common sense from unlabelled data. Existing approaches to analyzing human body skeletons address the problem similar to SSL models for image and video understanding, but pixel data is far more challenging than coordinates. This paper presents ATOM, an SSL model designed for skeleton-based data analysis. Unlike video-based SSL approaches, ATOM leverages atomic movements within skeleton actions to achieve a more fine-grained representation. The pro-posed architecture predicts the action order at the frame level, leading to improved perceptions and represen-tations of each action. ATOM outperforms state-of-the-art approaches in two well-known datasets (NTU RGB + D and NTU-120 RGB + D), and its weight transferability enables performance improvements on supervised and semi-supervised tasks, up to 4.4% (3.3% p.p.) and 14.1% (6.3% p.p.), respectively, in Top-1 Accuracy.
引用
收藏
页数:9
相关论文
共 50 条
  • [1] Spatiotemporal consistency enhancement self-supervised representation learning for action recognition
    Bi, Shuai
    Hu, Zhengping
    Zhao, Mengyao
    Li, Shufang
    Sun, Zhe
    [J]. SIGNAL IMAGE AND VIDEO PROCESSING, 2023, 17 (04) : 1485 - 1492
  • [2] Spatiotemporal consistency enhancement self-supervised representation learning for action recognition
    Shuai Bi
    Zhengping Hu
    Mengyao Zhao
    Shufang Li
    Zhe Sun
    [J]. Signal, Image and Video Processing, 2023, 17 : 1485 - 1492
  • [3] Contrastive self-supervised representation learning without negative samples for multimodal human action recognition
    Yang, Huaigang
    Ren, Ziliang
    Yuan, Huaqiang
    Xu, Zhenyu
    Zhou, Jun
    [J]. FRONTIERS IN NEUROSCIENCE, 2023, 17
  • [4] Motion Guided Attention Learning for Self-Supervised 3D Human Action Recognition
    Yang, Yang
    Liu, Guangjun
    Gao, Xuehao
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2022, 32 (12) : 8623 - 8634
  • [5] Self-Supervised Motion Perception for Spatiotemporal Representation Learning
    Liu, Chang
    Yao, Yuan
    Luo, Dezhao
    Zhou, Yu
    Ye, Qixiang
    [J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2023, 34 (12) : 9832 - 9846
  • [6] Self-supervised representation learning using multimodal Transformer for emotion recognition
    Goetz, Theresa
    Arora, Pulkit
    Erick, F. X.
    Holzer, Nina
    Sawant, Shrutika
    [J]. PROCEEDINGS OF THE 8TH INTERNATIONAL WORKSHOP ON SENSOR-BASED ACTIVITY RECOGNITION AND ARTIFICIAL INTELLIGENCE, IWOAR 2023, 2023,
  • [7] Video Motion Perception for Self-supervised Representation Learning
    Li, Wei
    Luo, Dezhao
    Fang, Bo
    Li, Xiaoni
    Zhou, Yu
    Wang, Weiping
    [J]. ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2022, PT IV, 2022, 13532 : 508 - 520
  • [8] Self-supervised representation learning for surgical activity recognition
    Paysan, Daniel
    Haug, Luis
    Bajka, Michael
    Oelhafen, Markus
    Buhmann, Joachim M.
    [J]. INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY, 2021, 16 (11) : 2037 - 2044
  • [9] Self-supervised Representation Learning for Fine Grained Human Hand Action Recognition in Industrial Assembly Lines
    Sturm, Fabian
    Sathiyababu, Rahul
    Allipilli, Harshitha
    Hergenroether, Elke
    Siegel, Melanie
    [J]. ADVANCES IN VISUAL COMPUTING, ISVC 2023, PT I, 2023, 14361 : 172 - 184
  • [10] Self-Supervised ECG Representation Learning for Emotion Recognition
    Sarkar, Pritam
    Etemad, Ali
    [J]. IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2022, 13 (03) : 1541 - 1554