Pose estimation at night in infrared images using a lightweight multi-stage attention network

被引:11
|
作者
Zang, Ying [1 ,2 ,3 ]
Fan, Chunpeng [5 ]
Zheng, Zeyu [1 ,4 ,5 ]
Yang, Dongsheng [1 ,2 ]
机构
[1] Univ Chinese Acad Sci, Beijing 100049, Peoples R China
[2] Chinese Acad Sci, Shenyang Inst Comp Technol, Shenyang 110168, Peoples R China
[3] Huzhou Univ, Sch Informat Engn, Huzhou 313000, Peoples R China
[4] Chinese Acad Sci, Shenyang Inst Automat, Shenyang 110168, Peoples R China
[5] Hangzhou PingxingShijie Co Ltd, Hangzhou 311203, Peoples R China
关键词
Pose estimation; Far-infrared image; LMANet; Spatial attention; Channel attention;
D O I
10.1007/s11760-021-01916-3
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Human Keypoints Detection is a relatively basic task in computer vision; it is the pre-task of human action recognition, behavior analysis and human-computer interaction. Since most abnormal actions occur at night, how to effectively extract skeleton sequence data in a low-light or completely dark environment poses a huge challenge for its identification. This paper proposes to use far infrared images to detection key points of the human body, which can solve the problem of human pose estimation under challenging weather conditions such as total darkness, smoke, inclement weather and glare. However, far-infrared images have some shortcomings, such as low resolution, noise and thermal characteristics; the skeleton data need to be provided in real time for the next stage of task. Based on the above reasons, this paper proposes a lightweight multi-stage attention network (LMANet) to detect the key points of human at night. This new network structure adds context information through the large receptive field, which helps to assist the detection of neighboring key points through this information, but for the sake of lightweight consideration, this article only extends the network to two stages. In addition, this article uses the attention module to effectively select channels with a large amount of information and highlight the features of key points, while eliminating background interference. In order to detect key points of the human in various complex environments, we use techniques such as difficult sample mining which improves the accuracy of key points with low confidence. Our network has been verified on two visible light datasets, fully demonstrating excellent performance. This paper successfully introduces far-infrared images into the field of pose estimation, because there is no public dataset for far-infrared pose estimation. In this paper, 700 images are selected for annotation from multiple public far-infrared object detection, segmentation and action recognition datasets; our algorithm is verified on this dataset; the effect is very good. After the paper is published, we will publish our key points of the human body annotated documents.
引用
收藏
页码:1757 / 1765
页数:9
相关论文
共 50 条
  • [1] Pose estimation at night in infrared images using a lightweight multi-stage attention network
    Ying Zang
    Chunpeng Fan
    Zeyu Zheng
    Dongsheng Yang
    [J]. Signal, Image and Video Processing, 2021, 15 : 1757 - 1765
  • [2] Reconsideration of multi-stage deep network for human pose estimation
    Verma, Pratishtha
    Srivastava, Rajeev
    [J]. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING-IMAGING AND VISUALIZATION, 2021, 9 (06): : 600 - 612
  • [3] SESPnet: a lightweight network with attention mechanism for spacecraft pose estimation
    Chen C.
    Jing Z.
    Pan H.
    Dun X.
    Huang J.
    Wu H.
    Cao S.
    [J]. Aerospace Systems, 2024, 7 (01) : 1 - 10
  • [4] A Multi-Stage Visible and Infrared Image Fusion Network Based on Attention Mechanism
    Zheng, Xin
    Yang, Qiyong
    Si, Pengbo
    Wu, Qiang
    [J]. SENSORS, 2022, 22 (10)
  • [5] ACCURATE HEAD POSE ESTIMATION BASED ON MULTI-STAGE REGRESSION
    Liu, Yinchuan
    Gong, Yufei
    Lu, Zheng
    Zhang, Xuetao
    [J]. 2022 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, ICIP, 2022, : 1326 - 1330
  • [6] Multi-Stage Classification Network for Automatic Age Estimation from Facial Images
    Quinn, Louis
    Lech, Margaret
    [J]. 2015 9TH INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING AND COMMUNICATION SYSTEMS (ICSPCS), 2015,
  • [7] A Lightweight Network for Human Pose Estimation Based on ECA Attention Mechanism
    Ji, Xu
    Niu, Yanmin
    [J]. ELECTRONICS, 2024, 13 (01)
  • [8] Lightweight Multi-Resolution Network for Human Pose Estimation
    Li, Pengxin
    Wang, Rong
    Zhang, Wenjing
    Liu, Yinuo
    Xu, Chenyue
    [J]. CMES-COMPUTER MODELING IN ENGINEERING & SCIENCES, 2024, 138 (03): : 2239 - 2255
  • [9] Multi-stage strength estimation network with cross attention for single channel speech enhancement
    Zhang, Zipeng
    Ding, Yuchen
    Chen, Wei
    Chen, Yutao
    Guo, Weiwei
    Liu, Houguang
    [J]. SIGNAL IMAGE AND VIDEO PROCESSING, 2024, 18 (10) : 6937 - 6948
  • [10] Multi-stage attention network for monaural speech enhancement
    Wang, Kunpeng
    Lu, Wenjing
    Liu, Peng
    Yao, Juan
    Li, Huafeng
    [J]. IET SIGNAL PROCESSING, 2023, 17 (03)