Deep Learning Based Hand Gesture Recognition and UAV Flight Controls

被引:0
|
作者
Bin Hu
Jiacun Wang
机构
[1] Monmouth University,
关键词
Deep learning; neural networks; hand gesture recognition; Leap Motion Controllers; drones;
D O I
暂无
中图分类号
学科分类号
摘要
Dynamic hand gesture recognition is a desired alternative means for human-computer interactions. This paper presents a hand gesture recognition system that is designed for the control of flights of unmanned aerial vehicles (UAV). A data representation model that represents a dynamic gesture sequence by converting the 4-D spatiotemporal data to 2-D matrix and a 1-D array is introduced. To train the system to recognize designed gestures, skeleton data collected from a Leap Motion Controller are converted to two different data models. As many as 9 124 samples of the training dataset, 1 938 samples of the testing dataset are created to train and test the proposed three deep learning neural networks, which are a 2-layer fully connected neural network, a 5-layer fully connected neural network and an 8-layer convolutional neural network. The static testing results show that the 2-layer fully connected neural network achieves an average accuracy of 96.7% on scaled datasets and 12.3% on non-scaled datasets. The 5-layer fully connected neural network achieves an average accuracy of 98.0% on scaled datasets and 89.1% on non-scaled datasets. The 8-layer convolutional neural network achieves an average accuracy of 89.6% on scaled datasets and 96.9% on non-scaled datasets. Testing on a drone-kit simulator and a real drone shows that this system is feasible for drone flight controls.
引用
收藏
页码:17 / 29
页数:12
相关论文
共 50 条
  • [1] Deep Learning Based Hand Gesture Recognition and UAV Flight Controls
    Hu, Bin
    Wang, Jiacun
    [J]. INTERNATIONAL JOURNAL OF AUTOMATION AND COMPUTING, 2020, 17 (01) : 17 - 29
  • [2] Deep Learning Based Hand Gesture Recognition and UAV Flight Controls
    Bin Hu
    Jiacun Wang
    [J]. International Journal of Automation and Computing, 2020, 17 (01) : 17 - 29
  • [3] Deep Learning Based Hand Gesture Recognition and UAV Flight Controls
    Hu, Bin
    Wang, Jiacun
    [J]. 2018 24TH IEEE INTERNATIONAL CONFERENCE ON AUTOMATION AND COMPUTING (ICAC' 18), 2018, : 20 - 25
  • [4] Research on the Hand Gesture Recognition Based on Deep Learning
    Sun, Jing-Hao
    Ji, Ting-Ting
    Zhang, Shu-Bin
    Yang, Jia-Kui
    Ji, Guang-Rong
    [J]. 2018 12TH INTERNATIONAL SYMPOSIUM ON ANTENNAS, PROPAGATION AND ELECTROMAGNETIC THEORY (ISAPE), 2018,
  • [5] Deep learning based hand gesture recognition in complex scenes
    Ni, Zihan
    Sang, Nong
    Tan, Cheng
    [J]. MIPPR 2017: PATTERN RECOGNITION AND COMPUTER VISION, 2017, 10609
  • [6] EMG based Hand Gesture Recognition using Deep Learning
    Ozdemir, Mehmet Akif
    Kisa, Deniz Hande
    Guren, Onan
    Onan, Aytug
    Akan, Aydin
    [J]. 2020 MEDICAL TECHNOLOGIES CONGRESS (TIPTEKNO), 2020,
  • [7] Deep Learning with Attention on Hand Gesture Recognition Based on sEMG
    Zhao, Huanjun
    Zheng, Bin
    Wang, Le
    [J]. 2022 PROGNOSTICS AND HEALTH MANAGEMENT CONFERENCE, PHM-LONDON 2022, 2022, : 316 - 320
  • [8] Hand Gesture Recognition Using Deep Learning
    Hussain, Soeb
    Saxena, Rupal
    Han, Xie
    Khan, Jameel Ahmed
    Shin, Hyunchul
    [J]. PROCEEDINGS INTERNATIONAL SOC DESIGN CONFERENCE 2017 (ISOCC 2017), 2017, : 48 - 49
  • [9] Gesture Recognition for UAV-based Rescue Operation based on Deep Learning
    Liu, Chang
    Sziranyi, Tamas
    [J]. PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING AND VISION ENGINEERING (IMPROVE), 2021, : 180 - 187
  • [10] Real Time Hand Gesture Recognition Applied for Flight Simulator Controls
    Zhuang Qianzheng
    Li Xiaodong
    Ren Jie
    Qiao Yuanyuan
    [J]. 2021 IEEE 7TH INTERNATIONAL CONFERENCE ON VIRTUAL REALITY (ICVR 2021), 2021, : 407 - 411