Preprocessing for Keypoint-Based Sign Language Translation without Glosses

被引:5
|
作者
Kim, Youngmin [1 ]
Baek, Hyeongboo [1 ]
机构
[1] Incheon Natl Univ INU, Dept Comp Sci & Engn, Incheon 22012, South Korea
关键词
computer vision; deep learning; sign language translation; video processing;
D O I
10.3390/s23063231
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
While machine translation for spoken language has advanced significantly, research on sign language translation (SLT) for deaf individuals remains limited. Obtaining annotations, such as gloss, can be expensive and time-consuming. To address these challenges, we propose a new sign language video-processing method for SLT without gloss annotations. Our approach leverages the signer's skeleton points to identify their movements and help build a robust model resilient to background noise. We also introduce a keypoint normalization process that preserves the signer's movements while accounting for variations in body length. Furthermore, we propose a stochastic frame selection technique to prioritize frames to minimize video information loss. Based on the attention-based model, our approach demonstrates effectiveness through quantitative experiments on various metrics using German and Korean sign language datasets without glosses.
引用
收藏
页数:16
相关论文
共 50 条
  • [41] Sign Language Translation
    Harini, R.
    Janani, R.
    Keerthana, S.
    Madhubala, S.
    Venkatasubramanian, S.
    2020 6TH INTERNATIONAL CONFERENCE ON ADVANCED COMPUTING AND COMMUNICATION SYSTEMS (ICACCS), 2020, : 883 - 886
  • [42] Spoken Language to Sign Language Translation System Based on HamNoSys
    Kang, Zhehan
    2019 INTERNATIONAL SYMPOSIUM ON SIGNAL PROCESSING SYSTEMS (SPSS 2019), 2019, : 159 - 164
  • [43] Study on Hand Keypoint Framework for Sign Language Recognition
    Amaliya, Sholikhatul
    Handayani, Anik Nur
    Akbar, Muhammad Iqbal
    Herwanto, Heru Wahyu
    Fukuda, Osamu
    Kurniawan, Wendy Cahya
    2021 7TH INTERNATIONAL CONFERENCE ON ELECTRICAL, ELECTRONICS AND INFORMATION ENGINEERING (ICEEIE 2021), 2021, : 446 - 451
  • [44] Adaptive Keypoint-based CU Depth Decision for HEVC Intra Coding
    Kim, Namuk
    Jeon, Seungsu
    Shim, Hiuk Jae
    Jeon, Byeungwoo
    Lim, Sung-Chang
    Ko, Hyunsuk
    2016 IEEE INTERNATIONAL SYMPOSIUM ON BROADBAND MULTIMEDIA SYSTEMS AND BROADCASTING (BMSB), 2016,
  • [45] Image-Goal Navigation via Keypoint-Based Reinforcement Learning
    Choi, Yunho
    Oh, Songhwai
    2021 18TH INTERNATIONAL CONFERENCE ON UBIQUITOUS ROBOTS (UR), 2021, : 18 - 21
  • [46] Keypoint-Based Automated Component Placement Inspection for Printed Circuit Boards
    Chung, Si-Tung
    Hwang, Wen-Jyi
    Tai, Tsung-Ming
    Oh, Ki-Yong
    APPLIED SCIENCES-BASEL, 2023, 13 (17):
  • [47] MACHINE TRANSLATION TO SIGN LANGUAGE USING POST-TRANSLATION REPLACEMENT WITHOUT PLACEHOLDERS
    Miyazaki, Taro
    Nakatani, Naoki
    Uchida, Tsubasa
    Kaneko, Hiroyuki
    Sano, Masanori
    2023 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING WORKSHOPS, ICASSPW, 2023,
  • [48] Characterizing powder materials using keypoint-based computer vision methods
    DeCost, Brian L.
    Holm, Elizabeth A.
    COMPUTATIONAL MATERIALS SCIENCE, 2017, 126 : 438 - 445
  • [49] Efficient Online Structured Output Learning for Keypoint-Based Object Tracking
    Hare, Sam
    Saffari, Amir
    Torr, Philip H. S.
    2012 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2012, : 1894 - 1901
  • [50] A keypoint-based object detection method with attention mechanism and feature fusion
    Wang, Hui
    Yang, Tangwen
    2020 CHINESE AUTOMATION CONGRESS (CAC 2020), 2020, : 2113 - 2118