Time-Series Flexible Resampling for Continuous and Real-Time Finger Character Recognition

Sign language is one of the crucial methods of communication for deaf or hearing-impaired people. Finger characters are often used to supplement sign language in conversations, especially when it is difficult to express some words by sign language. However, few people understand sign language. The realization of an automatic translation system for sign language or finger characters will facilitate communication with deaf or hearing-impaired people. To develop a finger character recognition system that is practical in the daily lives for smooth conversations, continuous and real-time recognition is required. This paper presents a novel real-time data processing framework, the time-series flexible resampling. This framework consists of three steps: detection, segmentation, and extraction. This framework enables continuous and real-time recognition for acyclic time-series data, in which stable states and changing states occur repeatedly. In addition, continuous and real-time finger character recognition is realized by applying time-series flexible resampling. The effectiveness of applying time-series flexible resampling to the finger character recognition system was verified in subject experiments.