Digital recognition from lip texture analysis

Digital recognition with lip images has become a key step of the interactive liveness detection for Chinese banking systems. However, the problem of the digital recognition is very challenging due to intra class variation of lip images, head pose variations, and uncontrolled illumination. This paper studies a deep learning architecture to model the appearance and the spatial-temporal information of lip texture. The lip texture in still image frames and the spatial-temporal relationship between these frames are jointly modeled by convolutional neural networks and long short-term memory. Two strategies are further exploited to find effective groups of ten digitals for training the deep models. As a result, more information can be utilized for accurate recognition based on lip texture analysis. Besides, two datasets of isolated digits in Chinese are established to simulate real-world liveness detection environments together with various attacks. Extensive experiments have been done to analyze the recognition accuracy of each digit and to provide some clues for determining appropriate digits for interactive liveness detection.