Convolution Neural Network Based Visual Speech Recognition System for Syllable Identification

Introduction: This paper introduces a novel and reliable approach for people with speech impairment to assist them in communicating effectively in real-time. A deep learning technique named as convolution neural network is used as its classifier. With the help of this algorithm, words are recognized from an input which is visual speech, disregarding the audible or acoustic property. Methods: This network extracts the features from mouth movements and different images, respec-tively. With the help of a source, non-audible mouth movements are taken as an input and then seg-regated as subsets to get the desired output. The Complete Datum is then arranged to recognize the word as an affricate. Results: Convolution neural network is one of the most effective algorithms that extract features, perform classification and provides the desired output from the input images for the speech recognition system. Conclusion: Recognizing the syllables at real-time from visual mouth movement input is the main objective of the proposed method. When the proposed system was tested, datum accuracy and quan-tity of training sets proved to be satisfactory. A small set of datum is taken as the first step of learn-ing. In future, a large set of datum can be considered for analyzing the data. Discussion: On the basis of the type of datum, the network proposed in this paper is tested for its precision level. A network is maintained to identify the syllables, but it fails when syllables are of the same set. There is a requirement of a higher end graphics processing units to reduce the time con-sumption and increase the efficiency of a network. © 2022 Bentham Science Publishers.