GA-based speaking mouth correlative speech feature abstraction

The image-based lip animation synthesis approach is one kind of promising method that synthesizes the believable talking head. This paper seeks to show an improvement in the accuracy of mouth prediction with the speech stimulus, as well as showing the method used to extract the speaking mouth correlative speech feature. Our lip animation synthesis system is based on the construction of a frame level audiovisual mapping model between the acoustic speech class and speaking mouth image class. Taking the mapping model as a basis, genetic algorithm is used to extract the speaking mouth correlative speech feature. The key step used in this study is: fitness and coding scheme designing. Experimental results show that the extracted speech feature has a better correlation with the corresponding speaking mouth, compared to the single or mixed LPCC and MFCC. More research will be done in this specialist field of study the multi-layer speaking mouth correlative speech feature abstraction structure, and will attempt to show that the speaking mouth correlative speech feature should have better results.