Facial expression recognition based on spatio-temporal interest points for depth sequences

Of the numerous facial expression recognition methods previously proposed, most are based on texture frames or sequences. Recently, the development of depth sensors has raised new possibilities of dealing with 3D data. The proposed method extracts histograms of oriented gradient (HOG) and optical flow (HOF) for STIPs directly from depth sequences rather than involving registration/deformation techniques to find correspondence in 3D scan data. Mutual information score (MIS) and weighted matching score (WMS) are, respectively, calculated on the basis of naive-Bayes mutual information maximization and constrained matching pairs. Finally, the MIS and WMS results are concatenated into feature vectors which are then fed into a support vector machine for facial expression classification. The proposed method is applied to the public BU-4DFE (Binghamton University 4D Facial Expression) database for six different facial expressions: anger, disgust, fear, happy, sad and surprise. Experimental results confirm that the proposed method is simple but effective.