Spatiotemporal Facial Features Encoding for Facial Expression Analysis in Image Sequences

Neurophysiology researchers concern on investigating the information processing that takes place inside the human cortex. Lately, the computer scientists try to simulate and build biological plausible systems for analyzing and encoding the spatiotemporal information in a similar way the biological brain cells do, incorporating the same biological constraints. In this paper we propose a method for extracting and encoding spatiotemporal information from face image sequences, representing various subjects expressing six basic emotions. Spatiotemporal features are first extracted from original patterns using a non-negative matrix decomposition and the resulting features are next converted into temporal pattern spikes which feed a leaky integrate-and-fire neuron with a dynamic synapse. The general framework aims at discriminating among the expressions through the timing of the output spike trains that form expression time clusters.