Demographic attribute estimation in face videos combining local information and quality assessment

Nowadays, video analysis applications are gaining popularity given the rise of CCTV systems and the availability of video cameras to the general public, such as cameras in mobile devices. Many image analysis and processing tasks have evolved toward video domain, with the advantage of redundant information obtained from several frames, which can help disambiguating many recognition outputs. In this context, there are also particular video problems to deal with, such as uncontrolled scenarios and poor image quality. Most existing works regarding facial demographic estimation are focused on still image datasets; therefore, we propose to address gender and age estimation in video scenarios. In order to handle known video problems such as low-quality image capture, occlusions and pose variations, we propose a threefold strategy to adapt current image-based attribute recognition algorithms. First, we employ a quality assessment step based on 12 metrics to select relevant good quality frames from a face video sequence. Second, we propose a component-based approach to determine the most discriminant local regions of the face for each specific attribute, under these varying conditions. Third, we evaluate different frame combination strategies to produce the final video prediction. In our experimental validation, conducted in 3 datasets (EURECOM Augmented, UvA-Nemo Smile and YouTube Faces datasets), we show the advantages of our proposed strategy for improving video-based demographic attribute classification.