Asymmetry-aware bilinear pooling in multi-modal data for head pose estimation

The head pose on roll and yaw directions is decided by the asymmetric appearance in human faces, and the contextual information of asymmetric appearance is encoded in a head pose related neighborhood. However, CNNs used in existing head pose estimation methods often evenly performs on the features of full image. Thus it is hard to collect the contextual information of such asymmetric appearance by those methods. To address this issue, this paper proposes a novel head pose estimation method that could perceive the asymmetric appearance in human faces. Specifically, the awareness of such asymmetry is undertaken by the local pairwise feature interaction in head pose related neighborhood via bilinear pooling. Evaluations on two public datasets demonstrate that our method could achieve promising results.