Social profiling through image understanding: Personality inference using convolutional neural networks

The role of images in the last ten years has changed radically due to the advent of social networks: from media objects mainly used to communicate visual information, images have become personal, associated with the people that create or interact with them (for example, giving a "like"). Therefore, in the same way that a post reveals something of its author, so now the images associated to a person may embed some of her individual characteristics, such as her personality traits. In this paper, we explore this new level of image understanding with the ultimate goal of relating a set of image preferences to personality traits by using a deep learning framework. In particular, our problem focuses on inferring both self-assessed (how the personality traits of a person can be guessed from her preferred image) and attributed traits (what impressions in terms of personality traits these images trigger in unacquainted people), learning a sort of wisdom of the crowds. Our characterization of each image is locked within the layers of a CNN, allowing us to discover more entangled attributes (aesthetic patterns and semantic information) and to better generalize the patterns that identify a trait. The experimental results show that the proposed method outperforms state-of-the-art results and captures what visually characterizes a certain trait: using a deconvolution strategy we found a clear distinction of features, patterns and content between low and high values in a given trait. (C) 2016 Elsevier Inc. All rights reserved.