Hierarchical memory representation of verbal and nonverbal features for emotion

Emotions are thought to be stored in long-term memory based on features as a memory unit of objects. Previous studies suggest that verbal features hierarchically represent emotions, such as sadness. Although emotion is associated with verbal and nonverbal features, it is not clear how these features interrelate in the memory representation of emotion. We hypothesized that some nonverbal features hierarchically function as “superordinate” or “subordinate” in sensory-perceptual dimensions representing emotions corresponding with verbal features. We focused on sadness as a target emotion and conducted a picture-word congruency judgment experiment. This study used 16 bodily-expression photographs (four-level trunk inclinations × four-level hand-arm postures) as sensory-perceptual features and 32 crying-related words as verbal features that prototypically represent sadness. Forty-eight participants assessed congruency concerning all combinations between bodily-expression pictures and crying-related words. Using a network graph analysis, bodily expressions were classified into three communities based on congruence ratings: the dominant bodily-expression community showed the highest verbal congruency composed of around-eye and overall-face hand-arm postures, irrespective of trunk inclination. Within this dominant community, trunk inclination subordinately modulated picture-word congruency according to bodily-activation properties. These findings support a feature-based representation of emotion through corresponding hierarchical structures between verbal and nonverbal features.