The Correspondence of Music Emotion and Timbre in Sustained Musical Instrument Sounds

Music is one of the strongest triggers of emotions. Melody, rhythm, and harmony are important triggers, but what about timbre? Do musical instruments have an emotional predisposition? For example, is the melancholy sound of the English horn due to its timbre or how it is used? Though music emotion recognition has received a lot of attention; researchers have only recently begun considering the relationship between emotion and timbre. Therefore, we designed listening tests to compare sounds from eight wind and bowed string instruments. We wanted to know if some sounds were consistently perceived as being happier or sadder in pairwise comparisons. Eight emotions were tested. The results showed strong emotional predispositions for each instrument. The emotions Happy, Joyful, Heroic, and Comic were strongly correlated with one another, and the violin, trumpet, and clarinet best evoked these emotions. Sad and Depressed were also strongly correlated and were best evoked by the horn and flute. Scary was an emotional outlier, and the oboe had an emotionally neutral disposition. We also correlated emotion and several spectral features and found that emotion correlated significantly with average spectral centroid and spectral centroid deviation for nearly all emotions. To determine which spectral features were most important aside from spectral centroid, we conducted follow-up listening tests of centroid-equalized sounds as well as static sounds. The results showed that the even/odd harmonic ratio significantly correlated with most emotions. This suggests that the even/odd harmonic ratio is perhaps the most salient timbral feature after attack time and brightness. These results provide new insights for orchestration.