The Evaluation of Singing Voice Accuracy: A Comparison Between Subjective and Objective Methods

Objective. Vocal accuracy of a sung performance can be evaluated by two methods: acoustic analyses and subjective judgments. Acoustic analyses have been presented as a more reliable solution but both methods are still used for the evaluation of singing voice accuracy. This article presents a first time direct comparison of these methods. Methods. One hundred sixty-six untrained singers were asked to sing the popular song "Happy Birthday.'' These recordings constituted the database analyzed. Acoustic analyses were performed to quantify the pitch interval deviation, number of contour errors, and number of tonality modulations for each recording. Additionally, 18 experts in singing voice or music rated the global pitch accuracy of these performances. Results. A high correlation occurred between acoustic measurements and subjective rating. The total model of acoustic analyses explained 81% of the variance of the judges' scores. Their rating was influenced by both tonality modulations and pitch interval deviation. Conclusions. This study highlights the congruence between objective and subjective measurements of vocal accuracy within this first time comparison. Our results confirm the relevance of the pitch interval deviation criterion in vocal accuracy assessment. Furthermore, the number of tonality modulations is also a salient criterion in perceptive rating and should be taken into account in studies using acoustic analyses.