Unsupervised Audio Segmentation based on Restricted Boltzmann Machines

In this paper the Conditional Restricted Boltzmann Machine (CRBM) is employed in the context of unsupervised audio segmentation. The CRBM acts as a temporal modeling method and learns, from a maximum likelihood perspective, the temporal relationships of the feature vectors that have been extracted from a large corpus of training data. After the CRBM has been trained, we quantify the correlation of the activation of the neurons of the hidden layer for successive feature vectors by means of an appropriately defined similarity function. A simple thresholding scheme is then applied on the output of the similarity function to segment automatically the audio recording. Our experiments have been carried out on a large corpus of documentaries. We provide an interpretation of the segmentation results and comment on the segmentation efficiency of the method.