VideoCube: A novel tool for video mining and classification

We propose a new tool to classify a video clip into one of n given classes (e.g., "news", "commercials", etc). The first novelty of our approach is a method to automatically derive a "vocabulary" from each class of video clips, using the powerful method of "Independent Component Analysis" (ICA). Second, the method is unified in that it works with both video and audio information, and gives vocabulary describing not only the still images, but also motion and the audio part. Furthermore, this vocabulary is natural in that it is closely related to human perceptual processing. More specifically, every class of video clips gives a list of "basis functions", which can compress its members very well. Once we represent video clips in "vocabularies", we can do classification and pattern discovery. For the classification of a video clip, we propose using compression: we test which of the "vocabularies" can compress the video clip best, and we assign it to the corresponding class. For data mining, we inspect the basis functions of each video genre class and identify genre characteristics such as fast motions/transitions, more harmonic audio, etc. In experiments on real data of 62 news and 43 commercial clips, our method achieved overall accuracy of approximate to 81%.