Robust Event Detection From Spoken Content In Consumer Domain Videos

In this paper, we propose an innovative integrated approach to leverage available spoken content while detecting events in consumer-generated multimedia data (i.e., You Tube videos). Spoken content in consumer videos exhibits several challenges. For example, unlike Broadcast News, the spoken audio is typically not labeled. Also, the audio track in consumer videos tends to be noisy and the spoken content is often sporadic. Here, we describe three recent improvements that are specifically targeted at overcoming the challenges in consumer videos: robust data-driven keyword selection, automatic discovery of word-classes for keyword expansion, and a keyword spotting approach for improving recall in noisy conditions. These improvements are integrated into the audio analysis component of the BBN VISER system. The VISER system embodies a state-of-the-art approach as substantiated by its performance on the 2011 TRECVID MED task. Experimental results on the 2011 TRECVID MED task clearly demonstrate the effectiveness of the three improvements.