Fast and robust short video clip search for copy detection

Query by video clip (QVC) has attracted wide research interests in multimedia information retrieval. In general, QVC may include feature extraction, similarity measure, database organization, and search or query scheme. Towards an effective and efficient solution, diverse applications have different considerations and challenges on the abovementioned phases. In this paper, we firstly attempt to broadly categorize most existing QVC work into 3 levels: concept based video retrieval, video title identification, and video copy detection. This 3-level categorization is expected to explicitly identify typical applications, robust requirements, likely features, and main challenges existing between mature techniques and hard performance requirements. A brief survey is presented to concretize the QVC categorization. Under this categorization, in this paper we focus on the copy detection task, wherein the challenges are mainly due to the design of compact and robust low level features (i.e. an effective signature) and a kind of fast searching mechanism. In order to effectively and robustly characterize the video segments of variable lengths, we design a novel global visual feature (a fixed-size 144-d signature) combining the spatial-temporal and the color range information. Different from previous key frame based shot representation, the ambiguity of key frame selection and the difficulty of detecting gradual shot transition could be avoided. Experiments have shown the signature is also insensitive to color shifting and variations from video compression. As our feature can be extracted directly from MPEG compressed domain, lower computational cost is required. In terms of fast searching, we employ the active search algorithm. Combining the proposed signature and the active search, we have achieved an efficient and robust solution for video copy detection. For example, we can search for a short video clip among the 10.5 hours MPEG-I video database in merely 2 seconds in the case of unknown query length, and in 0.011 second when fixing the query length as 10 seconds.