Personalized Video Summarization based on Behavior of Viewer

Recent progress of video recording hardware such as HDD video recorders or PCs with TV tuner enabled us to store hundreds or even thousands of hours of programs, and view any one of them on demand. In accordance with the growth of the capacity of video storage, efficient scheme of video access is studied; shot or scene boundary detection, content visualization for video browsing, content-based access based on annotation, content-based summarization, etc. Video summarization is one of the promising approaches for effective comprehension of video contents, which is obtained by detecting informative segments of video data and concatenate them to show as one consecutive video. Previous studies on video summarization are focused on detecting informative segments which correspond to attractive and/or impressive scenes based on textual annotation and audio-visual cues which are correlated to important/informative events. This method works well if we assume that the viewers of the summarized contents have the same purpose in summary viewing and the sense of 'importance' is the same regardless of viewers. However, this approach cannot cope with a case where criteria on importance differ depending on viewer. We propose a novel framework of video summarization based on the detection of viewer behavior during watching video contents as a solution for this issue. It captures eye movement and operation of remote controller of video player as the behavior of a viewer while watching a video program. The degree of importance is evaluated based on his/her behavior and video summarization is carried out so that it reflects diversity of viewers preference or interest.