An iteratively reweighting algorithm for dynamic video summarization

Information explosion has imposed unprecedented challenges on the conventional ways of video data consumption. Hence providing condensed and meaningful video summary to viewers has been recognized as a beneficial and attractive research in the multimedia community in recent years. Analyzing both the visual and textual modalities proves essential for an automatic video summarizer to pick up important contents from a video. However, most established studies in this direction either use heuristic rules or rely on simple ways of text analysis. This paper proposes an iteratively reweighting dynamic video summarization (IRDVS) algorithm based on the joint and adaptive use of the visual modality and accompanying subtitles. The proposed algorithm takes advantage of our developed SEmantic inDicator of videO seGment (SEDOG) feature for exploring the most representative concepts for describing the video. Meanwhile, the iteratively reweighting scheme effectively updates the dynamic surrogate of the original video by combining the high-level features in an adaptive manner. The proposed algorithm has been compared to four state-of-the-art video summarization approaches, namely the speech transcript-based (STVS) algorithm, attention model-based (AMVS) algorithm, sparse dictionary selection-based (DSVS) algorithm and heterogeneity image patch index-based (HIPVS) algorithm, on different video genres, including documentary, movie and TV news. Our results show that the proposed IRDVS algorithm can produce summarized videos with better quality.