A multi-stage deep adversarial network for video summarization with knowledge distillation

Video summarization is defined as the process of automatically identifying and extracting the relevant contents from a video that can best represent the contents of the video. The proposed model implements a video summarization framework based on generative adversarial network (GAN) for feature extraction and knowledge distillation for key frame or segment selection. The ideal characteristics of a video summary is diversity and representativeness. The primary stage of the proposed model based on adversarial learning ensures that the extracted features contain diverse and representative elements from the video. The generator is a convolutional recurrent autoencoder that learns the hidden representation of the video through the reconstruction loss. The generator model is followed by a discriminator that aims at improving the efficiency of the generator model by trying to discriminate between the original and reconstructed video samples. The adversarial network is followed by a knowledge distillation phase which acts as a key frame or segment selector by employing a simple network whose input data is retrieved from the preceding GAN model. Comprehensive evaluations conducted on public and custom datasets substantiate the relevance of GANs and knowledge distillation phase for video summarization. Quantitative and qualitative evaluations further prove that the proposed model produces remarkable results with summaries that are diverse, representative and concise.