A multi-modal approach to detect inappropriate cartoon video contents using deep learning networks

Children are more than ever exposed to all kinds of video contents on the Internet. Consequently, several studies have proposed different techniques to detect videos that can be harmful for children. However, we note so far, that no attention has been given to the cartoon characters and the underlying language used. To address this gap, we propose to evaluate the effectiveness of using actual images of cartoon characters and the language used in cartoons in categorising videos as being appropriate or inappropriate for children. We do so through the development of a multi modal classifier, which makes use of the output from two deep learning networks: LSTM for text analysis and VGGNet for image analysis. More specifically, the LSTM network is used to process user comments and closed captions associated with a video and the VGGNet network is used to recognize cartoon image characters. The LSTM model was trained and tested on a dataset comprising about 290,000 labelled text records, while the VGGNet model was trained and tested on a manually annotated image dataset of 6000 cartoon characters. A testing accuracy of 94% was obtained for the LSTM network while a testing accuracy of 99% was obtained for the VGGNet network. Our proposed approach was further evaluated using 50 actual videos intended for children from YouTube. Here also, a good accuracy of 72% was obtained using LSTM alone, while a better accuracy of 78% was obtained using VGGNet alone and an accuracy of 76% was obtained using the combined output from the LSTM and VGGNet networks. We conclude that closed captions, user comments and images of cartoon characters are all useful in detecting unsafe videos for children and can be considered as essential parameters to include when developing multimedia filtering tools.