Real-time detection of abnormal human activity using deep learning and temporal attention mechanism in video surveillance

In the modern era of technology, monitoring and controlling abnormal human activity is essentially required as these activities may harm society through physical harm to a human being, or by spreading hate crimes on the World Wide Web. Although many authors have contributed to address this problem, a desired solution that may work in a real-time scenario has yet to be achieved. Recently, deep learning models have gained attraction as processing power for a large volume of data. However, there is little work based on deep learning models for detecting abnormal human activity classification that has been done till now. In the proposed framework, a deep-learning method has been used to detect abnormal human activity by combining a convolutional neural network (CNN), a Recurrent Neural Network (RNN), and an attention module for attending the specific spatiotemporal characteristics from unprocessed video streams. This proposed architecture can accurately classify an aberrant human activity with its special category after processing the video. The proposed architecture's analytical results show an accuracy of 96.94%, 98.95%, and 62.04% with UCF50, UCF110, and UCF crime datasets, which is compared with the results of state-of-the-art algorithms (SOTA).