Video analytics-based intelligent surveillance system for smart buildings

The goal of the work is to automate video surveillance. The work holds its importance since the camera surveillance under manual supervision fails occasionally. Face images of authorized users in the building are trained, and for each face image, weight is calculated. When the test face comes into the building, the weight for the test face is calculated and compared with the existing weights. Based on the similarity between them, the person’s face is identified. For successful recognition of face across the frames, first face image has to be detected. To handle this task, we propose a hybrid algorithm using Haar cascade classifier and skin detector. The stand-alone performance of Haar cascade classifier and skin detector is analyzed, and the work discusses the need for hybridization. The proposed approach addresses the challenges in detecting faces such as orientation changes, varying illumination and partial occlusion. The performance of the system is comparatively analyzed with the videos of frontal and pose-varying faces, and the detection of face across frames is measured. From the experimental analysis, we infer that the detection rate of the proposed hybrid algorithm is 100% for frontal face video and 99.895% for pose-varying face video. The proposed hybrid algorithm is tested with VISOR dataset, and the proposed algorithm achieves precision of 95.20%. We also propose a deep learning framework based on the hybrid algorithm to detect face across the frames. The proposed framework generates more images by affine wrapping strategy and thus handling face orientation changes. The test data are labeled based on counting the prediction results of all the affine transformed images. To evaluate the performance of the system, the framework is tested with frames from VISOR dataset. From the experimental analysis, we infer that the precision of the proposed deep learning framework is 99.10%. Since the person’s face has been detected across the maximum number of frames, the next focus of the work is to identify the person whose face has been detected. For addressing the person identification problem, we use principal component analysis to reduce the dimension of the face vectors. The weight matrix or confidence for every face vector is calculated and checked for similarity with the weight matrix of test face vector. Based on the similarity between the two weight matrixes, a person’s face is identified. The built automated surveillance system is tested with NRC-IIT facial video database, from the experimental analysis, we infer that the system detects face across all the frames and out of ten videos, the system correctly identifies the face of the person in nine videos so that we calculate the recognition rate as 90%.