A Deep Neural Network based Multimodal Video Recognition System for Caring

Home caring usually refers to taking care of the elder, the young kids and the patients at home, and this depends on both caregivers and caring systems. The system which can provide sufficient and accurate information to doctors or caregivers without a delay will bring benefits for all the persons who need to be cared and allow doctor or care givers to take right action based on the information immediately. Available corresponding products in the market mainly are some smart home devices or some medical facilities based on electroencephalo-graph (EEG), electrocardiograph(ECG) or blood pressure check, however, these parameters cannot give doctors, caregivers or family members a direct feedback. To address the problem, this paper introduces a deep learning based design - a visual recognition system developed for clinical monitoring which can supervise both the emotions and gestures of patients at the same time and give responsible persons instant and direct feedback so that the right treatment will be taken by them. This product uses a Raspberry Pi computer with its camera as hardware and implementing several deep learning models to fulfill three main functions which are: Facial Recognition, Emotion Detection and Pose Estimation onto a portable device, which enhances the utilization of theapplication and this brings more possibilities. Compared to theavailable products, this application emphasizes monitoring per-sons via visual analysis and gives more direct feedback all thetime instead of traditional ways which are not always timely orpractical.