Human Activity Recognition Using UWB Radar and Cameras on a Mobile Robot

To realize accurate recognition of human activities, many studies have been proposed using vision sensors. Vision sensors can reliably detect human in normal environment, but it is affected by sudden illumination changes and complex conditions, which are the major obstacles to the reliability and robustness of the system. To solve this problem, we proposed a novel integration method to combine bi-static UWB radar and two cameras to realize the recognition of human activities. In this recognition system, cameras are used to localize people's regions while radar is used to obtain their 3D activity models, which can be matched in our 3D activity library in order to recognize their activities. To confirm the effectiveness of the proposed method, we compared the experimental results of recognition using vision sensors and those of recognition using the integration method in different environments. Significant improvement of the correct recognition rate is achieved in the experiment.