Real-time human posture reconstruction in wireless smart camera networks

While providing a variety of intriguing application opportunities, a vision sensor network poses three key challenges. High computation capacity is required for early vision functions to enable real-time performance. Wireless links limit image transmission in the network due to both bandwidth and energy concerns. Last but not least, there is a lack of established vision-based fusion mechanisms when a network of cameras is available. In this paper a distributed vision processing implementation of human pose interpretation on a wireless smart camera network is presented. The motivation for employing distributed processing is to both achieve real-time vision and provide scalability for developing more complex vision algorithms. The distributed processing operation includes two levels. One is that each smart camera processes its local vision data, achieving spatial parallelism. The other is that different functionalities of the whole line of vision processing are assigned to early vision and object-level processors, achieving functional parallelism based on the processor capabilities. Aiming for low power consumption and high image processing performance, the wireless smart camera is based on an SIMD (single-instruction multiple-data) video analysis processor an 8051 micro-controller as the local host, and wireless communication through the IEEE 802.15.4 standard. The vision algorithm implements 3D human pose reconstruction. From the live image data from the sensor the smart camera acquires critical joints of the subject in the scene through local processing. The results obtained by multiple smart cameras are then transmitted through the wireless channel to a central PC where the 3D pose is recovered and demonstrated in a virtual reality gaming application. The system operates in real time with a 30 frames/sec rate.