Estimating body animation parameters from depth images using analysis by synthesis

We present a general method to estimate MPEG-4 Body Animation Parameters (BAPs) from depth images by using an analysis-by-synthesis technique. A generic body model is first adapted to the geometry of the specific person. Then, the mean square error between the synthetic depth image, produced by model rendering, and the original depth image is minimized using the Downhill Simplex minimization method. Using this depth reconstruction error norm is seen to yield improved results, when compared to two alternative choices for the error norms, which were also evaluated. Results are presented for the specific application, where six animation parameters of the arm and the palm are estimated. In this case, an initial estimate is obtained by applying an Expectation-Maximization (EM) algorithm, which identifies three arm parts and two joint positions (elbow, wrist). This information is also used for reducing the search space and for automatic scale adaptation of each body part.