Reproducing Material Appearance of Real Objects using Mobile Augmented Reality

In this paper, we propose a system that can reproduce the material appearance of real objects using mobile augmented reality (AR). Our proposed system allows a user to manipulate a virtual object, whose model is generated from the shape and reflectance of a real object, using the user's own hand. The shape of the real object is reconstructed by integrating depth images of the object, which are captured using an RGB-D camera from different directions. The reflectance of the object is obtained by estimating the parameters of a reflectance model from the reconstructed shape and color images, assuming that a single light source is attached to the camera. We measured the shape and reflectance of some real objects and presented the material appearance of the objects using mobile AR. It was confirmed that users were able to obtain the perception of materials from changes in gloss and burnish of the objects by rotating the objects using their own hand.