Object Categorization Using Co-occurrence and Spatial Relationship with Human Interaction

The human interaction based framework for manipulable object categorization is proposed in this paper. In the proposed framework, co-occurrence and spatial relationship based features are developed to improve the categorization problem of the objects with high intra-class variation, deformable objects or the objects that are occluded. The descriptor in this framework is based on a co-occurrence of objects and hand poses, a relative position between objects and face, an object motion, and an object appearance. For co-occurrence based features, hand pose prototypes are generated by using K-means clustering. The co-occurrence vectors between objects and hand poses are observed from image frames and used as features. For spatial relationship based features, the histogram of relative positions between object and face and histogram of object motion vectors are applied. The evaluation is performed on six classes of objects in 180 videos. The proposed framework can improve the recognition rate by 30.1% in comparison with the object appearance baseline.