Object Tracking with a Multiagent Robot System and a Stereo Vision Camera

When working with a robot in terms of object manipulation the essential information is relative position between robot's tool center point (TCP) and the object of interest. This paper proposes a method of frame relative displacement and describes a working multiagent robot application that can be used for tracking, tooling or handling operations with the use of stereo vision in unstructured laboratory environment. Robot system is composed of two Fanuc robot arms, one of which carries a stereo vision camera system and the other which is guided in relation to object of interest. The latter robot has a marker that is used for navigation between the robot and the object of interest. Image processing, marker detection, 3-D coordinates extraction, coordinate system transformations, offset coordinates calculation and communication are handled using c++ multithread program and TCP/IP protocol. (C) 2014 The Authors. Published by Elsevier Ltd.