A multi-camera vision system for real-time tracking of parcels moving on a conveyor belt

We consider the problem of designing a vision system for tracking parcels moving on a conveyor belt. After computing parcels dimensions, i.e., length, width and height, at the entrance of the conveyor belt using a stereo camera pair, the vision system incorporates 30fps grayscale image input from 4 cameras equally spaced over the conveyor belt, and computes in real-time the location of each parcel over the belt. The corner points of the tracked parcels are then sent to a controller, which arranges the parcels into a single line at the output. We use Lucas-Kanade-Tomasi (LKT) feature tracking algorithm as the base of our tracking algorithm: Corner points of a parcel from the previous frame are fed into the LKT algorithm to get the new corner coordinates of the parcel in the current frame. Although this approach tracks a parcel for a few frames over the belt, it is not enough for long-term successful tracking of a parcel. To achieve successful parcel tracking, an edge mapping is added as a refinement step following LKT corner tracking. In this paper we detail the design and implementation of our tracking software and show that the proposed algorithms are novel and are able to track parcels in real-time.