Tracking of moving multi-object is an important problem on computer vision and the key lies in the correct correspondence of these targets. This paper proposes a new tracking algorithm. Based on the correct recognition the tracking procedure adopts adaptive template matching algorithm in tracking window. And then the paper presents an integration emendation in order to locate missing targets and find the other targets position. The algorithm improves the speed and robust of tracking algorithm. And finally this paper gives the analysis of the experiment results of adaptive template matching tracking algorithm.