At present, most of the existing multi-object tracking algorithms use the tracking-by-detection structure. On the one hand, these methods can not make full use of the intermediate features of the detector, on the other hand, the way to solve the similarity does not take into account the correlation between objects. At the same time, the existing multi-object tracking methods do not deal with the occluded object features. Based on the above problems, this paper proposes an end-to-end multi-object tracking algorithm, which uses the object deep features transmitted by the detector to directly generate the incidence matrix through the end-to-end association network; At the same time, considering the interference in occlusion, the self attention mechanism is used to enhance the features of the object. In terms of association strategy, this paper uses Hungarian matching algorithm to associate according to the association matrix. The algorithm has carried out a large number of experiments on KITTI data set, achieved 51.80% HOTA (high-order tracking accuracy) and 53.77% MOTA (multi-object tracking accuracy), and achieved considerable results compared with some existing mainstream methods.