Object tracking plays an important role in many intelligent transportation systems. Unfortunately, it remains a challenging task due to factors such as occlusion and target-appearance variation. In this paper, we present a new tracking algorithm to tackle the difficulties caused by these two factors. First, considering the target-appearance variation, we introduce the local-background-weighted histogram (LBWH) to describe-the target. In our LBWH, the local background is treated as the context of the target representation. Compared with traditional descriptors, the LBWH is more robust to the variability or the clutter of the potential background. Second, to deal with the occlusion case, a new forward-backward mean-shift (FBMS) algorithm is proposed by incorporating a forward-backward evaluation scheme, in which the tracking result is evaluated by the forward-backward error. Extensive experiments on various scenarios have demonstrated that our tracking algorithm outperforms the state-of-the-art approaches in tracking accuracy.