Soccer player recognition using spatial constellation features and jersey number recognition

Identifying players in soccer videos is a challenging task, especially in overview shots. Face recognition is not feasible due to low resolution, and jersey number recognition suffers from low resolution, motion blur and unsuitable player pose. Therefore, a method to improve visual identification using spatial constellations is proposed here. This method models a spatial constellation as a histogram over relative positions among all players of the team. Using constellation features might increase identification performance but is not expected to work well as a single mean of identification. Thus, this constellation-based recognition is combined with jersey number recognition using convolutional neural networks. Recognizing the numbers on a player's shirt is the most straight-forward approach, as there is a direct mapping between numbers and players. Using spatial constellation as a feature for identification is based on the assumption that players do not move randomly over the pitch. Players rather follow a tactical role such as central defender, winger, forward, etc. However in soccer, players do not strictly adhere to these roles, variations occur more or less frequently. By learning constellation models for each player, we avoid a categorical assignment of a player to one single tactical role and therefore incorporate each player's typical behaviour in terms of switching positions. The presented player identification process is expressed as an assignment problem. Here, an optimal assignment of manually labeled trajectories to known player identities is calculated. Using an assignment problem allows for a flexible fusion of constellation features and jersey numbers by combining the respective cost matrices. Evaluation is performed on 14 different shots of six different Bundesliga matches. By combining both modalities, the accuracy is improved from 0.69 to 0.82 when compared with jersey number recognition only. (C) 2017 Published by Elsevier Inc.