Feature learning and encoding for multi-script writer identification

Writer identification from handwriting samples has been an interesting research problem for the pattern recognition community in general and handwriting recognition community in particular. In most cases, however, it is assumed that writers produce writing samples in a single script only. A more challenging scenario is the multi-script writer identification where the training and test samples of writers belong to different scripts. This paper presents a deep learning-based solution for writer identification in a multi-script scenario. The technique relies on identifying keypoints in handwriting and extracting small patches around these keypoints. These patches are aimed to capture the writing gestures of individuals which are likely to be common across multiple scripts. Robust feature representations are learned from these patches using a deep convolutional neural network and the features are encoded using a newly proposed variant of the Vector of Locally Aggregated Descriptors (VLAD). Experiments on three bilingual handwriting datasets including writing samples in Arabic, English, French, Chinese and Farsi report promising identification rates and significantly outperform the current state-of-the-art on this problem.