A hybrid statistical and deep learning based technique for Persian part of speech tagging

In part of speech (POS) tagging, the main challenge is to predict the right tags for both in-vocabulary (IV) and out-of-vocabulary (OOV) words. Therefore, artificial neural networks, such as multi-layer perceptron (MLP) and long short term memory (LSTM), which seem to be efficient because of their high generality capability, have been applied to POS tagging to overcome this challenge. In this research, using word vectors as the input of MLP and LSTM neural networks, we do POS tagging in Persian language and compare the results of the neural models with a second-order hidden Markov model (HMM) which in fact is our benchmark. To investigate the effect of the number of hidden layers, we use both a single-layer and a two-layer MLP and LSTM neural network. Also, we have applied a bidirectional LSTM neural network to investigate the effect of a bidirectional learning algorithm on Persian POS tagging. The results obtained from different models in this research show that neural models have a far better performance in predicting the correct POS tags for OOV words, which can be due to their higher generality. Therefore, we have proposed a hybrid model which is a combination of the HMM and a single-layer bidirectional LSTM model as an innovative model in POS tagging. This hybrid model is successful in improving both HMM and neural models, increasing the accuracy to 97.29%.