Self-attention enhanced CNNs with average margin loss for chinese zero pronoun resolution

Recent neural network methods for Chinese zero pronoun resolution explore multiple models for generating representation vectors for zero pronouns and their candidate antecedents. Typically, the representation of zero pronouns are generated by their contextual information since they are simply gaps, which makes it hard to express them. To better interpret zero pronouns and their candidate antecedents, we here introduce a convolutional neural networks with internal self-attention module for encoding them. With the help of the Multi-hop attention mechanism, our model is able to focus on informative parts of the associated contextual texts, which produces an effective way to capture these important information. In addition, we propose a novel average margin loss by averaging the candidate antecedents scores, making the model learning more reasonable. Experimental results on OntoNotes 5.0 dataset show that our model gains the best performance on the task of Chinese zero pronoun resolution.