QRNN-Transformer: Recognizing Textual Entailment

In recent years, the Transformer model based on the self-attention mechanism has made significant progress in natural language processing and has also been applied in the text implication recognition task, achieving excellent results. However, the Transformer model still has deficiencies in modeling local information in the text. To improve the Transformer model, the QRNN-Transformer was proposed, which uses the QRNN network to divide the input text sequence into local short sequences to capture the local information of the input text. The self-attention is improved by combining the gating mechanism to make the model select tasks-related words or features. Extensive experiments have demonstrated the QRNN-Transformer model can effectively improve the accuracy of entailment relationship recognition on both English and Chinese datasets.(1)