Stacked Multi-head Attention for Multi-turn Response Selection in Retrieval-based Chatbots

In the field of deep learning, response selection is the key to retrieval-based chatbots. Faced with the challenge of contextual meaning comprehension and semantic matching, we propose matching a response through diverse information by stacked multi-head attention. First, we construct multi-granular representations on embedded input sentences by stacked multihead attention. Based on these representations, we build two different matching matrices for context-response segment pairs with another multi-head attention stack. Next, we use a two layer CNN to extract hidden information from the matching matrices. The results are matching scores which measure the correlation among every context and its candidate responses. According to the matching scores, one or more proper responses will be chosen from candidate responses. Multi-head attention reinforces the model's ability to focus on different positions, as experiment on the Ubuntu Dialogue Corpus vi and the Douban Conversation Corpus show that our model is superior to the baseline model.