Simultaneous Contextualization and Interpretation with Keyword Awareness

Most natural-language-processing methods are designed for estimating context given an entire set of sentences at once. However, dialogue is incremental in nature. SCAIN (Simultaneous Contextualization and Interpretation) is an algorithm for incremental dialogue processing. Along with the progress of the dialogue, it can solve the interdependence problem in which the interpretation of words depends on the context, and the context is determined by the interpreted words. However, SCAIN cannot process texts that contain more words insignificant to context estimation such as in longer texts. We propose SCAIN with keyword extraction (SCAIN/KE), which extracts keywords that contribute to context estimation and eliminates the effect of insignificant words so that it can process longer texts. In the case study, SCAIN/KE updates context and interpretation better than SCAIN and obtains the keywords that contribute to context estimation better than other statistical methods. In the experiments, we evaluated SCAIN/KE on solving the ambiguity of polysemous words using the Wikipedia disambiguation pages. The results indicate that SCAIN/KE is more accurate than SCAIN.