Natural language processing in support of decision-making: phrases and part-of-speech tagging

The use of natural language information can improve decision-making. Darwinian considerations suggest that language may have developed because it leads to improved decision-making and survival, justifying the study of language's contribution to decision-making. The study of information-based decision-making within the context of evolution provides a view of information use that allows us to both describe the phenomenon of information use as well as to explain why an information use occurs as it does. Increasing information retrieval performance using phrases and part-of-speech (POS) information is one example of a type of decision-making performance that is improved when using this linguistic information. By studying a set of phrases used in a text retrieval system, we are able to show the relative effectiveness of using multi-term phrases as opposed to individual terms, as well as the relative worth of POS tagged terms or phrases, as opposed to untagged terms or phrases. An explanation is suggested for why POS tags contribute less to higher order grammatical constructs. We propose a measure of those needs for POS disambiguation that can be addressed by tagging; some example terms are analyzed using this measure, and specific degrees of ambiguity are proposed. (C) 2001 Elsevier Science Ltd. All rights reserved.