Machine learning for question answering from tabular data

Question Answering (QA) systems automatically answer natural language questions in a human-like manner One of the practical approaches to open domain QA consists in extracting facts from free text offline and using a lookup mechanism when answering user's questions online. This approach is related to natural language interfaces to databases (NLIDBs) that were studied extensively from the 1970s to the 1990s. NLIDB systems employed a range of techniques, from simple pattern-matching rules to formal logical calculi such as the lambda calculus, but most were restricted to specific domains. In this paper we describe a machine learning approach to querying tabular data for QA which is not restricted to specific domains. Our approach consists of two steps: for an incoming question, we first use a classifier to identify appropriate tables and columns in a structured database, and then employ a free-text retrieval to look up answers. The system uses part-of-speech tagging, named-entity normalization and a statistical classifier trained on data from the TREC QA task. With the TREC QA data, our system is shown to significantly outperform an existing rule-based table lookup method.