Learning to predict problematic situations in a spoken dialogue system: Experiments with how may I help you?

Current spoken dialogue systems are deficient in their strategies for preventing, identifying and repairing problems that arise in the conversation. This paper reports results on learning to automatically identify and predict problematic human-computer dialogues in a corpus of 4774 dialogues collected with the How May I Help You spoken dialogue system. Our expectation is that the ability to predict problematic dialogues will allow the system's dialogue manager to modify its behavior to repair problems, and even perhaps, to prevent them. We train a problematic dialogue classifier using automatically-obtainable features that can identify problematic dialogues significantly better (23%) than the baseline. A classifier trained with only automatic features from the first exchange in the dialogue can predict problematic dialogues 7% more accurately than the baseline, and one trained with automatic features from the first two exchanges can perform 14% better than the baseline.