Automatic problem extraction and analysis from unstructured text in IT tickets

IT services are extremely human labor intensive, and a key focus is to provide efficient services at low cost. Automation of repeatable IT tasks using software service agents that reduce human effort is therefore an important component of service management. A large fraction of the work done by IT service personnel involves troubleshooting of problems. However, the complexity of IT systems makes automated problem determination and resolution a challenging research problem. Using a database of prior customer problems and solutions, we build a system that extracts knowledge about different classes of problems arising in the IT infrastructure, mine problem linkages to recent system changes, and identify the resolution activities to mitigate problems. The system, at its core, uses data mining, machine learning, and natural language parsing techniques. By using extracted knowledge, one can (i) understand the kind of problems and the root causes affecting the IT infrastructure, (ii) proactively remediate the causes so that they no longer result in problems, and (iii) estimate the scope for automation for service management. In the future, a large cost differentiator for any IT company will often involve being able to build automated service agents from these technologies, which will result in a reduction in human effort.