Data-Driven Development and Evaluation of Enskill English

Cloud computing offers developers of learning environments access to unprecedented amounts of learner data. This makes possible data-driven development (D3) of learning environments. In the D3 approach the learning environment is a data collection tool as well a learning tool. It continually collects data from interactions with learners, which is used in ongoing evaluation and iterative development. Iterative development cycles become very rapid, limited by the time required to analyze data and deploy system updates. D3 is particularly relevant to fielded AIED systems that operate in uncontrolled conditions, where learners may behave in unexpected ways. This article presents two snapshot case studies in the data-driven development of Enskill® English, a system for learning to speak English as a foreign language. In the first trial at the University of Novi Sad in Serbia two versions of Enskill English’s dialogue system were tested simultaneously: the released version and a new version incorporating statistical natural language processing technology. A new version was released and data were collected from a second snapshot evaluation at the University of Split, Croatia. Data from learners in Latin America and Europe were analyzed for comparison. The evaluations provided preliminary evidence that Enskill English is helpful for learning spoken English skills, and leading indicators that learner performance improves through practice with Enskill English. They suggest that Enskill English can be extended to meet the needs of more advanced learners who wish to use English in a professional context. Broader recommendations for data-driven development of intelligent learning environments are presented.