A Comparative Framework to Evaluate Recommender Systems in Technology Enhanced Learning: a Case Study

When proposing a novel recommender system, one difficult part is its evaluation. Especially in Technology Enhanced Learning (TEL), this phase is critical because those systems influence students or educators in educational tasks. Our research aims to propose a framework for conducting comparative experiments of different recommender systems in a same educational context. The framework is expected to provide the accuracy of subject systems within a single experiment, depicting the benefits of a novel system against others. We also present an application of such framework for a comparative experiment of popular systems in TEL like Google, Slideshare, Youtube, MERLOT, Connexions and ARIADNE. Our results show that the proposed framework has been effective in comparing the accuracy of those systems, with a clear picture of their performance compared one another. Moreover, the results of the experiment can be used as a benchmark when evaluating novel recommender systems in TEL.