Continuous Hyperparameter Optimization for Large-scale Recommender Systems

While the prediction accuracy of a large-scale recommender system can generally be improved by learning from more and more training data over time, it is unclear how well a fixed predictive model can handle the changing business dynamics in a real-world scenario. The adjustment of a predictive model is controlled by the hyperparameter settings of a selected algorithm. Although the problem of hyperparameter optimization has been studied for decades in various disciplines, the adaptiveness of the initially selected model is not as well understood. This paper presents an approach to continuously re-select hyperparameter settings of the algorithm in a large-scale retail recommender system. In particular, an automatic hyperparameter optimization technique is applied on collaborative filtering algorithms in order to improve prediction accuracy. Experiments have been conducted on a large-scale real retail dataset to challenge traditional assumption that a one-off initial hyperparameter optimization is sufficient. The proposed approach has been compared with a baseline approach and a widely used approach with two scalable collaborative filtering algorithms. The evaluations of our experiments are based on a 2-year real purchase transaction dataset of a large retail chain business, both its online e-commerce site and its offline retail stores. It is demonstrated that continuous hyperparameter optimization can effectively improve the prediction accuracy of a recommender system. This paper presents a new direction in improving the prediction performance of a large-scale recommender system.