Time-Aware Evaluation of Methods for Identifying Active Household Members in Recommender Systems

Online services are usually accessed via household accounts. A household account is typically shared by various users who live in the same house. This represents a problem for providing personalized services, such as recommendation. Identifying the household members who are interacting with an online system (e.g. an on-demand video service) in a given moment, is thus an interesting challenge for the recommender systems research community. Previous work has shown that methods based on the analysis of temporal patterns of users are highly accurate in the above task when they use randomly sampled test data. However, such evaluation methodology may not properly deal with the evolution of the users' preferences and behavior through time. In this paper we evaluate several methods' performance using time-aware evaluation methodologies. Results from our experiments show that the discrimination power of different time features varies considerably, and moreover, the accuracy achieved by the methods can be heavily penalized when using a more realistic evaluation methodology.