User Evaluations on Sentiment-based Recommendation Explanations

The explanation interface has been recognized as important in recommender systems because it can allow users to better judge the relevance of recommendations to their preferences and, hence, make more informed decisions. In different product domains, the specific purpose of explanation can be different. For high-investment products (e.g., digital cameras, laptops), how to educate the typical type of new buyers about product knowledge and, consequently, improve their preference certainty and decision quality is essentially crucial. With this objective, we have developed a novel tradeoff-oriented explanation interface that particularly takes into account sentiment features as extracted from product reviews to generate recommendations and explanations in a category structure. In this manuscript, we first reported the results of an earlier user study (in both before-after and counter-balancing setups) that compared our prototype system with the traditional one that purely considers static specifications for explanations. This experiment revealed that adding sentiment-based explanations can significantly increase users' product knowledge, preference certainty, perceived information usefulness, perceived recommendation transparency and quality, and purchase intention. In order to further identify the reason behind users' perception improvements on the sentiment-based explanation interface, we performed a follow-up lab controlled eye-tracking experiment that investigated how users viewed information and compared products on the interface. This study shows that incorporating sentiment features into the tradeoff-oriented explanations can significantly affect users' eye-gaze pattern. They were stimulated to not only notice bottom categories of products, but also, more frequently, to compare products across categories. The results also disclose users' inherent information needs for sentiment-based explanations, as they allow users to better understand the recommended products and gain more knowledge about static specifications.