Investigating Searchers' Mental Models to Inform Search Explanations

Modern web search engines use many signals to select and rank results in response to queries. However, searchers' mental models of search are relatively unsophisticated, hindering their ability to use search engines efficiently and effectively. Annotating results with more in-depth explanations could help, but search engine providers need to know what to explain. To this end, we report on a study of searchers' mental models of web selection and ranking, with more than 400 respondents to an online survey and 11 face-to-face interviews. Participants volunteered a range of factors and showed good understanding of important concepts such as popularity, wording, and personalization. However, they showed little understanding of recency or diversity and incorrect ideas of payment for ranking. Where there are already explanatory annotations on the results page such as "ad" markers and keyword highlighting participants were familiar with ranking concepts. This suggests that further explanatory annotations may be useful.