A day without a search engine: an experimental study of online and offline searches

With the evolution of the Web and development of web-based search engines, online searching has become a common method for obtaining information. Given this popularity, the question arises as to how much time people save by using search engines for their information needs compared to offline sources, as well as how online searching affects both search experiences and search outcomes. Using a random sample of queries from a major search engine and a sample of reference questions from the Internet Public Library (IPL), we conduct a real-effort experiment to compare online and offline search experiences and outcomes. We find that participants are significantly more likely to find an answer on the Web (100 %), compared to offline searching (between 87 % and 90 %). Restricting our analysis to the set of questions in which participants find answers in both treatments, a Web search takes on average 7 (9) minutes, whereas the corresponding offline search takes 22 (19) minutes for a search-engine (IPL) question. Furthermore, while raters judge library sources to be significantly more trustworthy and authoritative than the corresponding Web sources, they judge Web sources as significantly more relevant. Balancing all factors, we find that the overall source quality is not significantly different between the two treatments for the set of search-engine questions. However, for IPL questions, we find that non-Web sources are judged to have significantly higher overall quality than the corresponding Web sources. In comparison, for factual questions, Web search results are significantly more likely to be correct (66 % vs. 43 %). Lastly, post-search questionnaires reveal that participants find online searching more enjoyable than offline searching.