Query Evaluation for Suitable Search Engine Selection

Web Search Engines (SEs) have gained much attention in the last decade. More especially, traditional SEs such as Google, aids web searchers extensively in federated search, as a preliminary tool to locate the right vertical (specialized SEs like in the case of travel-sites) in order to launch their queries. The retrieval and ranking approaches differ across SEs and besides that, the basic underlying knowledge about each SE is not sufficient enough to make the best SE s election for different query categories. Therefore, how do searchers identify the most suitable SE that optimizes their query needs? In this paper, we evaluate three famous SEs from a front-end perspective using 7 sub-categories query-sets from BaseQuery and RelationalQuery. We apply the Normalized Discounted Cumulative Gain (NDCG), Mean Reciprocal Rank (MRR) and our proposed Robustness metric to assess the SEs. Using a 2 tailed t-test, we show that there is a significant d ifference a mong t he SEs i n 5 q uery categories, in terms of relevant retrieval and ranking.