Graph-based algorithms for ranking researchers: not all swans are white!

Scientific importance ranking has long been an important research topic in scientometrics. Many indices based on citation counts have been proposed. In recent years, several graph-based ranking algorithms have been studied and claimed to be reasonable and effective. However, most current researches fall short of a concrete view of what these graph-based ranking algorithms bring to bibliometric analysis. In this paper, we make a comparative study of state-of-the-art graph-based algorithms using the APS (American Physical Society) dataset. We focus on ranking researchers. Some interesting findings are made. Firstly, simple citation-based indices like citation count can return surprisingly better results than many cutting-edge graph-based ranking algorithms. Secondly, how we define researcher importance may have tremendous impacts on ranking performance. Thirdly, some ranking methods which at the first glance are totally different have high rank correlations. Finally, the data of which time period are chosen for ranking greatly influence ranking performance but still remains open for further study. We also try to give explanations to a large part of the above findings. The results of this study open a third eye on the current research status of bibliometric analysis.