The Usability of Altmetrics in Academic Evaluation

This article studied the usability of altmetrics in the academic evaluation based on the data from Altmetric.com . This study was carried out in five steps. Firstly we sorted out all the related studies to make exhaustive understanding of the existing methods and conclusions for evaluating the usability of altmetrics. Secondly, the evaluation method was proposed and the dataset was constructed, which consisted of 18 altmetrics indicators of papers from 10 disciplines and published in recent 10 years. Thirdly, for every indicator we studied the distribution of coverage and mean count over time to analyze the indicators' accumulation characteristics and focus on early literature. In the fourth part, the value of every indicator and the redundancy among values were studied. The approaches included correlation analysis between altmetrics indicators and citation, correlation analysis with each other among altmetrics indicators, and PCA among all indicators. In the last, we analyzed some factors that may affect the usability of altmetrics, such as discipline, JIF, country, language and so on. The conclusions of this article are as follows. Some altmetrics indicators just pay attention to newly published papers, while other indicators could have wider coverage. This situation causes different data accumulation process. It' s common to find redundancies among indicators in value. The value reflected by altmetrics is various and not limited to academic value. Besides, the application of altmetrics in academic evaluation must consider discipline, JIF, language, country and so on.