Measuring Scholarly Impact Using Modern Citation-Based Indices

Scholarly impact is studied frequently and used to make consequential decisions (e.g., hiring, tenure, promotion, research support, professional honors), and therefore it is important to measure it accurately. Developments in information technology and statistical methods provide promising new metrics to complement traditional information sources (e.g., peer reviews). The introduction of Hirsch's (2005) h index-the largest number h such that at least h articles are cited h times each, or the length of the largest square in a citations x articles array-sparked an explosion in research on the measurement of scholarly impact. We evaluate 22 metrics, including conventional measures, the h index, and many variations on the h theme. Our criteria encompass conceptual, empirical, and practical issues: ease of understanding, accuracy of calculation, effects on incentives, influence of extreme scores, and validity. Although the number of publications fares well on several criteria, the most attractive measures include h, several variations that credit citations outside the h square, and two variations that control for career stage. Additional data suggest that adjustments for self-citations or shared authorship probably would not improve thesemeasures much, if at all. We close by considering which measures are most suitable for research and practical applications.