Citation Metrics and Boyer's Model of Scholarship: How Do Bibliometrics and Altmetrics Respond to Research Impact?

This article explores how well bibliometrics and altmetrics reflect research impact in relation to Boyer's Model of the Scholarship. Indices used for both types of metrics are explored and discussed while including an analysis on primary methodological works performed on each in the literature to date. As confirmatory in nature, we chose as our focus of analysis the most highly cited, solo-authored, peer-reviewed article from each of two scholars at an R1 institution in the field of higher education: a rising junior scholar and a well-established senior scholar. In addition to analysis of bibliometric and altmetrics, extensive textual analysis was conducted on the works (n = 165) of both scholars to determine how substantitive in-text citations were from authors citing these scholars' research. Different authors coded each of the citing works, and both inter-rater reliability (k < 0.001, s(?) = 0.055) and intraclass correlation tests (r = .000) were conducted at the nominal and interval levels to provide strong reliability measures of the rubric with each showing a confidence interval of 95% for coding results. In-text citations for dissertations were actually found to be more substantive than articles at a level of significance (p < .05). The vast majority of references were found to have a score of 3 or below, implying impact of the original research was not strong. In relation to Boyer's domains, inference can be made that the majority of research literature associated with citation metrics, either bibliometrics or altmetrics, do not move beyond the Scholarship of Discovery into the Scholarship of Integration.