Methodological issues on statistical rigor of agreement analysis

With the intention to evaluate a current h(fa)-index which considered the relative contribution of each author, Sebo P and colleagues selected senior hospital physicians from six college hospitals in Switzerland between 1 and 14 March 2020. Using Web of Science, the authors assessed the h(fa)-index of all physicians with acquired non-zero h-index. They applied Cohen’s kappa statistic to calculate the inter-rater agreement between the h-index and the h(fa)-index. We argue that the agreement between the results of the two metrics is disputed by Cohen’s kappa statistic. Therefore, we recommend that an additional appropriate statistical approach, the weighted Cohen’s kappa statistic, should be used in this study.