A measure of ordinal concordance for the evaluation of University courses

被引:3
|
作者
Marasini, Donata [1 ]
Quatto, Piero [1 ]
Ripamonti, Enrico [1 ]
机构
[1] Univ Milano Bicocca, Stat Sect, Dept Econ Management & Stat, Piazza Ateneo Nuovo 1, I-20126 Milan, Italy
关键词
inter-rater agreement; ordinal measures; satisfaction; AGREEMENT; KAPPA;
D O I
10.1016/S2212-5671(14)00876-4
中图分类号
O1 [数学]; C [社会科学总论];
学科分类号
03 ; 0303 ; 0701 ; 070101 ;
摘要
In this paper we apply the s* statistic, aimed to measure the inter-rater agreement between observers in case of ordinal variables, to the evaluation of the quality of University courses. The objective is to measure the inter-rater agreement between students, along with their satisfaction, in order to verify the consistency of judgments expressed by independent observers. s* is a modification of a previously proposed index, which avoids the problem of paradoxes of Cohen's and Fleiss' kappa statistics. We present the s* index from both a descriptive and an inferential point of view. In particular, as far as statistical inference is concerned, we show that s* is a biased estimator of the inter-rater agreement in the population and, under the null hypothesis of inter-rater agreement by chance, s* is asymptotically normally distributed. (C) 2014 The Authors. Published by Elsevier B.V.This is an open access article under the CC BY- NC- ND license
引用
收藏
页码:39 / 46
页数:8
相关论文
共 50 条
  • [1] THE EVALUATION OF UNIVERSITY COURSES
    MILLER, AH
    [J]. STUDIES IN HIGHER EDUCATION, 1984, 9 (01) : 1 - 15
  • [2] An information theoretic measure for the evaluation of ordinal scale data
    W. J. Tastle
    M. J. Wierman
    [J]. Behavior Research Methods, 2006, 38 : 487 - 494
  • [3] An information theoretic measure for the evaluation of ordinal scale data
    Tastle, W. J.
    Wierman, M. J.
    [J]. BEHAVIOR RESEARCH METHODS, 2006, 38 (03) : 487 - 494
  • [4] An Evaluation of Practicum Courses at Farhangian University
    Masoumpanah, Zahra
    Tahririan, Mohammad Hassan
    Afzali, Katayoon
    Alibabaee, Ahmad
    [J]. IRANIAN JOURNAL OF LANGUAGE TEACHING RESEARCH, 2019, 7 (02) : 101 - 119
  • [5] On diagnostic accuracy measure with cut-points criterion for ordinal disease classification based on concordance and discordance
    Kersey, Jing
    Samawi, Hani
    Yin, Jingjing
    Rochani, Haresh
    Zhang, Xinyan
    [J]. JOURNAL OF APPLIED STATISTICS, 2023, 50 (08) : 1772 - 1789
  • [6] University teaching and evaluation of courses - popular? University teachers' perspective
    Spiel, C
    Wolf, P
    Popper, V
    [J]. ZEITSCHRIFT FUR PSYCHOLOGIE, 2002, 210 (01): : 27 - 39
  • [7] MEASURE OF ORDINAL CONSENSUS
    LEIK, RK
    [J]. PACIFIC SOCIOLOGICAL REVIEW, 1966, 9 (02): : 85 - 90
  • [8] Questionnaires for the evaluation of university courses by students and teachers
    Staufenbiel, T
    [J]. DIAGNOSTICA, 2000, 46 (04): : 169 - 181
  • [9] E-Evaluation of university telematics courses
    Lopez Meneses, Eloy
    Roso Diaz, Jose
    [J]. TEJUELO-DIDACTICA DE LA LENGUA Y LA LITERATURA, 2009, 4 (01): : 19 - 31
  • [10] DESIGN AND EVALUATION OF SCIENCE COURSES AT OPEN UNIVERSITY
    KAYE, AR
    [J]. INSTRUCTIONAL SCIENCE, 1973, 2 (02) : 119 - 191