Reliability Estimates for IRT-Based Forced-Choice Assessment Scores

被引:8
|
作者
Lin, Yin [1 ]
机构
[1] SHL, Thames Ditton, Surrey, England
关键词
Thurstonian; IRT; forced choice; reliability; test-retest; LIKERT SCALE; PERSONALITY; PERFORMANCE; SELECTION; FAKING; MODEL;
D O I
10.1177/1094428121999086
中图分类号
B849 [应用心理学];
学科分类号
040203 ;
摘要
Forced-choice (FC) assessments of noncognitive psychological constructs (e.g., personality, behavioral tendencies) are popular in high-stakes organizational testing scenarios (e.g., informing hiring decisions) due to their enhanced resistance against response distortions (e.g., faking good, impression management). The measurement precisions of FC assessment scores used to inform personnel decisions are of paramount importance in practice. Different types of reliability estimates are reported for FC assessment scores in current publications, while consensus on best practices appears to be lacking. In order to provide understanding and structure around the reporting of FC reliability, this study systematically examined different types of reliability estimation methods for Thurstonian IRT-based FC assessment scores: their theoretical differences were discussed, and their numerical differences were illustrated through a series of simulations and empirical studies. In doing so, this study provides a practical guide for appraising different reliability estimation methods for IRT-based FC assessment scores.
引用
收藏
页码:575 / 590
页数:16
相关论文
共 50 条
  • [1] Traditional scores versus IRT estimates on forced-choice tests based on a dominance model
    Hontangas, Pedro M.
    Leenen, Iwin
    de la Torre, Jimmy
    Ponsoda, Vicente
    Morillo, Daniel
    Abad, Francisco J.
    [J]. PSICOTHEMA, 2016, 28 (01) : 76 - 82
  • [2] AN IRT-BASED ASSESSMENT OF PACSLAC
    van Nispen, S.
    Candel, M. J.
    Zwakhalen, S.
    Hamers, J.
    Curfs, L. M.
    Berger, M. P.
    [J]. GERONTOLOGIST, 2009, 49 : 224 - 224
  • [3] Comparing Traditional and IRT Scoring of Forced-Choice Tests
    Hontangas, Pedro M.
    de la Torre, Jimmy
    Ponsoda, Vicente
    Leenen, Iwin
    Morillo, Daniel
    Abad, Francisco J.
    [J]. APPLIED PSYCHOLOGICAL MEASUREMENT, 2015, 39 (08) : 598 - 612
  • [4] The effects of computer administration on scores and item parameter estimates of an IRT-based licensure examination
    Sykes, RC
    Ito, K
    [J]. APPLIED PSYCHOLOGICAL MEASUREMENT, 1997, 21 (01) : 51 - 63
  • [5] ARTIFACTUAL RELIABILITY OF FORCED-CHOICE SCALES
    TENOPYR, ML
    [J]. JOURNAL OF APPLIED PSYCHOLOGY, 1988, 73 (04) : 749 - 751
  • [6] Prophecy formulas for assessing the reliability of IRT-based abilities
    Raju, N
    Oshima, TC
    [J]. INTERNATIONAL JOURNAL OF PSYCHOLOGY, 2004, 39 (5-6) : 462 - 462
  • [7] Fitting a Thurstonian IRT model to forced-choice data using Mplus
    Brown, Anna
    Maydeu-Olivares, Alberto
    [J]. BEHAVIOR RESEARCH METHODS, 2012, 44 (04) : 1135 - 1147
  • [8] Effects of Applicant Faking on Forced-Choice and Likert Scores
    Pavlov, Goran
    Maydeu-Olivares, Alberto
    Fairchild, Amanda J.
    [J]. ORGANIZATIONAL RESEARCH METHODS, 2019, 22 (03) : 710 - 739
  • [9] Contributions to Constructing Forced-Choice Questionnaires Using the Thurstonian IRT Model
    Sun, Luning
    Qin, Zijie
    Wang, Shan
    Tian, Xuetao
    Luo, Fang
    [J]. MULTIVARIATE BEHAVIORAL RESEARCH, 2024, 59 (02) : 229 - 250
  • [10] Fitting a Thurstonian IRT model to forced-choice data using Mplus
    Anna Brown
    Alberto Maydeu-Olivares
    [J]. Behavior Research Methods, 2012, 44 : 1135 - 1147