Initial assessment of reliability of a self-administered web-based neuropsychological test battery

Introduction: Web-based neuropsychological testing can be an important tool in meeting the increasing demands for neuropsychological assessment in the clinic and in large research studies. The primary aim of this study was to investigate practice effects and reliability of self-administered web-based neuropsychological tests in Memoro. Due to lack of consistent analysis, and reporting of reliability in the literature, especially intraclass correlation coefficients (ICC), we highlight how using different ICC measures results in different estimates of reliability. Method: 61 (31 females) participants (mean age 53.3 years) completed the Memoro tests twice with a median of 14 days between testing. Results: Practice effects were detected for all cognitive measures (d = 0.32-0.61), most pronounced for memory measures. Reliability estimated using two-way random effects single measure absolute agreement ICC(2,1) were between 0.55 and 0.74. Two-way mixed effects average measure consistency ICC(3,2), ranged from 0.79 to 0.89. Reliability was highest for the processing speed task and lower for the memory tasks. Conclusions: Memoro tests had test-retest reliability similar to that of traditional, computerized and web-based test batteries used clinically and in research. It is important to carefully choose and specify the ICC implemented, as ICC(2,1) and ICC(3,2) give different results and reflect reliability of different measures. (C) 2016 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).