Using item analysis to evaluate the validity and reliability of an existing online information literacy skills assessment instrument

At the end of 2012 a short course, the Certificate of Information Literacy, was registered by the Cape Peninsula University of Technology Libraries and was first introduced in 2013. The course consisted of five modules and a summative assessment. The assessment, accessed via the Learning Management System, comprised 100 multiple-choice questions of which students received a randomly selected fifty to answer in ninety minutes. This article reports on the effectiveness and validity of the online assessment for the Certificate of Information Literacy: whether the assessment instrument accurately measures the information literacy skills of the students and whether the test can be considered valid and reliable. There are two parts to the study, but this paper focuses on the first part which is an item analysis on the assessment data of the students who completed the test in 2013. The results highlight weaknesses and problematic areas in the test. The second part of the study will focus on comparing the results students received in the test with the results of a subject-specific essay assignment to measure the level of application of their information literacy skills.