Malaysian Smart School courseware usability study: The effectiveness of analytical evaluation technique compared to empirical study

Usability evaluation methods are used to evaluate the interaction of the human with the computer for the purpose of identifying aspects of interaction that could be improved t increase usability. This research was carried out to determine the effectiveness of analytical evaluation technique that was designed, developed and named as Jalan Rentasan Kognitif (JRK) evaluation technique. This evaluation technique was conducted on thirty-seven surrogate users that consist of courseware developers, teachers and university students using the Malaysian Smart School Mathematics courseware. The usability result was then compared statistically with the empirical usability evaluation called Task Analysis Exploratory through Observation using video camera (TAEO). The respondents of the empirical method were secondary school students who used the same courseware as one of the teaming aids in school. Usability indicators used in the study were usability problems and users' satisfaction. The study showed that the effectiveness of the two techniques was quite similar if they were used independently during the evaluation process. The usability indicators obtained in the study could be used as references in the usability comparisons of other educational courseware.