An examination of four user-based software evaluation methods

Usability has become an important factor in the design of interactive software. In the quest to develop usable software, user participation in the design and evaluation process has been advocated. Four methods; logged data, questionnaires, interviews, and verbal protocol analyses have been the prominent base evaluation methods used, sometimes being combined in an attempt to provide a comprehensive evaluation. However, little is known about the strengths and weaknesses of these methods. In this study, these four methods were used to evaluate three different software types (spreadsheet, word processor, and database) with 148 subjects participating. The data obtained enabled a number of conclusions to be drawn regarding the usefulness of the methods when used individually and in combination.