CASE-STUDY OF USING DIFFERENT EVALUATION TECHNIQUES IN SMALL COMPANY

A number of usability evaluation techniques were compared in an evaluation of seven interface prototypes. A detailed real-time log of the interaction was made and video and audio records of the interactions kept. Additionally, a novel experience elicitation technique, based on knowledge elicitation techniques, was used. Keystroke level analysis and BNF grammar analyses were also prepared. The study enabled a comparison of the different methods to be made and recommendations as to their use drawn up. The case study was conducted on the premises of a small, UK-based software company, Brameur Ltd. The company employs around 10 full-time staff and a number of part-time staff. Six of the full-time staff were involved in the evaluation case study. Some important issues that the case study seeks to address concern not only the adequacy and appropriateness of the techniques in terms of how well they reflect the usability of the test interfaces and software products in general, but also the cost of using them in terms of time, specialist equipment and expertise, disruption to normal work activities, and so on. Suggestions are made about how to make some of the techniques more usable within the 'small company' scenario. An important conclusion from the study is that a combination of techniques most truly reveals the real state of usability of the interfaces.