A systematic literature review of the Design Critique method

Context: The Design Critique (DC) method is becoming more common in Human-Computer Interaction (HCI) and User Experience (UX) studies as the need for new evaluation methods of emerging technologies is increasing. However, there is an clear lack of guidelines on how to conduct DC studies in the UX context.Objective: The goal of this paper is to provide an overview of the DC method in the fields of UX. In addition, this paper aims to propose a generic process of running DC studies in the same context.Methods: We present a systematic literature review of the DC method. Moreover, we conduct a course of thematic analysis on the selected papers to identify the various DC processes and explore the following attributes: participant categories, data collection methods, and data analysis methods in each process.Results: We identified three different trends of DC processes: detailed, moderate and minimal. In addition, we proposed a generic DC process consisting of 10 steps divided into three main phases: preparation, conducting design critique, and pro-processing. We found that domain experts represent the majority of studies participants. Using interviews to collect qualitative data and using script coding analysis are the two most common methods of collecting and analyzing data.Conclusion: Conducting DC studies can improve overall systems usability by addressing design flaws at an early stage of development. The process of conducting a DC varies, depending on the project goals and states. The DC method aligns well with the small light-weight steps approach in Agile methods.