Evaluating the effect of gamification on the deployment of digital cultural probes for children

This research proposes a digital Cultural Probe (CP) for children in the context of energy use and awareness. A CP is a user research method that allows participants to self-report by means of tools, artefacts, and tasks that they complete at their own pace. Our CP aims to gather data about the energy related habits and knowledge of the children and their families and also aims to provoke reflection and raise their awareness of energy and environmental issues. It is well known that the main drawback of CPs is their low completion rate and the consequent lack of gathered data. Based on the impact of gamification on users' motivation and engagement, we aim to test the hypothesis that "Gamification may encourage an active and fruitful participation of children in Cultural Probes". CP consists of four tasks designed to gather information about the children and their families' electric consumption and saving habits. We developed the CP as an Android app which was evaluated with children from two primary schools in Barcelona, who interacted with two different app versions (gamified and non-gamified). We designed a comparative experiment that gathered data on CP completion rate, CP task completion rate, and the time required to complete the CP. To our surprise, we found a slight difference in favour of the non-gamified app in all of these measures, though this difference was not statistically significant. However, the difference in the children's awareness of and attitude to energy consumption before and after the realisation of the CP was significant as measured by the Wilcoxon signed-rank test. This was not the case for the effect on energy awareness of the children's relatives. The lessons we learned, which may also prove to be useful for researchers that aim to design cultural probes, are that user research methods for children can be effective enough when designed to be interactive and dynamic (i.e. digital CPs) and that the number of CP tasks may also be a factor to take into account. When the number of tasks is low, the CP is likely to achieve a high completion rate and the additional effort of gamifying the application may be not worthwhile. Then, gamification may support a CP with a large number of tedious tasks. When designers want children to reflect on key issues, the CP should be thoughtfully designed because it is not easy to foster children's reflection and reasoning.