Issues in Evaluating Ambient Displays in the Wild: Two Case Studies

In this paper we discus the complex task of evaluating ambient displays, concentrating on issues within in-situ deployments. We start by describing how these technologies have been evaluated in lab settings, where the focus has been primarily on issues of usability, and argue strongly for the necessity of in-situ evaluation. We then present two case studies involving in-situ evaluations, and from these derive issues that hindered the researchers from being able to delve more deeply into the overall impact of their implementations. We conclude with our own suggestions on possible alternatives to explore for evaluating ambient displays, which are based on the issues derived from our case studies.