A 5-Step Framework for Measuring the Quality of Experience (QoE) of Internet of Things (IoT) Services

The ultimate benefit of the Internet of Things (IoT) is to improve quality of life by supporting the automation and interconnection of various services; this interconnection includes public facilities in platforms such as smart cities. IoT implementation is believed to improve services and yield additional benefits. However, assessment of how the IoT brings such an experience to users is still a challenge that is not yet standardized. Given that little research has discussed this issue, this paper aims to formulate a framework for measuring the quality of experience (QoE) regarding IoT services on the basis of the Absolute Category Rating with Hidden Reference (ACR-HR) scale, which measures the perception of experience by having the IoT users provide scores for their experiences before and after IoT implementation. The proposed framework consists of 5 steps: setting up the focus of the IoT services and the QoE parameters being assessed, defining users of IoT services, conducting a mean opinion score (MOS) survey of the users, calculating the DMOS (differential MOS) on the basis of the ACR-HR quantitative scale, and finally providing the strategic implications to those who implement the service. To test the proposed framework, we have conducted a proof of concept by measuring the QoE for the IoT services in the Jakarta Smart City platform. Six institutional users operate the IoT for their public services, ranging from fire and rescue brigades to public bus transportation. The results imply that the IoT-based services provide a good experience to institutional users when compared to the state of services without IoT. A strategic implication is that some features of Jakarta Smart City must be evaluated and improved to increase the level of satisfaction of users. To conclude, our framework is sufficiently reliable in measuring the QoE for IoT-based services and can be recommended for use for any other IoT-type service.