VR and GUI based Human-Robot Interaction Behavior Collection for Modeling the Subjective Evaluation of the Interaction Quality

Formulating the evaluation criteria for the quality of human-robot interaction (QoHRI) is an important research topic for improving the functions of dialogue systems and interactive social robots. An ideal method for QoHRI evaluation is the subjective evaluation using questionnaires with various human evaluators. However, it is time-consuming and inapplicable for the scoring method for robot competitions and autonomous learning by interactive robots; hence we focus on a data-driven approach that models the evaluation criterion by approximating the subjective evaluation results based on the HRI behavior data. Our first research question is: How can we collect a wide variety of interaction behavior data that include both good- and bad-quality interactions? To collect various interaction data while moderating the QoHRI, we propose a VR and GUI-based interaction generator in which humans and robots can interact with each other, which is the first contribution of this study. To investigate whether the proposed system can cover a wide variety of interactions, we introduce a metric of interaction datasets coverage from the perspective of the subjective evaluation approximation of QoHRI. We validated the usefulness of the proposed system by comparing three datasets in a robot competition domain, which is the second contribution of this study.