Sketch Recognition for Interactive Game Experiences Using Neural Networks

Human freehand sketches can provide various scenarios to the interfaces with their intuitive, illustrative, and abstract nature. Although freehand sketches have been powerful tools for communication and have been studied in different contexts, their capacity to create compelling interactions in games is still under-explored. In this study, we present a new game based on sketch recognition. Specifically, we train various neural networks (Recurrent Neural Networks and Convolutional Neural Networks) and use different classification algorithms (Support Vector Machines and k-Nearest Neighbors) on sketches to create an interactive game interface where the player can contribute to the game by drawing. To measure usability, technology acceptance, immersion, and playfulness aspects, 18 participants played the game and answered the questionnaires composed of four different scales. Technical results and user tests demonstrate the capability and potential of sketch integration as a communication tool to construct an effective and responsive visual medium for novel interactive game experiences.