Home-based Immersive Web Rehabilitation Gaming with Audiovisual Sensors

Early, intensive, and repetitive physical rehabilitation is critical, but it can be difficult to keep patients motivated and engaged. The use of games in immersive reality makes physiotherapy more enjoyable and engaging. During play, many unpleasant emotions can occur as a result of interacting with the joyful activities of the game. This difficulty can be overcome by adjusting game features to the players' emotions and body gestures. This work contributes to the Sensor Enabled Affective Computing for Enhancing Medical Care (SenseCare) project for remote home healthcare applications, and its related SenseCare KM-EP (Knowledge Management-Ecosystem Portal) Affective Computing (AC) platform. In this paper, we propose web-based Augmented Reality (AR) and Virtual Reality (VR) games for home rehabilitation deploying audiovisual sensors (e.g., Camera, Microphone, Kinect), to monitor the patient's emotional well-being and calculate body estimation gestures during the gameplay session, with the aim of giving a quick return to the therapist. In parallel with SenseCare's methods for audiovisual monitoring of gamers, methods for using VR and AR web-based games using the WebXR API are also introduced, and customer satisfaction was determined through an online survey, showing an 81.8% satisfaction rate.