Development and expand of 3D rehabilitation system using Mixed Reality technology

Patients often refuse desk training and test of cognitive function that hasn't changed since about 50 years ago. On the other hand, the tasks using Mixed Reality (MR) have the advantage and maintain patient's motivation easily. We have been developed cognitive rehabilitation tasks using MR in collaboration with Kansai Medical University. At present, we develop a novel rehabilitation system by using Microsoft HoloLens2, based on previous experience and knowledge. We created "numbers cancellation task" that imitated the Trail Making Test, "flower path task" for dual tasks, and "search task" based on daily life. We devised a select method for many clinical situations. This system enables us to move the cursor by head tracking or eye tracking, and we can select the objects by tapping or gazing. Additionally, there are also two-mode, "touch" and "pinch", as the hand tracking function. Therapists can confirm the patient's first-person view (real space and virtual reality object) on the PC in real-time. Thus, when they are performing the task, it is possible to understand whether or not the patients are correctly instructed by advice. © 2021, Japan Soc. of Med. Electronics and Biol. Engineering. All rights reserved.