Mixed Reality Serious Games and Gamification for smart education

The appeal of Mixed Reality (MR) digital games arouses interest among researchers and education specialists who since their recent proliferation, they have been trying to introduce their motivating potential in learning contexts. Our main research question in this work focuses on whether MR digital games can, via novel Presence (feeling of 'being and doing there' in a virtual or augmented world) and MR gamification (dynamics, mechanics, components) support and foster future learning and teaching, to address a wide variety and variation of educational contexts. To accomplish the above, a consistent computational framework that supports new types of Mixed-Reality Serious Games and Gamification (MRSGs) is established in this work that features MR gesture-based and game-based learning. The introduced term Mixed Reality Serious Games (MRSGs), refers to digital mini game-shells that allow the learners and teachers to sense the feeling of 'Presence' experienced under a novel MR educational learning framework, in both Virtual Reality (VR) as well as Augmented Reality (AR) formal and informal learning. The former (VR) allows for the unique feeling of 'being there' and 'doing there' in the virtual world, that will be transforming the overall game-based learning experience, via latest innovations as well as recent progress in low-cost h/w Head Mounted Displays (HMDs). The latter (AR) blends real and virtual elements so that the 3D virtual element is registered accurately in the real world and interacted freely by the learner via various mobile displays, including smart glasses, natural, gesture-based interaction (mobile RGB and RGB-D), MR virtual characters and gamified learning processes. Game-based learning also involves the incorporation of games into lessons. The principal aim of applying games in education is to increase students' engagement and motivation. In our case studies we provide two mini MRSGs in VR and AR that accompany the primary school history class and particular the period of the Minoan Civilization, as manifested by the archaeological site of Ancient Knossos in Heraklion, Greece. The AR MRSG implements a desktop-based holographic application using the Meta-AR glasses. This MRSG consist of three mini game shells in which the student has to complete various learning tasks by using gesture-based interaction. These tasks consist of puzzle-based and constructional games tasks. The VR MRSG implements a similar approach of a Virtual tour in Knossos with quests that student need to accomplish to collect rewards. The student has to fully explore the palace, interact with special characters and complete their quests. These MRSGs are a first attempt to formally study latest h/w and s/w advances in Mixed Reality technologies, applied in gesture-based and game-based learning in both formal and informal educational contexts. Moreover we compared the gamification elements for each enabling technology and define the gamification dynamics, mechanics and components that need to be utilized in each MR environment.