Shared Augmented Reality Experience Between a Microsoft Flight Simulator User and a User in the Real World

Our demo consists of an application that allows a user with an augmented reality (AR) display to watch another user, flying an airplane in the Microsoft Flight Simulator 2020 (MSFS), at their respective location in the real world. To do that, we take the location data of a virtual 3D airplane model in a virtual representation of the world of a user playing MSFS, and stream it via a server to a mobile AR device. The mobile device user can then see the same 3D airplane model at exactly that real world location, that corresponds to the location of the virtual 3D airplane model in the virtual representation of the world. The mobile device user can also see the plane movement updated according to the 3D airplane movement in the virtual world. We implemented the application on both a cellphone and a see-through headset.